2024-04-26T19:42:33.764 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T19:42:33.765 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139 --verbose --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/orig.config.yaml 2024-04-26T19:42:33.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T19:42:33.913 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} email: null first_in_suite: false job_id: '7675139' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBEzGRU0dTkd9eWG3RAlBbi2rnXbO0iiOWKWLuT0Meeh6hPZW84FBM5W4f2LbVYu4lntI/ulpt4tRizukPCYg8E= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJa7XxPFaYvWi5b57ZvtKtbXU4jp98AtFTbUQrRkJVNZLsTUJw5q1OPeE+acWtRoleTDEBpK5qxzBbwqPudc+AE= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFpMOwrbJaC0lv5AsyTUC35QLqTW2sqkGJJrDtx+sss6b2zwyCLOI0yAq04x4haoWMpe/WDBFKIcUul/H+LuBw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T19:42:33.913 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T19:42:33.913 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T19:42:33.914 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T19:42:33.915 INFO:teuthology.task.internal:Checking packages... 2024-04-26T19:42:33.936 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T19:42:33.937 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T19:42:33.937 INFO:teuthology.packaging:ref: None 2024-04-26T19:42:33.937 INFO:teuthology.packaging:tag: None 2024-04-26T19:42:33.937 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T19:42:33.937 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:42:33.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T19:42:34.072 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T19:42:34.073 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T19:42:34.080 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T19:42:34.080 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T19:42:34.087 INFO:teuthology.task.internal:Saving configuration 2024-04-26T19:42:34.107 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T19:42:34.113 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T19:42:34.139 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:50.039084', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBEzGRU0dTkd9eWG3RAlBbi2rnXbO0iiOWKWLuT0Meeh6hPZW84FBM5W4f2LbVYu4lntI/ulpt4tRizukPCYg8E='} 2024-04-26T19:42:34.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:50.040767', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJa7XxPFaYvWi5b57ZvtKtbXU4jp98AtFTbUQrRkJVNZLsTUJw5q1OPeE+acWtRoleTDEBpK5qxzBbwqPudc+AE='} 2024-04-26T19:42:34.184 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:50.037291', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFpMOwrbJaC0lv5AsyTUC35QLqTW2sqkGJJrDtx+sss6b2zwyCLOI0yAq04x4haoWMpe/WDBFKIcUul/H+LuBw='} 2024-04-26T19:42:34.184 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T19:42:34.190 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T19:42:34.190 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T19:42:34.191 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T19:42:34.191 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T19:42:34.288 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbfcffd6160>, signals=[15]) 2024-04-26T19:42:34.288 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T19:42:34.294 INFO:teuthology.task.internal:Opening connections... 2024-04-26T19:42:34.294 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T19:42:34.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:42:34.374 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-04-26T19:42:34.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:42:34.447 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-04-26T19:42:34.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:42:34.520 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T19:42:34.526 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-04-26T19:42:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-04-26T19:42:34.545 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-04-26T19:42:34.601 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-04-26T19:42:34.601 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-04-26T19:42:34.601 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T19:42:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T19:42:34.603 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-04-26T19:42:34.623 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-04-26T19:42:34.641 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-04-26T19:42:34.641 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-04-26T19:42:34.698 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-04-26T19:42:34.698 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-04-26T19:42:34.698 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-04-26T19:42:34.698 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-04-26T19:42:34.698 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-04-26T19:42:34.698 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T19:42:34.699 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T19:42:34.699 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-04-26T19:42:34.723 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-04-26T19:42:34.741 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-04-26T19:42:34.741 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-04-26T19:42:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-04-26T19:42:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-04-26T19:42:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-04-26T19:42:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-04-26T19:42:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-04-26T19:42:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T19:42:34.799 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T19:42:34.799 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-04-26T19:42:34.823 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T19:42:34.834 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T19:42:34.927 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T19:42:34.927 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-04-26T19:42:34.930 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-04-26T19:42:34.932 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-04-26T19:42:34.948 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T19:42:34.955 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T19:42:34.955 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-04-26T19:42:34.986 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-04-26T19:42:34.989 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-04-26T19:42:35.080 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T19:42:35.153 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T19:42:35.167 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T19:42:35.168 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-26T19:42:35.168 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-26T19:42:35.168 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T19:42:35.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:42:35.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T19:42:35.343 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:42:35.343 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T19:42:35.343 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-04-26T19:42:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-437.el9.x86_64 2024-04-26T19:42:35.361 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:42:35.361 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T19:42:35.361 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum upgrade -y linux-firmware 2024-04-26T19:42:38.025 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.0 MB 00:01 2024-04-26T19:42:58.740 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 991 kB/s | 19 MB 00:19 2024-04-26T19:43:04.782 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-04-26T19:43:06.166 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-04-26T19:43:09.176 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 21 MB 00:02 2024-04-26T19:43:14.192 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-26T19:43:16.146 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-04-26T19:43:16.147 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-04-26T19:43:16.147 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-04-26T19:43:16.257 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T19:43:16.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:43:16.560 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T19:43:16.560 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T19:43:16.562 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi131.front.sepia.ceph.com to complete... 2024-04-26T19:46:46.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26 19:46:46 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T19:46:46.353 INFO:teuthology.orchestra.run.smithi131.stderr:FINISHED --2024-04-26 19:46:46-- 2024-04-26T19:46:46.353 INFO:teuthology.orchestra.run.smithi131.stderr:Total wall clock time: 3m 30s 2024-04-26T19:46:46.353 INFO:teuthology.orchestra.run.smithi131.stderr:Downloaded: 1 files, 341M in 3m 29s (1.63 MB/s) 2024-04-26T19:46:46.357 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T19:46:46.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T19:46:48.463 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying... ######################################## 2024-04-26T19:46:48.647 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing... ######################################## 2024-04-26T19:46:48.761 INFO:teuthology.orchestra.run.smithi131.stdout:Updating / installing... 2024-04-26T19:46:56.751 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T19:47:34.263 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T19:47:34.264 DEBUG:teuthology.orchestra.run.smithi131:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T19:47:34.367 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:34.367 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T19:47:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T19:47:34.369 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T19:47:34.370 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T19:47:34.371 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T19:47:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T19:47:34.374 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T19:47:34.375 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T19:47:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T19:47:34.377 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T19:47:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T19:47:34.379 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T19:47:34.380 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T19:47:34.381 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T19:47:34.382 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T19:47:34.382 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T19:47:34.382 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T19:47:34.382 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T19:47:34.382 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T19:47:34.382 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T19:47:34.382 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T19:47:34.383 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T19:47:34.383 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T19:47:34.383 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T19:47:34.383 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T19:47:34.383 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T19:47:34.383 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T19:47:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T19:47:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T19:47:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T19:47:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T19:47:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T19:47:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T19:47:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T19:47:34.385 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T19:47:34.385 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T19:47:34.385 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T19:47:34.385 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T19:47:34.385 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T19:47:34.385 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T19:47:34.385 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T19:47:34.386 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T19:47:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T19:47:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T19:47:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T19:47:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T19:47:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T19:47:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T19:47:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T19:47:34.388 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T19:47:34.388 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T19:47:34.388 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T19:47:34.388 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T19:47:34.388 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T19:47:34.388 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T19:47:34.389 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T19:47:34.389 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T19:47:34.389 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T19:47:34.389 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T19:47:34.389 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T19:47:34.389 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T19:47:34.390 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T19:47:34.390 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T19:47:34.390 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T19:47:34.390 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T19:47:34.390 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T19:47:34.390 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T19:47:34.390 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T19:47:34.391 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T19:47:34.392 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T19:47:34.393 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T19:47:34.394 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T19:47:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T19:47:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T19:47:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T19:47:34.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T19:47:34.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T19:47:34.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T19:47:34.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T19:47:34.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T19:47:34.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T19:47:34.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T19:47:34.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T19:47:34.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T19:47:34.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T19:47:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T19:47:34.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T19:47:34.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T19:47:34.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T19:47:34.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T19:47:34.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T19:47:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T19:47:34.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T19:47:34.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T19:47:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T19:47:34.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T19:47:34.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T19:47:34.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T19:47:34.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T19:47:34.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T19:47:34.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T19:47:34.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T19:47:34.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T19:47:34.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T19:47:34.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T19:47:34.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T19:47:34.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T19:47:34.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T19:47:34.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T19:47:34.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T19:47:34.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T19:47:34.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T19:47:34.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T19:47:34.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T19:47:34.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T19:47:34.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T19:47:34.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T19:47:34.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T19:47:34.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T19:47:34.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T19:47:34.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T19:47:34.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T19:47:34.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T19:47:34.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T19:47:34.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T19:47:34.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T19:47:34.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T19:47:34.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T19:47:34.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T19:47:34.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T19:47:34.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T19:47:34.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T19:47:34.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T19:47:34.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T19:47:34.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T19:47:34.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T19:47:34.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T19:47:34.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T19:47:34.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T19:47:34.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T19:47:34.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T19:47:34.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T19:47:34.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T19:47:34.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T19:47:34.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T19:47:34.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T19:47:34.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T19:47:34.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T19:47:34.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T19:47:34.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T19:47:34.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T19:47:34.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T19:47:34.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T19:47:34.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T19:47:34.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T19:47:34.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T19:47:34.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T19:47:34.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T19:47:34.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T19:47:34.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T19:47:34.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T19:47:34.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T19:47:34.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T19:47:34.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T19:47:34.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T19:47:34.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T19:47:34.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T19:47:34.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T19:47:34.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T19:47:34.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T19:47:34.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T19:47:34.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T19:47:34.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T19:47:34.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T19:47:34.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T19:47:34.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T19:47:34.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T19:47:34.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T19:47:34.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T19:47:34.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T19:47:34.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T19:47:34.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T19:47:34.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T19:47:34.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T19:47:34.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T19:47:34.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T19:47:34.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T19:47:34.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T19:47:34.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T19:47:34.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T19:47:34.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T19:47:34.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T19:47:34.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T19:47:34.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T19:47:34.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T19:47:34.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T19:47:34.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T19:47:34.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T19:47:34.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T19:47:34.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T19:47:34.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T19:47:34.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T19:47:34.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T19:47:34.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T19:47:34.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T19:47:34.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T19:47:34.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T19:47:34.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T19:47:34.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T19:47:34.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T19:47:34.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T19:47:34.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T19:47:34.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T19:47:34.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T19:47:34.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T19:47:34.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T19:47:34.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T19:47:34.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T19:47:34.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T19:47:34.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T19:47:34.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T19:47:34.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T19:47:34.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T19:47:34.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T19:47:34.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T19:47:34.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T19:47:34.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T19:47:34.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T19:47:34.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T19:47:34.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T19:47:34.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T19:47:34.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T19:47:34.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T19:47:34.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T19:47:34.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T19:47:34.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T19:47:34.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T19:47:34.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T19:47:34.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T19:47:34.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T19:47:34.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T19:47:34.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T19:47:34.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T19:47:34.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T19:47:34.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T19:47:34.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T19:47:34.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T19:47:34.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T19:47:34.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T19:47:34.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T19:47:34.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T19:47:34.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T19:47:34.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T19:47:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T19:47:34.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T19:47:34.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T19:47:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T19:47:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T19:47:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T19:47:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T19:47:34.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T19:47:34.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T19:47:34.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T19:47:34.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T19:47:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T19:47:34.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T19:47:34.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T19:47:34.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T19:47:34.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T19:47:34.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T19:47:34.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T19:47:34.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T19:47:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T19:47:34.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T19:47:34.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T19:47:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T19:47:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T19:47:34.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T19:47:34.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T19:47:34.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T19:47:34.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T19:47:34.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T19:47:34.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T19:47:34.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T19:47:34.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T19:47:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T19:47:34.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T19:47:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T19:47:34.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T19:47:34.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T19:47:34.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T19:47:34.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T19:47:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T19:47:34.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T19:47:34.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T19:47:34.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T19:47:34.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T19:47:34.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T19:47:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T19:47:34.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T19:47:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T19:47:34.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T19:47:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T19:47:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T19:47:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T19:47:34.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T19:47:34.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T19:47:34.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T19:47:34.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T19:47:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T19:47:34.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T19:47:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T19:47:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T19:47:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T19:47:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T19:47:34.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T19:47:34.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T19:47:34.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T19:47:34.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T19:47:34.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T19:47:34.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T19:47:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T19:47:34.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T19:47:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T19:47:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T19:47:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T19:47:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T19:47:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T19:47:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T19:47:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T19:47:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T19:47:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T19:47:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T19:47:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T19:47:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T19:47:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T19:47:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T19:47:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T19:47:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T19:47:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T19:47:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T19:47:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T19:47:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T19:47:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T19:47:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T19:47:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T19:47:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T19:47:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T19:47:34.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T19:47:34.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T19:47:34.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T19:47:34.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T19:47:34.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T19:47:34.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T19:47:34.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T19:47:34.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T19:47:34.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T19:47:34.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T19:47:34.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T19:47:34.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T19:47:34.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T19:47:34.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T19:47:34.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T19:47:34.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T19:47:34.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T19:47:34.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T19:47:34.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T19:47:34.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T19:47:34.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T19:47:34.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T19:47:34.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T19:47:34.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T19:47:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T19:47:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T19:47:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T19:47:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T19:47:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T19:47:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T19:47:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T19:47:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T19:47:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T19:47:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T19:47:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T19:47:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T19:47:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T19:47:34.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T19:47:34.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T19:47:34.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T19:47:34.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T19:47:34.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T19:47:34.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T19:47:34.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T19:47:34.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T19:47:34.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T19:47:34.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T19:47:34.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T19:47:34.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T19:47:34.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T19:47:34.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T19:47:34.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T19:47:34.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T19:47:34.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T19:47:34.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T19:47:34.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T19:47:34.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T19:47:34.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T19:47:34.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T19:47:34.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T19:47:34.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T19:47:34.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T19:47:34.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T19:47:34.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T19:47:34.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T19:47:34.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T19:47:34.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T19:47:34.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T19:47:34.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T19:47:34.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T19:47:34.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T19:47:34.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T19:47:34.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T19:47:34.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T19:47:34.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T19:47:34.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T19:47:34.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T19:47:34.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T19:47:34.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T19:47:34.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T19:47:34.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T19:47:34.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T19:47:34.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T19:47:34.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T19:47:34.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T19:47:34.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T19:47:34.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T19:47:34.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T19:47:34.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T19:47:34.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T19:47:34.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T19:47:34.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T19:47:34.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T19:47:34.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T19:47:34.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T19:47:34.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T19:47:34.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T19:47:34.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T19:47:34.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T19:47:34.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T19:47:34.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T19:47:34.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T19:47:34.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T19:47:34.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T19:47:34.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T19:47:34.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T19:47:34.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T19:47:34.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T19:47:34.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T19:47:34.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T19:47:34.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T19:47:34.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T19:47:34.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T19:47:34.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T19:47:34.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T19:47:34.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T19:47:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T19:47:34.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T19:47:34.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T19:47:34.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T19:47:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T19:47:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T19:47:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T19:47:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T19:47:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T19:47:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T19:47:34.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T19:47:34.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T19:47:34.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T19:47:34.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T19:47:34.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T19:47:34.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T19:47:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T19:47:34.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T19:47:34.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T19:47:34.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T19:47:34.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T19:47:34.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T19:47:34.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T19:47:34.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T19:47:34.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T19:47:34.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T19:47:34.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T19:47:34.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T19:47:34.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T19:47:34.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T19:47:34.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T19:47:34.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T19:47:34.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T19:47:34.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T19:47:34.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T19:47:34.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T19:47:34.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T19:47:34.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T19:47:34.561 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:34.561 DEBUG:teuthology.orchestra.run.smithi131:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T19:47:34.586 INFO:teuthology.orchestra.run.smithi131.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T19:47:34.586 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T19:47:34.586 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T19:47:34.586 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-26T19:47:34.586 INFO:teuthology.orchestra.run.smithi131.stdout:for file in vmlinuz System.map config; do 2024-04-26T19:47:34.586 INFO:teuthology.orchestra.run.smithi131.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T19:47:34.586 INFO:teuthology.orchestra.run.smithi131.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout: fi 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:done 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T19:47:34.587 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:34.588 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-26T19:47:34.588 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-04-26T19:47:34.667 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-04-26T19:47:34.667 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-04-26T19:47:34.667 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.06 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 68.el9 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Group : Unspecified 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 8264254 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : builder@centos.org 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : CentOS 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T19:47:34.668 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:47:34.669 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T19:47:34.670 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T19:47:34.670 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:34.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T19:47:35.209 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-04-26T19:47:36.542 INFO:teuthology.orchestra.run.smithi131.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T19:47:36.563 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-04-26T19:47:36.565 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-04-26T19:47:36.600 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.u3ufK7LCpU 2024-04-26T19:47:36.600 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.u3ufK7LCpU 2024-04-26T19:47:36.667 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.u3ufK7LCpU 2024-04-26T19:47:36.802 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.u3ufK7LCpU is 6KB 2024-04-26T19:47:36.851 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.u3ufK7LCpU 2024-04-26T19:47:36.866 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T19:47:36.926 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T19:47:36.926 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T19:47:36.926 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T19:47:36.926 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T19:47:36.927 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:37.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-04-26T19:48:07.100 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T19:48:07.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-26T19:48:07.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:48:25.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-04-26T19:48:34.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-26T19:48:34.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:48:37.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-04-26T19:48:49.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-26T19:48:49.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:49:07.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-04-26T19:49:23.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-26T19:49:23.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:49:24.571 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-04-26T19:49:25.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-26T19:49:25.171 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T19:49:25.171 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-04-26T19:49:25.211 INFO:teuthology.orchestra.run.smithi131.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T19:49:25.211 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T19:49:25.211 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T19:49:25.211 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T19:49:26.212 DEBUG:teuthology.parallel:result is None 2024-04-26T19:49:26.212 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T19:49:26.220 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T19:49:26.220 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T19:49:26.227 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T19:49:26.231 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T19:49:26.273 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T19:49:26.279 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T19:49:26.288 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T19:49:26.288 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T19:49:26.293 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T19:49:26.295 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T19:49:26.347 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T19:49:26.355 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T19:49:26.355 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T19:49:26.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:49:26.380 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T19:49:26.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:49:26.396 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T19:49:26.403 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T19:49:26.422 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T19:49:26.449 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:49:26.473 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:49:26.473 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:49:26.500 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:49:26.516 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:49:26.553 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:49:26.556 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T19:49:26.563 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T19:49:26.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T19:49:26.568 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T19:49:26.570 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T19:49:26.626 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T19:49:26.635 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T19:49:26.636 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T19:49:26.640 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T19:49:26.642 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T19:49:26.672 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:49:26.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:49:26.867 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:49:26.935 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:49:27.020 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:49:27.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T19:49:27.148 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:49:27.196 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:49:27.251 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:49:27.338 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:49:27.380 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:49:27.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T19:49:27.446 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:49:27.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:49:27.846 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:49:27.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:49:27.932 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:49:27.932 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T19:49:27.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-04-26T19:49:28.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-04-26T19:49:28.006 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-04-26T19:49:28.039 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:49:28.077 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:49:28.351 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T19:49:28.358 INFO:teuthology.task.internal:Starting timer... 2024-04-26T19:49:28.359 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T19:49:28.373 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T19:49:28.384 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T19:49:28.384 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-04-26T19:49:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-04-26T19:49:28.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-04-26T19:49:28.592 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-04-26T19:49:28.594 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-04-26T19:49:28.675 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-04-26T19:49:28.675 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T19:49:28.675 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-04-26T19:49:28.723 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-04-26T19:49:28.723 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-04-26T19:49:28.746 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-04-26T19:49:28.746 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-04-26T19:49:28.766 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-04-26T19:49:28.766 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-04-26T19:49:28.766 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:49:28.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:49:28.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:49:28.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:49:28.861 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:49:28.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:49:28.888 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T19:49:28.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-04-26T19:49:29.008 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-04-26T19:49:29.034 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-04-26T19:49:29.060 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T19:49:29.070 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T19:49:29.085 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T19:49:29.089 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi148.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T19:54:34.869 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-04-26T19:54:34.870 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-26T19:54:34.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:54:34.975 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-04-26T19:54:35.145 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-26T19:54:35.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-26T19:54:35.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:54:35.222 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-04-26T19:54:35.302 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-26T19:54:35.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-26T19:54:35.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:54:35.378 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-26T19:54:35.462 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-26T19:54:35.462 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T19:54:35.475 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T19:54:35.475 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T19:54:35.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:54:35.483 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T19:54:35.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:54:35.485 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T19:54:35.486 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:54:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T19:54:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T19:54:35.530 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T19:54:35.544 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T19:54:35.551 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-04-26T19:54:35.561 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-04-26T19:54:35.567 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-04-26T19:54:35.572 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T19:54:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T19:54:35.577 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-04-26T19:54:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T19:54:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T19:54:35.603 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T19:54:35.611 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T19:54:35.640 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:54:35.644 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:54:35.644 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-26T19:54:35.644 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.644 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.644 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.644 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.660 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:54:35.663 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:54:35.663 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-26T19:54:35.663 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.663 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.663 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.663 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.683 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-04-26T19:54:35.717 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-04-26T19:54:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T19:54:35.801 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T19:54:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:54:35.940 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:54:35.941 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-04-26T19:54:35.941 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.941 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.941 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.941 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:54:35.942 INFO:teuthology.run_tasks:Running task install... 2024-04-26T19:54:35.952 DEBUG:teuthology.task.install:project ceph 2024-04-26T19:54:35.952 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T19:54:35.952 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T19:54:35.952 INFO:teuthology.task.install:Using flavor: default 2024-04-26T19:54:35.958 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T19:54:35.958 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T19:54:35.959 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T19:54:35.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:54:35.961 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T19:54:35.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:54:35.963 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T19:54:35.963 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:54:36.177 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T19:54:36.177 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T19:54:36.189 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T19:54:36.189 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T19:54:36.196 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T19:54:36.196 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T19:54:36.352 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T19:54:36.352 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:54:36.352 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T19:54:36.361 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T19:54:36.361 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:54:36.361 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T19:54:36.370 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T19:54:36.370 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:54:36.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T19:54:36.387 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T19:54:36.387 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T19:54:36.395 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T19:54:36.396 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T19:54:36.444 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T19:54:36.444 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T19:54:36.453 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T19:54:36.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T19:54:36.526 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T19:54:36.535 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T19:54:36.572 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T19:54:36.589 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-04-26T19:54:36.591 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-04-26T19:54:36.598 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-04-26T19:54:36.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-04-26T19:54:36.686 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T19:54:36.767 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-04-26T19:54:36.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-04-26T19:54:36.820 INFO:teuthology.orchestra.run.smithi157.stdout:48 files removed 2024-04-26T19:54:36.828 INFO:teuthology.orchestra.run.smithi148.stdout:48 files removed 2024-04-26T19:54:36.847 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T19:54:36.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T19:54:37.104 INFO:teuthology.orchestra.run.smithi131.stdout:48 files removed 2024-04-26T19:54:37.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T19:54:37.492 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 218 kB/s | 84 kB 00:00 2024-04-26T19:54:37.503 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 216 kB/s | 84 kB 00:00 2024-04-26T19:54:37.867 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-26T19:54:37.920 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T19:54:38.079 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 169 kB/s | 84 kB 00:00 2024-04-26T19:54:38.218 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-26T19:54:38.254 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-04-26T19:54:38.729 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-26T19:54:38.843 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T19:54:39.038 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T19:54:39.239 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-26T19:54:39.987 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T19:54:40.903 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-26T19:54:41.231 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T19:54:42.280 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T19:54:45.079 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T19:54:45.592 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.2 MB 00:00 2024-04-26T19:54:46.705 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-26T19:54:47.232 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-26T19:54:47.889 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-26T19:54:51.441 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 21 MB 00:04 2024-04-26T19:54:53.156 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 7.4 kB/s | 1.7 kB 00:00 2024-04-26T19:54:53.677 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 839 kB/s | 6.2 MB 00:07 2024-04-26T19:54:55.290 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:54:55.291 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:54:55.301 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T19:54:55.302 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T19:54:55.303 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T19:54:55.304 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T19:54:55.305 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T19:54:55.308 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T19:54:55.308 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T19:54:55.369 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-26T19:54:55.388 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T19:54:55.389 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T19:54:55.390 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T19:54:55.391 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T19:54:55.392 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T19:54:55.393 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T19:54:55.394 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T19:54:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T19:54:55.396 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T19:54:55.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T19:54:55.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T19:54:55.399 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout:Install 148 Packages 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 4 Packages 2024-04-26T19:54:55.400 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:54:55.401 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 222 M 2024-04-26T19:54:55.401 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-26T19:54:55.589 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-26T19:54:55.991 INFO:teuthology.orchestra.run.smithi157.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-26T19:54:56.233 INFO:teuthology.orchestra.run.smithi157.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.6 MB/s | 889 kB 00:00 2024-04-26T19:54:56.283 INFO:teuthology.orchestra.run.smithi157.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.8 MB/s | 145 kB 00:00 2024-04-26T19:54:56.533 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T19:54:56.558 INFO:teuthology.orchestra.run.smithi157.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-26T19:54:56.717 INFO:teuthology.orchestra.run.smithi157.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-04-26T19:54:57.126 INFO:teuthology.orchestra.run.smithi157.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 12 MB/s | 4.7 MB 00:00 2024-04-26T19:54:57.192 INFO:teuthology.orchestra.run.smithi157.stdout:(7/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-04-26T19:54:58.634 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:54:58.635 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:54:58.646 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T19:54:58.646 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T19:54:58.647 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T19:54:58.648 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T19:54:58.650 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T19:54:58.651 INFO:teuthology.orchestra.run.smithi157.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 17 MB 00:01 2024-04-26T19:54:58.652 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T19:54:58.653 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T19:54:58.701 INFO:teuthology.orchestra.run.smithi157.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 506 kB/s | 25 kB 00:00 2024-04-26T19:54:58.713 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-26T19:54:58.733 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-26T19:54:58.733 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-26T19:54:58.733 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-26T19:54:58.734 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-26T19:54:58.734 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T19:54:58.734 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T19:54:58.734 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:54:58.734 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T19:54:58.734 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T19:54:58.735 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T19:54:58.735 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T19:54:58.735 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T19:54:58.735 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T19:54:58.735 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T19:54:58.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T19:54:58.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T19:54:58.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T19:54:58.736 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T19:54:58.736 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T19:54:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T19:54:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T19:54:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T19:54:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T19:54:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T19:54:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T19:54:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T19:54:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T19:54:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T19:54:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T19:54:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T19:54:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T19:54:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T19:54:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T19:54:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T19:54:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T19:54:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T19:54:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T19:54:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T19:54:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:54:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T19:54:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T19:54:58.740 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-04-26T19:54:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T19:54:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T19:54:58.742 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T19:54:58.743 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T19:54:58.744 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T19:54:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T19:54:58.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T19:54:58.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T19:54:58.748 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T19:54:58.749 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T19:54:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T19:54:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T19:54:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T19:54:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T19:54:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T19:54:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T19:54:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T19:54:58.751 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T19:54:58.752 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:Install 148 Packages 2024-04-26T19:54:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 4 Packages 2024-04-26T19:54:58.754 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:54:58.754 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 222 M 2024-04-26T19:54:58.755 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-26T19:54:58.910 INFO:teuthology.orchestra.run.smithi157.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 6.8 MB/s | 21 MB 00:03 2024-04-26T19:54:59.018 INFO:teuthology.orchestra.run.smithi157.stdout:(11/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 5.8 MB/s | 11 MB 00:01 2024-04-26T19:54:59.044 INFO:teuthology.orchestra.run.smithi157.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 225 kB 00:00 2024-04-26T19:54:59.077 INFO:teuthology.orchestra.run.smithi157.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 464 kB/s | 27 kB 00:00 2024-04-26T19:54:59.103 INFO:teuthology.orchestra.run.smithi157.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 535 kB/s | 32 kB 00:00 2024-04-26T19:54:59.187 INFO:teuthology.orchestra.run.smithi157.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T19:54:59.229 INFO:teuthology.orchestra.run.smithi157.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.8 MB/s | 747 kB 00:00 2024-04-26T19:54:59.262 INFO:teuthology.orchestra.run.smithi157.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T19:54:59.321 INFO:teuthology.orchestra.run.smithi157.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 5.3 MB/s | 501 kB 00:00 2024-04-26T19:54:59.421 INFO:teuthology.orchestra.run.smithi157.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 452 kB/s | 45 kB 00:00 2024-04-26T19:54:59.466 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 21 MB 00:03 2024-04-26T19:54:59.530 INFO:teuthology.orchestra.run.smithi157.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T19:54:59.647 INFO:teuthology.orchestra.run.smithi157.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.4 MB/s | 161 kB 00:00 2024-04-26T19:54:59.739 INFO:teuthology.orchestra.run.smithi157.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 5.4 MB 00:00 2024-04-26T19:54:59.764 INFO:teuthology.orchestra.run.smithi157.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 321 kB 00:00 2024-04-26T19:54:59.814 INFO:teuthology.orchestra.run.smithi157.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.9 MB/s | 298 kB 00:00 2024-04-26T19:54:59.848 INFO:teuthology.orchestra.run.smithi157.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T19:54:59.890 INFO:teuthology.orchestra.run.smithi157.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T19:55:00.024 INFO:teuthology.orchestra.run.smithi157.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-26T19:55:00.099 INFO:teuthology.orchestra.run.smithi157.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 457 kB/s | 34 kB 00:00 2024-04-26T19:55:00.183 INFO:teuthology.orchestra.run.smithi148.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-04-26T19:55:00.190 INFO:teuthology.orchestra.run.smithi157.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.5 MB/s | 142 kB 00:00 2024-04-26T19:55:00.316 INFO:teuthology.orchestra.run.smithi157.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 6.6 MB/s | 3.1 MB 00:00 2024-04-26T19:55:00.666 INFO:teuthology.orchestra.run.smithi157.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.4 MB/s | 3.5 MB 00:00 2024-04-26T19:55:00.741 INFO:teuthology.orchestra.run.smithi157.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 3.3 MB/s | 249 kB 00:00 2024-04-26T19:55:00.808 INFO:teuthology.orchestra.run.smithi157.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 742 kB/s | 49 kB 00:00 2024-04-26T19:55:00.875 INFO:teuthology.orchestra.run.smithi157.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 247 kB/s | 16 kB 00:00 2024-04-26T19:55:00.961 INFO:teuthology.orchestra.run.smithi157.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.1 MB/s | 273 kB 00:00 2024-04-26T19:55:01.103 INFO:teuthology.orchestra.run.smithi157.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.3 MB/s | 762 kB 00:00 2024-04-26T19:55:01.178 INFO:teuthology.orchestra.run.smithi157.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 243 kB/s | 18 kB 00:00 2024-04-26T19:55:01.237 INFO:teuthology.orchestra.run.smithi157.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-04-26T19:55:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-26T19:55:01.421 INFO:teuthology.orchestra.run.smithi157.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-26T19:55:01.513 INFO:teuthology.orchestra.run.smithi157.stdout:(41/152): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-04-26T19:55:01.543 INFO:teuthology.orchestra.run.smithi148.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 654 kB/s | 889 kB 00:01 2024-04-26T19:55:01.580 INFO:teuthology.orchestra.run.smithi157.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-26T19:55:01.646 INFO:teuthology.orchestra.run.smithi157.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T19:55:01.689 INFO:teuthology.orchestra.run.smithi157.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-04-26T19:55:01.693 INFO:teuthology.orchestra.run.smithi148.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 969 kB/s | 145 kB 00:00 2024-04-26T19:55:01.722 INFO:teuthology.orchestra.run.smithi157.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-26T19:55:01.764 INFO:teuthology.orchestra.run.smithi157.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-26T19:55:01.814 INFO:teuthology.orchestra.run.smithi157.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-26T19:55:02.101 INFO:teuthology.orchestra.run.smithi148.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-26T19:55:02.281 INFO:teuthology.orchestra.run.smithi157.stdout:(48/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 14 MB/s | 49 MB 00:03 2024-04-26T19:55:02.348 INFO:teuthology.orchestra.run.smithi157.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 689 kB 00:00 2024-04-26T19:55:02.399 INFO:teuthology.orchestra.run.smithi157.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-26T19:55:02.432 INFO:teuthology.orchestra.run.smithi157.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-04-26T19:55:02.451 INFO:teuthology.orchestra.run.smithi148.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-04-26T19:55:02.457 INFO:teuthology.orchestra.run.smithi157.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 609 kB/s | 107 kB 00:00 2024-04-26T19:55:02.491 INFO:teuthology.orchestra.run.smithi157.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-04-26T19:55:02.516 INFO:teuthology.orchestra.run.smithi157.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-26T19:55:02.541 INFO:teuthology.orchestra.run.smithi157.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-26T19:55:02.575 INFO:teuthology.orchestra.run.smithi157.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-26T19:55:02.600 INFO:teuthology.orchestra.run.smithi157.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T19:55:02.626 INFO:teuthology.orchestra.run.smithi157.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-26T19:55:02.659 INFO:teuthology.orchestra.run.smithi157.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-26T19:55:02.726 INFO:teuthology.orchestra.run.smithi157.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-26T19:55:02.743 INFO:teuthology.orchestra.run.smithi157.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-26T19:55:02.777 INFO:teuthology.orchestra.run.smithi157.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-26T19:55:02.794 INFO:teuthology.orchestra.run.smithi157.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-04-26T19:55:02.827 INFO:teuthology.orchestra.run.smithi157.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-26T19:55:02.853 INFO:teuthology.orchestra.run.smithi157.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 599 kB/s | 35 kB 00:00 2024-04-26T19:55:02.886 INFO:teuthology.orchestra.run.smithi157.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-26T19:55:03.020 INFO:teuthology.orchestra.run.smithi157.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T19:55:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-26T19:55:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-26T19:55:03.185 INFO:teuthology.orchestra.run.smithi148.stdout:(6/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-04-26T19:55:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-26T19:55:03.255 INFO:teuthology.orchestra.run.smithi157.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 911 kB 00:00 2024-04-26T19:55:03.305 INFO:teuthology.orchestra.run.smithi157.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-26T19:55:03.339 INFO:teuthology.orchestra.run.smithi157.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-04-26T19:55:03.369 INFO:teuthology.orchestra.run.smithi148.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-04-26T19:55:03.430 INFO:teuthology.orchestra.run.smithi157.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-26T19:55:03.456 INFO:teuthology.orchestra.run.smithi157.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-04-26T19:55:03.497 INFO:teuthology.orchestra.run.smithi157.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-04-26T19:55:03.523 INFO:teuthology.orchestra.run.smithi157.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-26T19:55:03.548 INFO:teuthology.orchestra.run.smithi157.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-26T19:55:03.573 INFO:teuthology.orchestra.run.smithi157.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-26T19:55:03.598 INFO:teuthology.orchestra.run.smithi157.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-26T19:55:03.749 INFO:teuthology.orchestra.run.smithi157.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-26T19:55:03.824 INFO:teuthology.orchestra.run.smithi157.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-26T19:55:03.908 INFO:teuthology.orchestra.run.smithi157.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.9 MB/s | 6.0 MB 00:00 2024-04-26T19:55:03.933 INFO:teuthology.orchestra.run.smithi157.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 711 kB/s | 77 kB 00:00 2024-04-26T19:55:03.958 INFO:teuthology.orchestra.run.smithi157.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-26T19:55:03.983 INFO:teuthology.orchestra.run.smithi157.stdout:(86/152): python3-oauthlib-3.1.1-5.el9.noarch.r 948 kB/s | 222 kB 00:00 2024-04-26T19:55:04.009 INFO:teuthology.orchestra.run.smithi157.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-26T19:55:04.034 INFO:teuthology.orchestra.run.smithi157.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-26T19:55:04.059 INFO:teuthology.orchestra.run.smithi157.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T19:55:04.084 INFO:teuthology.orchestra.run.smithi157.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T19:55:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-04-26T19:55:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-04-26T19:55:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-04-26T19:55:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-26T19:55:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 610 kB/s | 36 kB 00:00 2024-04-26T19:55:04.270 INFO:teuthology.orchestra.run.smithi157.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-04-26T19:55:04.295 INFO:teuthology.orchestra.run.smithi157.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-26T19:55:04.321 INFO:teuthology.orchestra.run.smithi157.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T19:55:04.371 INFO:teuthology.orchestra.run.smithi157.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T19:55:04.396 INFO:teuthology.orchestra.run.smithi157.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 933 kB/s | 94 kB 00:00 2024-04-26T19:55:04.413 INFO:teuthology.orchestra.run.smithi157.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-26T19:55:04.505 INFO:teuthology.orchestra.run.smithi157.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-04-26T19:55:04.614 INFO:teuthology.orchestra.run.smithi157.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-04-26T19:55:04.927 INFO:teuthology.orchestra.run.smithi148.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 6.8 MB/s | 11 MB 00:01 2024-04-26T19:55:04.977 INFO:teuthology.orchestra.run.smithi148.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 507 kB/s | 25 kB 00:00 2024-04-26T19:55:05.105 INFO:teuthology.orchestra.run.smithi157.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 629 kB/s | 309 kB 00:00 2024-04-26T19:55:05.131 INFO:teuthology.orchestra.run.smithi157.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 876 kB/s | 548 kB 00:00 2024-04-26T19:55:05.289 INFO:teuthology.orchestra.run.smithi157.stdout:(106/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-04-26T19:55:05.314 INFO:teuthology.orchestra.run.smithi157.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 93 kB/s | 19 kB 00:00 2024-04-26T19:55:05.440 INFO:teuthology.orchestra.run.smithi157.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-04-26T19:55:05.466 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-26T19:55:05.573 INFO:teuthology.orchestra.run.smithi157.stdout:(109/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 92 kB/s | 26 kB 00:00 2024-04-26T19:55:05.598 INFO:teuthology.orchestra.run.smithi157.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-04-26T19:55:05.749 INFO:teuthology.orchestra.run.smithi157.stdout:(111/152): libarrow-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-04-26T19:55:05.874 INFO:teuthology.orchestra.run.smithi157.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 503 kB/s | 151 kB 00:00 2024-04-26T19:55:05.908 INFO:teuthology.orchestra.run.smithi157.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-04-26T19:55:05.941 INFO:teuthology.orchestra.run.smithi157.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-26T19:55:05.975 INFO:teuthology.orchestra.run.smithi157.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 432 kB/s | 43 kB 00:00 2024-04-26T19:55:06.017 INFO:teuthology.orchestra.run.smithi157.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-26T19:55:06.042 INFO:teuthology.orchestra.run.smithi157.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-26T19:55:06.084 INFO:teuthology.orchestra.run.smithi157.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-26T19:55:06.151 INFO:teuthology.orchestra.run.smithi157.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-26T19:55:06.176 INFO:teuthology.orchestra.run.smithi157.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-26T19:55:06.201 INFO:teuthology.orchestra.run.smithi157.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-04-26T19:55:06.277 INFO:teuthology.orchestra.run.smithi157.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-26T19:55:06.344 INFO:teuthology.orchestra.run.smithi157.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-26T19:55:06.377 INFO:teuthology.orchestra.run.smithi157.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-26T19:55:06.461 INFO:teuthology.orchestra.run.smithi157.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-26T19:55:06.486 INFO:teuthology.orchestra.run.smithi157.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-26T19:55:06.528 INFO:teuthology.orchestra.run.smithi157.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-04-26T19:55:06.562 INFO:teuthology.orchestra.run.smithi157.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-26T19:55:06.587 INFO:teuthology.orchestra.run.smithi157.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-04-26T19:55:06.671 INFO:teuthology.orchestra.run.smithi157.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-26T19:55:06.705 INFO:teuthology.orchestra.run.smithi157.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 674 kB/s | 79 kB 00:00 2024-04-26T19:55:06.746 INFO:teuthology.orchestra.run.smithi157.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-26T19:55:06.772 INFO:teuthology.orchestra.run.smithi157.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-04-26T19:55:06.830 INFO:teuthology.orchestra.run.smithi157.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-26T19:55:06.856 INFO:teuthology.orchestra.run.smithi157.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-26T19:55:06.881 INFO:teuthology.orchestra.run.smithi157.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-26T19:55:06.931 INFO:teuthology.orchestra.run.smithi157.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-26T19:55:06.956 INFO:teuthology.orchestra.run.smithi157.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-26T19:55:06.998 INFO:teuthology.orchestra.run.smithi157.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-26T19:55:07.049 INFO:teuthology.orchestra.run.smithi157.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 503 kB/s | 59 kB 00:00 2024-04-26T19:55:07.065 INFO:teuthology.orchestra.run.smithi157.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-04-26T19:55:07.099 INFO:teuthology.orchestra.run.smithi157.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-26T19:55:07.166 INFO:teuthology.orchestra.run.smithi157.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-26T19:55:07.191 INFO:teuthology.orchestra.run.smithi157.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 714 kB/s | 90 kB 00:00 2024-04-26T19:55:07.275 INFO:teuthology.orchestra.run.smithi157.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-26T19:55:07.300 INFO:teuthology.orchestra.run.smithi157.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-04-26T19:55:07.326 INFO:teuthology.orchestra.run.smithi157.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-26T19:55:07.542 INFO:teuthology.orchestra.run.smithi157.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-04-26T19:55:07.701 INFO:teuthology.orchestra.run.smithi157.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 8.5 MB/s | 3.4 MB 00:00 2024-04-26T19:55:07.743 INFO:teuthology.orchestra.run.smithi157.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 1.3 MB 00:00 2024-04-26T19:55:07.902 INFO:teuthology.orchestra.run.smithi148.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.5 MB/s | 21 MB 00:08 2024-04-26T19:55:07.910 INFO:teuthology.orchestra.run.smithi157.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-04-26T19:55:07.926 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:55:07.927 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:55:07.948 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T19:55:07.949 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T19:55:07.950 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T19:55:07.952 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T19:55:07.954 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T19:55:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T19:55:07.958 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T19:55:07.959 INFO:teuthology.orchestra.run.smithi157.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-04-26T19:55:07.962 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:55:07.963 INFO:teuthology.orchestra.run.smithi157.stdout:Total 18 MB/s | 222 MB 00:12 2024-04-26T19:55:08.019 INFO:teuthology.orchestra.run.smithi148.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 225 kB 00:00 2024-04-26T19:55:08.024 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-04-26T19:55:08.039 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-04-26T19:55:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-04-26T19:55:08.039 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-04-26T19:55:08.039 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-04-26T19:55:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T19:55:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T19:55:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T19:55:08.042 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T19:55:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T19:55:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T19:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T19:55:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T19:55:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T19:55:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T19:55:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T19:55:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout:Install 148 Packages 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 4 Packages 2024-04-26T19:55:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:55:08.052 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 222 M 2024-04-26T19:55:08.052 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-04-26T19:55:08.077 INFO:teuthology.orchestra.run.smithi148.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 465 kB/s | 27 kB 00:00 2024-04-26T19:55:08.136 INFO:teuthology.orchestra.run.smithi148.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 540 kB/s | 32 kB 00:00 2024-04-26T19:55:08.354 INFO:teuthology.orchestra.run.smithi148.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.4 MB/s | 747 kB 00:00 2024-04-26T19:55:08.429 INFO:teuthology.orchestra.run.smithi148.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.1 MB/s | 163 kB 00:00 2024-04-26T19:55:08.496 INFO:teuthology.orchestra.run.smithi148.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T19:55:08.771 INFO:teuthology.orchestra.run.smithi148.stdout:(17/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 3.0 MB/s | 17 MB 00:05 2024-04-26T19:55:08.796 INFO:teuthology.orchestra.run.smithi148.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 1.6 MB/s | 501 kB 00:00 2024-04-26T19:55:08.855 INFO:teuthology.orchestra.run.smithi148.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 774 kB/s | 45 kB 00:00 2024-04-26T19:55:08.922 INFO:teuthology.orchestra.run.smithi148.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.0 MB/s | 135 kB 00:00 2024-04-26T19:55:08.989 INFO:teuthology.orchestra.run.smithi148.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T19:55:09.056 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-26T19:55:09.097 INFO:teuthology.orchestra.run.smithi148.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.9 MB/s | 321 kB 00:00 2024-04-26T19:55:09.203 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-26T19:55:09.203 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-26T19:55:09.206 INFO:teuthology.orchestra.run.smithi148.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.7 MB/s | 298 kB 00:00 2024-04-26T19:55:09.331 INFO:teuthology.orchestra.run.smithi148.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 796 kB/s | 99 kB 00:00 2024-04-26T19:55:09.351 INFO:teuthology.orchestra.run.smithi131.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-26T19:55:09.390 INFO:teuthology.orchestra.run.smithi148.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T19:55:09.585 INFO:teuthology.orchestra.run.smithi131.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.7 MB/s | 889 kB 00:00 2024-04-26T19:55:09.651 INFO:teuthology.orchestra.run.smithi131.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.2 MB/s | 145 kB 00:00 2024-04-26T19:55:10.052 INFO:teuthology.orchestra.run.smithi131.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-04-26T19:55:10.310 INFO:teuthology.orchestra.run.smithi131.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 5.9 MB/s | 1.5 MB 00:00 2024-04-26T19:55:10.382 INFO:teuthology.orchestra.run.smithi148.stdout:(26/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 9.1 MB/s | 49 MB 00:05 2024-04-26T19:55:10.590 INFO:teuthology.orchestra.run.smithi148.stdout:(27/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 3.0 MB/s | 5.4 MB 00:01 2024-04-26T19:55:10.607 INFO:teuthology.orchestra.run.smithi148.stdout:(28/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 759 kB/s | 171 kB 00:00 2024-04-26T19:55:10.716 INFO:teuthology.orchestra.run.smithi148.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3306.g 273 kB/s | 34 kB 00:00 2024-04-26T19:55:10.741 INFO:teuthology.orchestra.run.smithi148.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.0 MB/s | 142 kB 00:00 2024-04-26T19:55:10.858 INFO:teuthology.orchestra.run.smithi148.stdout:(31/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 2.1 MB/s | 3.1 MB 00:01 2024-04-26T19:55:10.967 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-26T19:55:10.968 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-26T19:55:11.025 INFO:teuthology.orchestra.run.smithi148.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.5 MB/s | 249 kB 00:00 2024-04-26T19:55:11.028 INFO:teuthology.orchestra.run.smithi131.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-26T19:55:11.092 INFO:teuthology.orchestra.run.smithi148.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 741 kB/s | 49 kB 00:00 2024-04-26T19:55:11.176 INFO:teuthology.orchestra.run.smithi148.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 197 kB/s | 16 kB 00:00 2024-04-26T19:55:11.186 INFO:teuthology.orchestra.run.smithi131.stdout:(7/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-26T19:55:11.293 INFO:teuthology.orchestra.run.smithi148.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 273 kB 00:00 2024-04-26T19:55:11.462 INFO:teuthology.orchestra.run.smithi131.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 8.6 MB/s | 21 MB 00:02 2024-04-26T19:55:11.535 INFO:teuthology.orchestra.run.smithi148.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-26T19:55:11.537 INFO:teuthology.orchestra.run.smithi131.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 341 kB/s | 25 kB 00:00 2024-04-26T19:55:11.601 INFO:teuthology.orchestra.run.smithi148.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 274 kB/s | 18 kB 00:00 2024-04-26T19:55:11.710 INFO:teuthology.orchestra.run.smithi148.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-04-26T19:55:11.860 INFO:teuthology.orchestra.run.smithi148.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-04-26T19:55:11.885 INFO:teuthology.orchestra.run.smithi148.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-26T19:55:11.944 INFO:teuthology.orchestra.run.smithi148.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-26T19:55:12.020 INFO:teuthology.orchestra.run.smithi148.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-26T19:55:12.044 INFO:teuthology.orchestra.run.smithi148.stdout:(43/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-26T19:55:12.095 INFO:teuthology.orchestra.run.smithi148.stdout:(44/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 2.6 MB/s | 3.5 MB 00:01 2024-04-26T19:55:12.120 INFO:teuthology.orchestra.run.smithi148.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-26T19:55:12.145 INFO:teuthology.orchestra.run.smithi148.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-26T19:55:12.170 INFO:teuthology.orchestra.run.smithi148.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T19:55:12.271 INFO:teuthology.orchestra.run.smithi148.stdout:(48/152): python3-requests-2.25.1-8.el9.noarch. 711 kB/s | 125 kB 00:00 2024-04-26T19:55:12.938 INFO:teuthology.orchestra.run.smithi148.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-04-26T19:55:13.079 INFO:teuthology.orchestra.run.smithi131.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 8.2 MB/s | 17 MB 00:02 2024-04-26T19:55:13.137 INFO:teuthology.orchestra.run.smithi131.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 225 kB 00:00 2024-04-26T19:55:13.196 INFO:teuthology.orchestra.run.smithi131.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 472 kB/s | 27 kB 00:00 2024-04-26T19:55:13.254 INFO:teuthology.orchestra.run.smithi131.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 552 kB/s | 32 kB 00:00 2024-04-26T19:55:13.413 INFO:teuthology.orchestra.run.smithi131.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.6 MB/s | 747 kB 00:00 2024-04-26T19:55:13.421 INFO:teuthology.orchestra.run.smithi148.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 540 kB/s | 689 kB 00:01 2024-04-26T19:55:13.438 INFO:teuthology.orchestra.run.smithi148.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 43 kB/s | 21 kB 00:00 2024-04-26T19:55:13.480 INFO:teuthology.orchestra.run.smithi131.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 163 kB 00:00 2024-04-26T19:55:13.558 INFO:teuthology.orchestra.run.smithi131.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T19:55:13.655 INFO:teuthology.orchestra.run.smithi131.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 5.0 MB/s | 501 kB 00:00 2024-04-26T19:55:13.772 INFO:teuthology.orchestra.run.smithi148.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 589 kB/s | 943 kB 00:01 2024-04-26T19:55:13.864 INFO:teuthology.orchestra.run.smithi131.stdout:(18/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.0 MB/s | 11 MB 00:02 2024-04-26T19:55:13.948 INFO:teuthology.orchestra.run.smithi131.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 549 kB/s | 45 kB 00:00 2024-04-26T19:55:13.980 INFO:teuthology.orchestra.run.smithi148.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-04-26T19:55:14.057 INFO:teuthology.orchestra.run.smithi131.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T19:55:14.165 INFO:teuthology.orchestra.run.smithi131.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.5 MB/s | 161 kB 00:00 2024-04-26T19:55:14.305 INFO:teuthology.orchestra.run.smithi148.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 355 kB/s | 313 kB 00:00 2024-04-26T19:55:14.332 INFO:teuthology.orchestra.run.smithi131.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.9 MB/s | 5.4 MB 00:00 2024-04-26T19:55:14.374 INFO:teuthology.orchestra.run.smithi131.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.5 MB/s | 321 kB 00:00 2024-04-26T19:55:14.408 INFO:teuthology.orchestra.run.smithi131.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.9 MB/s | 298 kB 00:00 2024-04-26T19:55:14.441 INFO:teuthology.orchestra.run.smithi131.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T19:55:14.467 INFO:teuthology.orchestra.run.smithi131.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T19:55:14.481 INFO:teuthology.orchestra.run.smithi148.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-04-26T19:55:14.525 INFO:teuthology.orchestra.run.smithi131.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-26T19:55:14.576 INFO:teuthology.orchestra.run.smithi131.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 696 kB/s | 34 kB 00:00 2024-04-26T19:55:14.798 INFO:teuthology.orchestra.run.smithi148.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-04-26T19:55:15.135 INFO:teuthology.orchestra.run.smithi131.stdout:(29/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 14 MB/s | 49 MB 00:03 2024-04-26T19:55:15.198 INFO:teuthology.orchestra.run.smithi131.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 240 kB/s | 142 kB 00:00 2024-04-26T19:55:15.273 INFO:teuthology.orchestra.run.smithi148.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 202 kB/s | 160 kB 00:00 2024-04-26T19:55:15.285 INFO:teuthology.orchestra.run.smithi131.stdout:(31/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.7 MB/s | 3.1 MB 00:00 2024-04-26T19:55:15.365 INFO:teuthology.orchestra.run.smithi148.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-04-26T19:55:15.444 INFO:teuthology.orchestra.run.smithi131.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 11 MB/s | 3.5 MB 00:00 2024-04-26T19:55:15.477 INFO:teuthology.orchestra.run.smithi131.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.3 MB/s | 249 kB 00:00 2024-04-26T19:55:15.534 INFO:teuthology.orchestra.run.smithi131.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 551 kB/s | 49 kB 00:00 2024-04-26T19:55:15.561 INFO:teuthology.orchestra.run.smithi131.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 199 kB/s | 16 kB 00:00 2024-04-26T19:55:15.595 INFO:teuthology.orchestra.run.smithi131.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 4.5 MB/s | 273 kB 00:00 2024-04-26T19:55:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 316 kB/s | 18 kB 00:00 2024-04-26T19:55:15.746 INFO:teuthology.orchestra.run.smithi131.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 4.1 MB/s | 762 kB 00:00 2024-04-26T19:55:15.879 INFO:teuthology.orchestra.run.smithi131.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-26T19:55:16.463 INFO:teuthology.orchestra.run.smithi131.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 99 kB/s | 80 kB 00:00 2024-04-26T19:55:16.557 INFO:teuthology.orchestra.run.smithi148.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 489 kB/s | 582 kB 00:01 2024-04-26T19:55:16.582 INFO:teuthology.orchestra.run.smithi148.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 506 kB/s | 662 kB 00:01 2024-04-26T19:55:16.787 INFO:teuthology.orchestra.run.smithi131.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 215 kB/s | 195 kB 00:00 2024-04-26T19:55:16.884 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-26T19:55:16.963 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T19:55:17.044 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T19:55:17.074 INFO:teuthology.orchestra.run.smithi148.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 49 kB/s | 25 kB 00:00 2024-04-26T19:55:17.141 INFO:teuthology.orchestra.run.smithi148.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 85 kB/s | 47 kB 00:00 2024-04-26T19:55:17.232 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T19:55:17.273 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T19:55:17.299 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T19:55:17.322 INFO:teuthology.orchestra.run.smithi131.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 510 kB/s | 803 kB 00:01 2024-04-26T19:55:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T19:55:17.355 INFO:teuthology.orchestra.run.smithi131.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 284 kB/s | 253 kB 00:00 2024-04-26T19:55:17.668 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T19:55:17.782 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:55:17.883 INFO:teuthology.orchestra.run.smithi148.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 745 kB/s | 3.0 MB 00:04 2024-04-26T19:55:17.903 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:55:17.933 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:55:17.958 INFO:teuthology.orchestra.run.smithi148.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 231 kB/s | 188 kB 00:00 2024-04-26T19:55:18.000 INFO:teuthology.orchestra.run.smithi148.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 316 kB/s | 292 kB 00:00 2024-04-26T19:55:18.025 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:55:18.055 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:55:18.106 INFO:teuthology.orchestra.run.smithi131.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 136 kB/s | 106 kB 00:00 2024-04-26T19:55:18.197 INFO:teuthology.orchestra.run.smithi131.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 161 kB/s | 135 kB 00:00 2024-04-26T19:55:18.208 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T19:55:18.251 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T19:55:18.279 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:55:18.382 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:55:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:55:18.425 INFO:teuthology.orchestra.run.smithi148.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 65 kB/s | 35 kB 00:00 2024-04-26T19:55:18.468 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:55:18.490 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:55:18.510 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:55:18.732 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:55:18.764 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:55:18.875 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T19:55:18.939 INFO:teuthology.orchestra.run.smithi131.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 150 kB/s | 125 kB 00:00 2024-04-26T19:55:18.968 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T19:55:18.989 INFO:teuthology.orchestra.run.smithi131.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 570 kB/s | 1.2 MB 00:02 2024-04-26T19:55:19.009 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T19:55:19.056 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T19:55:19.106 INFO:teuthology.orchestra.run.smithi131.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 236 kB/s | 215 kB 00:00 2024-04-26T19:55:19.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T19:55:19.156 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T19:55:19.199 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T19:55:19.231 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T19:55:19.240 INFO:teuthology.orchestra.run.smithi131.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-26T19:55:19.259 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T19:55:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 613 kB/s | 107 kB 00:00 2024-04-26T19:55:19.328 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T19:55:19.332 INFO:teuthology.orchestra.run.smithi131.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 236 kB/s | 21 kB 00:00 2024-04-26T19:55:19.374 INFO:teuthology.orchestra.run.smithi131.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-26T19:55:19.381 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T19:55:19.399 INFO:teuthology.orchestra.run.smithi131.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 33 kB 00:00 2024-04-26T19:55:19.412 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T19:55:19.433 INFO:teuthology.orchestra.run.smithi131.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-04-26T19:55:19.447 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T19:55:19.475 INFO:teuthology.orchestra.run.smithi131.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 244 kB/s | 18 kB 00:00 2024-04-26T19:55:19.500 INFO:teuthology.orchestra.run.smithi131.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 168 kB/s | 11 kB 00:00 2024-04-26T19:55:19.523 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T19:55:19.542 INFO:teuthology.orchestra.run.smithi131.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-26T19:55:19.558 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T19:55:19.570 INFO:teuthology.orchestra.run.smithi131.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 662 kB/s | 45 kB 00:00 2024-04-26T19:55:19.601 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T19:55:19.630 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T19:55:19.701 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T19:55:19.731 INFO:teuthology.orchestra.run.smithi131.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-26T19:55:19.745 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T19:55:19.770 INFO:teuthology.orchestra.run.smithi131.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-04-26T19:55:19.796 INFO:teuthology.orchestra.run.smithi131.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 403 kB/s | 25 kB 00:00 2024-04-26T19:55:19.819 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T19:55:19.830 INFO:teuthology.orchestra.run.smithi131.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 821 kB/s | 47 kB 00:00 2024-04-26T19:55:19.913 INFO:teuthology.orchestra.run.smithi131.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-04-26T19:55:19.947 INFO:teuthology.orchestra.run.smithi131.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-26T19:55:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T19:55:19.980 INFO:teuthology.orchestra.run.smithi131.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-26T19:55:20.007 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T19:55:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 249 kB/s | 35 kB 00:00 2024-04-26T19:55:20.439 INFO:teuthology.orchestra.run.smithi131.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-04-26T19:55:20.551 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T19:55:20.598 INFO:teuthology.orchestra.run.smithi131.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.3 MB/s | 4.7 MB 00:00 2024-04-26T19:55:20.628 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T19:55:20.656 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T19:55:20.657 INFO:teuthology.orchestra.run.smithi131.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-26T19:55:20.690 INFO:teuthology.orchestra.run.smithi131.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-04-26T19:55:20.699 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T19:55:20.737 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T19:55:20.742 INFO:teuthology.orchestra.run.smithi148.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 702 kB/s | 1.6 MB 00:02 2024-04-26T19:55:20.779 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T19:55:20.790 INFO:teuthology.orchestra.run.smithi131.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-04-26T19:55:20.849 INFO:teuthology.orchestra.run.smithi131.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T19:55:20.940 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T19:55:20.966 INFO:teuthology.orchestra.run.smithi131.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.7 MB/s | 1.0 MB 00:00 2024-04-26T19:55:21.007 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T19:55:21.062 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T19:55:21.066 INFO:teuthology.orchestra.run.smithi131.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-26T19:55:21.136 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T19:55:21.166 INFO:teuthology.orchestra.run.smithi131.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-26T19:55:21.174 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T19:55:21.202 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T19:55:21.225 INFO:teuthology.orchestra.run.smithi131.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 445 kB/s | 26 kB 00:00 2024-04-26T19:55:21.249 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T19:55:21.284 INFO:teuthology.orchestra.run.smithi131.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 333 kB/s | 19 kB 00:00 2024-04-26T19:55:21.285 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T19:55:21.328 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T19:55:21.370 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T19:55:21.418 INFO:teuthology.orchestra.run.smithi131.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 7.6 MB/s | 5.8 MB 00:00 2024-04-26T19:55:21.451 INFO:teuthology.orchestra.run.smithi131.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-04-26T19:55:21.476 INFO:teuthology.orchestra.run.smithi131.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 613 kB/s | 35 kB 00:00 2024-04-26T19:55:21.508 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T19:55:21.535 INFO:teuthology.orchestra.run.smithi131.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-26T19:55:21.550 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T19:55:21.580 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T19:55:21.593 INFO:teuthology.orchestra.run.smithi131.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-26T19:55:21.622 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T19:55:21.652 INFO:teuthology.orchestra.run.smithi131.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T19:55:21.702 INFO:teuthology.orchestra.run.smithi131.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 856 kB/s | 42 kB 00:00 2024-04-26T19:55:21.717 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T19:55:21.755 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T19:55:21.795 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T19:55:21.821 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T19:55:21.846 INFO:teuthology.orchestra.run.smithi131.stdout:(85/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.0 MB/s | 6.0 MB 00:00 2024-04-26T19:55:21.846 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T19:55:21.887 INFO:teuthology.orchestra.run.smithi131.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-04-26T19:55:21.920 INFO:teuthology.orchestra.run.smithi131.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T19:55:21.921 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T19:55:21.954 INFO:teuthology.orchestra.run.smithi131.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-26T19:55:21.971 INFO:teuthology.orchestra.run.smithi131.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T19:55:22.013 INFO:teuthology.orchestra.run.smithi131.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 935 kB/s | 54 kB 00:00 2024-04-26T19:55:22.050 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T19:55:22.072 INFO:teuthology.orchestra.run.smithi131.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 720 kB/s | 42 kB 00:00 2024-04-26T19:55:22.079 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T19:55:22.106 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T19:55:22.140 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T19:55:22.147 INFO:teuthology.orchestra.run.smithi131.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-04-26T19:55:22.180 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T19:55:22.212 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T19:55:22.214 INFO:teuthology.orchestra.run.smithi131.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-26T19:55:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T19:55:22.276 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T19:55:22.289 INFO:teuthology.orchestra.run.smithi131.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 484 kB/s | 36 kB 00:00 2024-04-26T19:55:22.309 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T19:55:22.348 INFO:teuthology.orchestra.run.smithi131.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 824 kB/s | 47 kB 00:00 2024-04-26T19:55:22.351 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T19:55:22.408 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T19:55:22.415 INFO:teuthology.orchestra.run.smithi131.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T19:55:22.467 INFO:teuthology.orchestra.run.smithi148.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 528 kB/s | 911 kB 00:01 2024-04-26T19:55:22.473 INFO:teuthology.orchestra.run.smithi131.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-26T19:55:22.498 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T19:55:22.541 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T19:55:22.565 INFO:teuthology.orchestra.run.smithi131.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-04-26T19:55:22.580 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T19:55:22.624 INFO:teuthology.orchestra.run.smithi131.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T19:55:22.627 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T19:55:22.726 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T19:55:22.741 INFO:teuthology.orchestra.run.smithi131.stdout:(100/152): python3-numpy-1.20.1-5.el9.x86_64.rp 4.0 MB/s | 5.1 MB 00:01 2024-04-26T19:55:22.766 INFO:teuthology.orchestra.run.smithi131.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 317 kB/s | 45 kB 00:00 2024-04-26T19:55:22.782 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T19:55:22.908 INFO:teuthology.orchestra.run.smithi131.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-04-26T19:55:23.042 INFO:teuthology.orchestra.run.smithi131.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-04-26T19:55:23.209 INFO:teuthology.orchestra.run.smithi148.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 180 kB/s | 133 kB 00:00 2024-04-26T19:55:23.492 INFO:teuthology.orchestra.run.smithi148.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 825 kB/s | 4.4 MB 00:05 2024-04-26T19:55:23.517 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T19:55:23.570 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T19:55:23.595 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T19:55:23.695 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T19:55:23.754 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T19:55:23.809 INFO:teuthology.orchestra.run.smithi131.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.8 MB/s | 16 MB 00:01 2024-04-26T19:55:23.841 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T19:55:23.868 INFO:teuthology.orchestra.run.smithi131.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 374 kB/s | 309 kB 00:00 2024-04-26T19:55:23.881 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T19:55:24.043 INFO:teuthology.orchestra.run.smithi131.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 483 kB/s | 548 kB 00:01 2024-04-26T19:55:24.059 INFO:teuthology.orchestra.run.smithi148.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 795 kB/s | 4.7 MB 00:06 2024-04-26T19:55:24.063 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T19:55:24.095 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T19:55:24.173 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T19:55:24.235 INFO:teuthology.orchestra.run.smithi131.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 136 kB/s | 26 kB 00:00 2024-04-26T19:55:24.277 INFO:teuthology.orchestra.run.smithi131.stdout:(108/152): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-04-26T19:55:24.427 INFO:teuthology.orchestra.run.smithi131.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 254 kB/s | 49 kB 00:00 2024-04-26T19:55:24.653 INFO:teuthology.orchestra.run.smithi131.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-26T19:55:24.686 INFO:teuthology.orchestra.run.smithi131.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 586 kB/s | 151 kB 00:00 2024-04-26T19:55:24.701 INFO:teuthology.orchestra.run.smithi148.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 136 kB/s | 87 kB 00:00 2024-04-26T19:55:24.712 INFO:teuthology.orchestra.run.smithi131.stdout:(112/152): libunwind-1.6.2-1.el9.x86_64.rpm 155 kB/s | 67 kB 00:00 2024-04-26T19:55:24.818 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T19:55:24.846 INFO:teuthology.orchestra.run.smithi131.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-04-26T19:55:24.887 INFO:teuthology.orchestra.run.smithi131.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-04-26T19:55:24.927 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T19:55:24.929 INFO:teuthology.orchestra.run.smithi131.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-26T19:55:25.038 INFO:teuthology.orchestra.run.smithi131.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 168 kB/s | 32 kB 00:00 2024-04-26T19:55:25.080 INFO:teuthology.orchestra.run.smithi131.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 78 kB/s | 15 kB 00:00 2024-04-26T19:55:25.113 INFO:teuthology.orchestra.run.smithi131.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 936 kB/s | 172 kB 00:00 2024-04-26T19:55:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-26T19:55:25.234 INFO:teuthology.orchestra.run.smithi148.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 593 kB/s | 1.0 MB 00:01 2024-04-26T19:55:25.306 INFO:teuthology.orchestra.run.smithi131.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 184 kB/s | 35 kB 00:00 2024-04-26T19:55:25.415 INFO:teuthology.orchestra.run.smithi131.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-04-26T19:55:25.532 INFO:teuthology.orchestra.run.smithi131.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 672 kB/s | 303 kB 00:00 2024-04-26T19:55:25.600 INFO:teuthology.orchestra.run.smithi131.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 785 kB/s | 144 kB 00:00 2024-04-26T19:55:25.682 INFO:teuthology.orchestra.run.smithi131.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-04-26T19:55:25.733 INFO:teuthology.orchestra.run.smithi131.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 53 kB/s | 11 kB 00:00 2024-04-26T19:55:25.783 INFO:teuthology.orchestra.run.smithi131.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 97 kB/s | 18 kB 00:00 2024-04-26T19:55:25.866 INFO:teuthology.orchestra.run.smithi131.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 126 kB/s | 23 kB 00:00 2024-04-26T19:55:25.917 INFO:teuthology.orchestra.run.smithi131.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 106 kB/s | 19 kB 00:00 2024-04-26T19:55:25.967 INFO:teuthology.orchestra.run.smithi131.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 107 kB/s | 20 kB 00:00 2024-04-26T19:55:26.068 INFO:teuthology.orchestra.run.smithi148.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 293 kB/s | 244 kB 00:00 2024-04-26T19:55:26.075 INFO:teuthology.orchestra.run.smithi131.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-26T19:55:26.126 INFO:teuthology.orchestra.run.smithi131.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2024-04-26T19:55:26.151 INFO:teuthology.orchestra.run.smithi131.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 430 kB/s | 79 kB 00:00 2024-04-26T19:55:26.179 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T19:55:26.260 INFO:teuthology.orchestra.run.smithi131.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 315 kB/s | 58 kB 00:00 2024-04-26T19:55:26.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T19:55:26.295 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T19:55:26.328 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T19:55:26.335 INFO:teuthology.orchestra.run.smithi131.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-04-26T19:55:26.410 INFO:teuthology.orchestra.run.smithi131.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 960 kB/s | 272 kB 00:00 2024-04-26T19:55:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 449 kB/s | 90 kB 00:00 2024-04-26T19:55:26.511 INFO:teuthology.orchestra.run.smithi131.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 161 kB/s | 28 kB 00:00 2024-04-26T19:55:26.587 INFO:teuthology.orchestra.run.smithi131.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 175 kB/s | 31 kB 00:00 2024-04-26T19:55:26.629 INFO:teuthology.orchestra.run.smithi131.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-04-26T19:55:26.640 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T19:55:26.687 INFO:teuthology.orchestra.run.smithi131.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 338 kB/s | 59 kB 00:00 2024-04-26T19:55:26.690 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:55:26.763 INFO:teuthology.orchestra.run.smithi131.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 204 kB/s | 36 kB 00:00 2024-04-26T19:55:26.781 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:55:26.812 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T19:55:26.813 INFO:teuthology.orchestra.run.smithi131.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 278 kB/s | 51 kB 00:00 2024-04-26T19:55:26.848 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T19:55:26.851 INFO:teuthology.orchestra.run.smithi148.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 317 kB/s | 248 kB 00:00 2024-04-26T19:55:26.885 INFO:teuthology.orchestra.run.smithi131.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-04-26T19:55:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 540 kB/s | 90 kB 00:00 2024-04-26T19:55:26.997 INFO:teuthology.orchestra.run.smithi131.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-04-26T19:55:27.056 INFO:teuthology.orchestra.run.smithi131.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 117 kB/s | 20 kB 00:00 2024-04-26T19:55:27.089 INFO:teuthology.orchestra.run.smithi131.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-26T19:55:27.331 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T19:55:27.332 INFO:teuthology.orchestra.run.smithi131.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-04-26T19:55:27.376 INFO:teuthology.orchestra.run.smithi148.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 49 kB/s | 26 kB 00:00 2024-04-26T19:55:27.418 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:55:27.472 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:55:27.503 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T19:55:27.877 INFO:teuthology.orchestra.run.smithi148.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-04-26T19:55:28.065 INFO:teuthology.orchestra.run.smithi131.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.4 MB/s | 3.4 MB 00:01 2024-04-26T19:55:28.332 INFO:teuthology.orchestra.run.smithi131.stdout:(150/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-04-26T19:55:28.652 INFO:teuthology.orchestra.run.smithi148.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 222 kB/s | 172 kB 00:00 2024-04-26T19:55:29.269 INFO:teuthology.orchestra.run.smithi148.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-04-26T19:55:29.424 INFO:teuthology.orchestra.run.smithi131.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 661 kB/s | 1.3 MB 00:02 2024-04-26T19:55:29.793 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:55:29.824 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:55:30.364 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:55:30.412 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:55:30.488 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:55:30.488 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T19:55:30.488 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T19:55:30.489 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:30.521 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:55:30.677 INFO:teuthology.orchestra.run.smithi148.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 792 kB/s | 5.8 MB 00:07 2024-04-26T19:55:31.703 INFO:teuthology.orchestra.run.smithi148.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 245 kB/s | 250 kB 00:01 2024-04-26T19:55:31.865 INFO:teuthology.orchestra.run.smithi131.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 645 kB/s | 2.4 MB 00:03 2024-04-26T19:55:31.871 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:55:31.871 INFO:teuthology.orchestra.run.smithi131.stdout:Total 9.3 MB/s | 222 MB 00:23 2024-04-26T19:55:32.161 INFO:teuthology.orchestra.run.smithi148.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 819 kB/s | 6.0 MB 00:07 2024-04-26T19:55:32.528 INFO:teuthology.orchestra.run.smithi148.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 269 kB/s | 222 kB 00:00 2024-04-26T19:55:32.786 INFO:teuthology.orchestra.run.smithi148.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 124 kB/s | 77 kB 00:00 2024-04-26T19:55:33.080 INFO:teuthology.orchestra.run.smithi148.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 77 kB/s | 42 kB 00:00 2024-04-26T19:55:33.622 INFO:teuthology.orchestra.run.smithi148.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 322 kB/s | 269 kB 00:00 2024-04-26T19:55:33.822 INFO:teuthology.orchestra.run.smithi148.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 214 kB/s | 159 kB 00:00 2024-04-26T19:55:34.431 INFO:teuthology.orchestra.run.smithi148.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 88 kB/s | 51 kB 00:00 2024-04-26T19:55:34.473 INFO:teuthology.orchestra.run.smithi148.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 328 kB/s | 279 kB 00:00 2024-04-26T19:55:34.725 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-04-26T19:55:34.984 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-04-26T19:55:34.985 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-04-26T19:55:34.998 INFO:teuthology.orchestra.run.smithi148.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 95 kB/s | 54 kB 00:00 2024-04-26T19:55:35.557 INFO:teuthology.orchestra.run.smithi148.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 75 kB/s | 42 kB 00:00 2024-04-26T19:55:35.715 INFO:teuthology.orchestra.run.smithi148.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 810 kB/s | 5.1 MB 00:06 2024-04-26T19:55:36.132 INFO:teuthology.orchestra.run.smithi148.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 44 kB 00:00 2024-04-26T19:55:36.508 INFO:teuthology.orchestra.run.smithi148.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 279 kB/s | 221 kB 00:00 2024-04-26T19:55:36.691 INFO:teuthology.orchestra.run.smithi148.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 64 kB/s | 36 kB 00:00 2024-04-26T19:55:37.050 INFO:teuthology.orchestra.run.smithi148.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 87 kB/s | 47 kB 00:00 2024-04-26T19:55:37.550 INFO:teuthology.orchestra.run.smithi148.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 355 kB/s | 305 kB 00:00 2024-04-26T19:55:37.625 INFO:teuthology.orchestra.run.smithi148.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 100 kB/s | 58 kB 00:00 2024-04-26T19:55:37.670 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-04-26T19:55:37.672 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-04-26T19:55:38.225 INFO:teuthology.orchestra.run.smithi148.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 139 kB/s | 94 kB 00:00 2024-04-26T19:55:38.251 INFO:teuthology.orchestra.run.smithi148.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-04-26T19:55:38.409 INFO:teuthology.orchestra.run.smithi148.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-04-26T19:55:38.626 INFO:teuthology.orchestra.run.smithi148.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-04-26T19:55:38.776 INFO:teuthology.orchestra.run.smithi148.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 81 kB/s | 45 kB 00:00 2024-04-26T19:55:39.102 INFO:teuthology.orchestra.run.smithi148.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-26T19:55:39.127 INFO:teuthology.orchestra.run.smithi148.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 883 kB/s | 309 kB 00:00 2024-04-26T19:55:39.178 INFO:teuthology.orchestra.run.smithi148.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-26T19:55:39.327 INFO:teuthology.orchestra.run.smithi148.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 174 kB/s | 26 kB 00:00 2024-04-26T19:55:39.386 INFO:teuthology.orchestra.run.smithi148.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-26T19:55:39.411 INFO:teuthology.orchestra.run.smithi148.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-04-26T19:55:39.461 INFO:teuthology.orchestra.run.smithi148.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-04-26T19:55:39.487 INFO:teuthology.orchestra.run.smithi148.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-26T19:55:39.545 INFO:teuthology.orchestra.run.smithi148.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-04-26T19:55:39.571 INFO:teuthology.orchestra.run.smithi148.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-26T19:55:39.621 INFO:teuthology.orchestra.run.smithi148.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 584 kB/s | 43 kB 00:00 2024-04-26T19:55:39.655 INFO:teuthology.orchestra.run.smithi148.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-04-26T19:55:39.697 INFO:teuthology.orchestra.run.smithi148.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-26T19:55:39.739 INFO:teuthology.orchestra.run.smithi148.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-26T19:55:39.772 INFO:teuthology.orchestra.run.smithi148.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-26T19:55:39.814 INFO:teuthology.orchestra.run.smithi148.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-26T19:55:39.848 INFO:teuthology.orchestra.run.smithi148.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 471 kB/s | 35 kB 00:00 2024-04-26T19:55:39.890 INFO:teuthology.orchestra.run.smithi148.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-26T19:55:40.032 INFO:teuthology.orchestra.run.smithi148.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-26T19:55:40.057 INFO:teuthology.orchestra.run.smithi148.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 865 kB/s | 144 kB 00:00 2024-04-26T19:55:40.099 INFO:teuthology.orchestra.run.smithi148.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-04-26T19:55:40.124 INFO:teuthology.orchestra.run.smithi148.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-26T19:55:40.166 INFO:teuthology.orchestra.run.smithi148.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-26T19:55:40.200 INFO:teuthology.orchestra.run.smithi148.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-26T19:55:40.233 INFO:teuthology.orchestra.run.smithi148.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-04-26T19:55:40.292 INFO:teuthology.orchestra.run.smithi148.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-26T19:55:40.309 INFO:teuthology.orchestra.run.smithi148.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-04-26T19:55:40.367 INFO:teuthology.orchestra.run.smithi148.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-26T19:55:40.384 INFO:teuthology.orchestra.run.smithi148.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-26T19:55:40.443 INFO:teuthology.orchestra.run.smithi148.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-26T19:55:40.485 INFO:teuthology.orchestra.run.smithi148.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-26T19:55:40.519 INFO:teuthology.orchestra.run.smithi148.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T19:55:40.552 INFO:teuthology.orchestra.run.smithi148.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-26T19:55:40.578 INFO:teuthology.orchestra.run.smithi148.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-04-26T19:55:40.628 INFO:teuthology.orchestra.run.smithi148.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T19:55:40.645 INFO:teuthology.orchestra.run.smithi148.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-04-26T19:55:40.686 INFO:teuthology.orchestra.run.smithi148.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 614 kB/s | 36 kB 00:00 2024-04-26T19:55:40.712 INFO:teuthology.orchestra.run.smithi148.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 760 kB/s | 51 kB 00:00 2024-04-26T19:55:40.762 INFO:teuthology.orchestra.run.smithi148.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T19:55:40.788 INFO:teuthology.orchestra.run.smithi148.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T19:55:40.838 INFO:teuthology.orchestra.run.smithi148.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-26T19:55:40.863 INFO:teuthology.orchestra.run.smithi148.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-26T19:55:40.913 INFO:teuthology.orchestra.run.smithi148.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-26T19:55:40.972 INFO:teuthology.orchestra.run.smithi148.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-26T19:55:41.934 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:55:41.934 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-04-26T19:55:41.934 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-04-26T19:55:41.934 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-04-26T19:55:41.934 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-04-26T19:55:41.934 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-04-26T19:55:41.935 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-04-26T19:55:41.935 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-04-26T19:55:41.935 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:42.099 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:55:42.115 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:55:42.115 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:42.115 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T19:55:42.115 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:42.514 INFO:teuthology.orchestra.run.smithi148.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-04-26T19:55:42.698 INFO:teuthology.orchestra.run.smithi148.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-04-26T19:55:42.806 INFO:teuthology.orchestra.run.smithi148.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-26T19:55:43.215 INFO:teuthology.orchestra.run.smithi148.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 1.5 MB/s | 3.4 MB 00:02 2024-04-26T19:55:43.628 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:55:43.654 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:55:43.654 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:43.654 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T19:55:43.654 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:55:43.654 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:55:43.654 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:43.879 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:55:43.901 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:55:43.902 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:43.902 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T19:55:43.902 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:55:43.902 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:55:43.902 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:44.346 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:55:44.369 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:55:44.369 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:44.369 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T19:55:44.369 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:55:44.369 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:55:44.369 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:44.463 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-04-26T19:55:44.477 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T19:55:44.534 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T19:55:44.589 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T19:55:44.632 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T19:55:44.712 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T19:55:44.778 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T19:55:44.822 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T19:55:44.836 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T19:55:44.892 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T19:55:44.901 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T19:55:44.932 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T19:55:44.991 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:55:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T19:55:45.132 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:55:45.180 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T19:55:45.212 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T19:55:45.240 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T19:55:45.464 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T19:55:45.572 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:55:45.600 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:55:45.630 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:55:45.671 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T19:55:45.733 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T19:55:45.746 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:55:45.762 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:55:45.781 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:55:45.909 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:55:45.974 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:55:46.360 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T19:55:46.429 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T19:55:46.452 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:55:46.602 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:55:46.637 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:55:46.731 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:55:46.763 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:55:46.811 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:55:46.843 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:55:46.847 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:55:47.212 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:55:47.272 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:55:47.471 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T19:55:47.599 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T19:55:47.656 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T19:55:47.705 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T19:55:47.808 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:55:47.811 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T19:55:47.839 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:55:47.868 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T19:55:47.932 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T19:55:47.942 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:55:47.990 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T19:55:48.030 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T19:55:48.100 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T19:55:48.124 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:55:48.147 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:55:48.147 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:48.147 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T19:55:48.147 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:55:48.147 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:55:48.148 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:48.148 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T19:55:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:55:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:55:48.220 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T19:55:48.282 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T19:55:48.308 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T19:55:48.341 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T19:55:48.357 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T19:55:48.452 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T19:55:48.494 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T19:55:48.551 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T19:55:48.585 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T19:55:48.704 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T19:55:48.770 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T19:55:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T19:55:49.054 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T19:55:49.141 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T19:55:49.279 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T19:55:49.311 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:55:49.334 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:55:49.334 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:49.334 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T19:55:49.334 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:55:49.334 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:55:49.334 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:49.378 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:55:49.401 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:55:49.401 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:49.401 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T19:55:49.401 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:49.445 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:55:49.468 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:55:49.468 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:49.468 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T19:55:49.469 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:55:49.469 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:55:49.469 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:49.771 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:55:49.794 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:55:49.795 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:55:49.795 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T19:55:49.795 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:55:49.795 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:55:49.795 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:55:49.848 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T19:55:49.966 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T19:55:50.007 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T19:55:50.054 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T19:55:50.104 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T19:55:50.168 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T19:55:50.428 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T19:55:50.520 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T19:55:50.603 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T19:55:50.701 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T19:55:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T19:55:50.819 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T19:55:50.889 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T19:55:50.935 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T19:55:50.998 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T19:55:51.061 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T19:55:51.262 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T19:55:51.315 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T19:55:51.352 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T19:55:51.408 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T19:55:51.537 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T19:55:51.619 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T19:55:51.678 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T19:55:51.719 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T19:55:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T19:55:51.948 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T19:55:52.139 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T19:55:52.192 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T19:55:52.237 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T19:55:52.300 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T19:55:52.367 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T19:55:52.424 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T19:55:52.488 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T19:55:52.534 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T19:55:52.584 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T19:55:52.661 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T19:55:52.706 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T19:55:52.847 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T19:55:52.912 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T19:55:52.956 INFO:teuthology.orchestra.run.smithi148.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 900 kB/s | 16 MB 00:18 2024-04-26T19:55:52.956 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T19:55:52.961 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:55:52.961 INFO:teuthology.orchestra.run.smithi148.stdout:Total 4.1 MB/s | 222 MB 00:54 2024-04-26T19:55:53.026 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T19:55:53.173 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T19:55:53.252 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T19:55:54.017 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-26T19:55:54.160 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-26T19:55:54.160 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-26T19:55:54.255 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T19:55:54.311 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T19:55:54.344 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T19:55:54.383 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T19:55:54.433 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T19:55:54.469 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T19:55:54.474 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T19:55:54.506 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T19:55:54.533 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T19:55:54.572 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T19:55:54.681 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T19:55:54.723 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T19:55:54.729 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T19:55:54.771 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T19:55:54.801 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T19:55:54.964 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T19:55:55.001 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T19:55:55.120 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T19:55:55.395 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:55:55.432 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:55:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:55:55.946 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T19:55:56.002 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-26T19:55:56.002 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-26T19:55:56.091 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T19:55:56.297 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T19:55:56.387 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T19:55:56.428 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T19:55:56.511 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T19:55:56.541 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T19:55:56.541 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:55:56.633 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:55:56.658 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:55:56.658 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:55:56.749 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:55:56.823 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:55:57.703 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T19:55:57.810 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T19:55:57.857 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T19:55:57.915 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T19:55:58.337 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T19:55:58.381 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:55:58.487 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:55:58.520 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T19:55:58.577 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T19:55:59.161 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T19:55:59.200 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:55:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:55:59.327 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T19:56:02.020 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-26T19:56:02.083 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:56:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T19:56:02.132 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:56:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T19:56:02.209 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T19:56:02.243 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T19:56:02.277 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T19:56:02.320 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T19:56:02.625 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:02.625 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T19:56:02.625 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T19:56:02.625 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T19:56:02.625 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T19:56:02.625 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T19:56:02.625 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T19:56:02.626 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T19:56:02.627 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T19:56:02.639 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T19:56:02.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T19:56:02.641 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T19:56:02.642 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T19:56:02.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T19:56:02.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T19:56:02.645 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T19:56:02.646 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T19:56:02.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T19:56:02.648 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T19:56:02.648 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T19:56:02.648 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T19:56:02.648 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:56:02.648 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T19:56:02.648 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T19:56:02.753 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:56:02.794 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:56:02.845 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:56:02.866 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:56:02.895 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:56:02.970 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:56:02.999 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:56:03.071 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:56:03.071 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T19:56:03.071 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T19:56:03.071 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:03.113 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:56:03.152 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T19:56:03.205 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T19:56:03.233 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:56:03.336 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:56:03.358 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:56:03.421 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:56:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:56:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:56:03.668 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:56:03.692 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:56:03.800 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T19:56:03.904 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T19:56:03.941 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T19:56:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T19:56:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T19:56:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T19:56:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T19:56:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T19:56:04.179 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T19:56:04.275 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T19:56:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T19:56:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T19:56:04.435 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T19:56:04.512 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T19:56:04.679 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.680 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:04.681 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T19:56:04.683 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T19:56:04.684 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T19:56:04.685 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T19:56:04.686 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T19:56:04.687 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T19:56:04.688 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T19:56:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T19:56:04.690 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T19:56:04.690 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T19:56:04.690 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-26T19:56:04.718 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T19:56:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T19:56:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T19:56:04.863 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T19:56:04.907 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T19:56:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T19:56:05.069 DEBUG:teuthology.parallel:result is None 2024-04-26T19:56:05.127 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T19:56:05.184 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T19:56:05.716 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T19:56:05.797 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T19:56:05.834 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T19:56:05.868 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T19:56:05.906 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T19:56:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T19:56:06.112 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T19:56:06.168 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T19:56:06.223 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T19:56:06.296 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T19:56:06.327 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T19:56:06.402 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T19:56:06.402 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T19:56:06.437 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T19:56:06.480 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T19:56:06.523 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T19:56:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T19:56:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T19:56:06.733 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T19:56:06.774 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T19:56:06.869 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T19:56:06.908 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T19:56:06.948 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T19:56:06.974 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T19:56:06.999 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T19:56:07.074 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T19:56:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T19:56:07.231 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T19:56:07.259 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T19:56:07.293 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T19:56:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T19:56:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T19:56:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T19:56:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T19:56:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T19:56:07.503 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T19:56:07.538 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T19:56:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T19:56:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T19:56:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T19:56:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T19:56:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T19:56:08.633 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T19:56:08.681 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T19:56:08.715 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T19:56:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T19:56:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T19:56:08.968 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T19:56:09.000 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T19:56:09.179 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T19:56:09.207 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T19:56:09.284 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T19:56:09.934 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T19:56:10.046 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T19:56:11.279 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T19:56:11.360 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T19:56:11.379 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T19:56:11.405 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T19:56:11.716 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T19:56:11.758 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:56:11.848 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:56:11.880 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T19:56:11.916 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T19:56:12.394 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T19:56:12.420 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:56:12.474 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:56:12.505 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T19:56:14.735 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:56:14.851 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:56:15.399 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:56:15.438 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:56:15.519 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:56:15.519 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T19:56:15.519 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T19:56:15.519 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:15.547 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:56:16.216 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-04-26T19:56:16.217 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:16.464 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:56:16.512 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:56:16.512 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:16.512 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T19:56:16.512 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:18.133 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:56:18.210 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:56:18.210 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:18.210 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T19:56:18.210 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:56:18.210 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:56:18.210 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:18.533 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:56:18.604 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:56:18.604 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:18.604 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T19:56:18.604 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:56:18.604 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:56:18.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:19.117 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:56:19.186 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:56:19.186 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:19.186 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T19:56:19.186 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:56:19.186 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:56:19.186 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:19.320 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T19:56:19.381 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T19:56:19.456 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T19:56:19.691 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T19:56:19.747 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T19:56:19.832 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T19:56:19.966 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:56:20.133 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:56:20.189 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T19:56:20.230 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T19:56:20.262 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T19:56:20.922 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:56:20.967 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:56:21.025 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T19:56:21.110 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T19:56:21.169 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:56:22.487 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:56:22.529 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:56:23.634 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:56:23.670 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:56:23.817 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:56:23.996 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T19:56:24.023 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:56:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:56:24.095 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:24.095 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T19:56:24.096 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:56:24.096 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:56:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:24.157 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:56:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:56:24.323 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T19:56:24.385 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T19:56:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T19:56:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:56:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:56:25.511 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:25.511 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T19:56:25.511 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:56:25.511 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:56:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:25.561 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:56:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:56:25.630 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:25.630 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T19:56:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:56:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:56:25.789 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:25.789 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T19:56:25.789 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:56:25.789 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:56:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:56:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:56:26.258 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:26.258 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T19:56:26.258 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:56:26.258 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:56:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-04-26T19:56:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:27.200 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:56:27.215 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:56:27.215 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:27.215 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T19:56:27.215 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:56:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:56:28.712 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:28.712 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T19:56:28.712 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:56:28.712 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:56:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:56:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:56:28.967 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:28.967 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T19:56:28.967 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:56:28.967 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:56:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:29.404 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:56:29.426 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:56:29.426 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:29.426 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T19:56:29.426 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:56:29.426 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:56:29.426 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:29.526 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T19:56:29.583 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T19:56:29.647 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T19:56:29.833 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T19:56:29.881 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T19:56:29.958 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T19:56:30.046 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:56:30.185 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:56:30.222 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T19:56:30.254 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T19:56:30.281 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T19:56:30.675 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:56:30.705 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:56:30.737 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T19:56:30.815 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T19:56:30.845 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:56:31.305 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T19:56:31.360 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T19:56:31.442 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T19:56:31.492 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T19:56:31.540 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T19:56:31.883 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:56:31.913 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:56:31.925 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T19:56:31.984 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T19:56:32.037 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T19:56:32.870 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:56:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:56:32.968 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:56:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:56:33.011 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:56:33.074 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:56:33.176 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T19:56:33.198 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:56:33.220 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:56:33.221 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:33.221 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T19:56:33.221 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:56:33.221 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:56:33.221 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:33.266 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:56:33.277 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:56:33.374 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T19:56:33.431 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T19:56:34.351 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T19:56:34.377 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:56:34.399 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:56:34.399 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:34.399 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T19:56:34.399 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:56:34.399 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:56:34.399 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:34.416 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T19:56:34.444 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:56:34.465 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:56:34.465 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:34.466 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T19:56:34.466 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:34.511 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:56:34.533 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:56:34.533 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:34.533 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T19:56:34.534 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:56:34.534 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:56:34.534 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:34.550 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T19:56:34.638 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T19:56:34.838 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:56:34.843 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T19:56:34.860 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:56:34.860 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:56:34.860 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T19:56:34.860 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:56:34.860 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:56:34.860 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:34.884 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T19:56:34.884 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:56:35.042 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:56:35.083 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:56:35.083 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:56:35.223 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:56:35.356 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:35.652 INFO:teuthology.orchestra.run.smithi131.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T19:56:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T19:56:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T19:56:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T19:56:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T19:56:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T19:56:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T19:56:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T19:56:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T19:56:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:56:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:56:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:56:41.271 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T19:56:41.343 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T19:56:41.384 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T19:56:41.467 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T19:56:41.497 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T19:56:41.497 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:56:41.581 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:56:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:56:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T19:56:41.669 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T19:56:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T19:56:41.671 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T19:56:41.671 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T19:56:41.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T19:56:41.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T19:56:41.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T19:56:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T19:56:41.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T19:56:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T19:56:41.687 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T19:56:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T19:56:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T19:56:41.690 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T19:56:41.691 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T19:56:41.692 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:56:41.692 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T19:56:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:56:41.771 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:44.153 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:44.153 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.154 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.155 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T19:56:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.157 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T19:56:44.158 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.159 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T19:56:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T19:56:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.163 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:56:44.164 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-04-26T19:56:44.817 DEBUG:teuthology.parallel:result is None 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T19:56:47.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T19:56:47.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T19:56:47.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T19:56:47.838 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T19:56:47.838 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T19:56:47.838 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T19:56:47.838 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T19:56:47.839 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T19:56:47.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T19:56:47.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T19:56:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T19:56:47.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T19:56:47.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T19:56:47.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T19:56:47.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T19:56:47.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T19:56:47.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T19:56:47.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:56:47.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T19:56:49.910 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:56:49.910 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:49.910 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.911 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T19:56:49.913 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.914 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T19:56:49.915 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.916 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T19:56:49.917 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T19:56:49.918 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.919 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T19:56:49.920 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T19:56:49.921 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T19:56:49.922 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T19:56:49.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-26T19:56:49.922 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-26T19:56:50.279 DEBUG:teuthology.parallel:result is None 2024-04-26T19:56:50.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:56:50.426 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T19:56:50.481 INFO:teuthology.orchestra.run.smithi131.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T19:56:50.482 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T19:56:50.482 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T19:56:50.484 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:56:50.703 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T19:56:50.725 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T19:56:50.725 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T19:56:50.726 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T19:56:50.728 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:56:50.947 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T19:56:50.973 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T19:56:50.973 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T19:56:50.973 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T19:56:50.975 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T19:56:50.976 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:56:50.976 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T19:56:51.046 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:56:51.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T19:56:51.071 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:56:51.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T19:56:51.102 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T19:56:51.103 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:56:51.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T19:56:51.163 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T19:56:51.268 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:56:51.269 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T19:56:51.292 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T19:56:51.358 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:56:51.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T19:56:51.388 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T19:56:51.455 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T19:56:51.455 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:56:51.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T19:56:51.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T19:56:51.622 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:56:51.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T19:56:51.645 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T19:56:51.709 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:56:51.709 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T19:56:51.733 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T19:56:51.797 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T19:56:51.797 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:56:51.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T19:56:51.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T19:56:51.976 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:56:51.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T19:56:52.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T19:56:52.064 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:56:52.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T19:56:52.088 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T19:56:52.153 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T19:56:52.253 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2}} 2024-04-26T19:56:52.253 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:56:52.254 INFO:tasks.cephadm:Cluster fsid is 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:56:52.254 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T19:56:52.254 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.131', 'mon.b': '172.21.15.148', 'mon.c': '172.21.15.157'} 2024-04-26T19:56:52.254 INFO:tasks.cephadm:First mon is mon.a on smithi131 2024-04-26T19:56:52.255 INFO:tasks.cephadm:First mgr is x 2024-04-26T19:56:52.255 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T19:56:52.255 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-04-26T19:56:52.338 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-04-26T19:56:52.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-04-26T19:56:52.386 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T19:56:52.386 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:56:52.608 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T19:56:52.742 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:56:52.744 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T19:56:52.744 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:54.245 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 19:56 /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:54.246 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:54.961 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 19:56 /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:54.961 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:55.532 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 19:56 /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:55.532 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:55.595 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:55.613 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T19:56:55.647 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T19:56:55.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T19:56:55.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T19:56:55.657 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T19:56:57.273 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T19:56:57.285 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T19:56:58.139 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T19:57:34.396 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T19:57:34.397 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T19:57:34.397 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T19:57:34.397 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-04-26T19:57:34.397 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T19:57:34.397 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T19:57:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T19:57:40.176 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-26T19:57:40.176 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T19:57:40.176 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T19:57:40.176 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-04-26T19:57:40.176 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T19:57:40.176 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-04-26T19:57:40.176 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-26T19:57:53.172 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-26T19:57:53.172 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T19:57:53.173 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T19:57:53.173 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-04-26T19:57:53.173 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T19:57:53.173 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-26T19:57:53.173 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-26T19:57:53.196 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-04-26T19:57:53.288 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-04-26T19:57:53.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-04-26T19:57:53.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-04-26T19:57:53.433 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-04-26T19:57:53.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-04-26T19:57:53.486 INFO:tasks.cephadm:Writing seed config... 2024-04-26T19:57:53.487 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T19:57:53.488 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T19:57:53.489 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T19:57:53.490 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T19:57:53.491 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T19:57:53.491 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:57:53.491 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T19:57:53.547 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2027e7ca-0407-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T19:57:53.547 DEBUG:teuthology.orchestra.run.smithi131:mon.a> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T19:57:53.552 DEBUG:teuthology.orchestra.run.smithi131:mgr.x> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T19:57:53.598 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T19:57:53.598 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.131 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T19:57:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:57:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', '2027e7ca-0407-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.131', '--skip-admin-label'] 2024-04-26T19:57:53.870 INFO:teuthology.orchestra.run.smithi131.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T19:57:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying podman|docker is present... 2024-04-26T19:57:53.932 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T19:57:53.932 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying lvm2 is present... 2024-04-26T19:57:53.932 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying time synchronization is in place... 2024-04-26T19:57:53.950 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T19:57:53.950 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T19:57:53.965 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T19:57:53.965 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-04-26T19:57:53.982 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-04-26T19:57:54.000 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-04-26T19:57:54.000 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-04-26T19:57:54.000 INFO:teuthology.orchestra.run.smithi131.stdout:Repeating the final host check... 2024-04-26T19:57:54.059 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T19:57:54.059 INFO:teuthology.orchestra.run.smithi131.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T19:57:54.060 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl is present 2024-04-26T19:57:54.060 INFO:teuthology.orchestra.run.smithi131.stdout:lvcreate is present 2024-04-26T19:57:54.073 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T19:57:54.073 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T19:57:54.089 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T19:57:54.089 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-04-26T19:57:54.109 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-04-26T19:57:54.130 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-04-26T19:57:54.131 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-04-26T19:57:54.131 INFO:teuthology.orchestra.run.smithi131.stdout:Host looks OK 2024-04-26T19:57:54.131 INFO:teuthology.orchestra.run.smithi131.stdout:Cluster fsid: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:57:54.132 INFO:teuthology.orchestra.run.smithi131.stdout:Acquiring lock 140163055742112 on /run/cephadm/2027e7ca-0407-11ef-bc93-c7b262605968.lock 2024-04-26T19:57:54.132 INFO:teuthology.orchestra.run.smithi131.stdout:Lock 140163055742112 acquired on /run/cephadm/2027e7ca-0407-11ef-bc93-c7b262605968.lock 2024-04-26T19:57:54.133 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 3300 ... 2024-04-26T19:57:54.134 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 6789 ... 2024-04-26T19:57:54.135 INFO:teuthology.orchestra.run.smithi131.stdout:Base mon IP(s) is [172.21.15.131:3300, 172.21.15.131:6789], mon addrv is [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-04-26T19:57:54.149 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.131 metric 100 2024-04-26T19:57:54.149 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.131 metric 100 2024-04-26T19:57:54.160 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T19:57:54.160 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-26T19:57:54.160 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1780sec hoplimit 64 pref medium 2024-04-26T19:57:54.171 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T19:57:54.171 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T19:57:54.171 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T19:57:54.171 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T19:57:54.171 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-04-26T19:57:54.171 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T19:57:54.174 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-04-26T19:57:54.174 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-04-26T19:57:54.174 INFO:teuthology.orchestra.run.smithi131.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T19:57:54.176 INFO:teuthology.orchestra.run.smithi131.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T19:57:54.178 INFO:teuthology.orchestra.run.smithi131.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T19:57:54.858 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T19:57:54.858 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T19:57:54.858 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T19:57:54.858 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T19:57:54.858 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T19:57:54.858 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T19:57:54.858 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T19:57:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T19:57:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T19:57:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T19:57:57.255 INFO:teuthology.orchestra.run.smithi131.stdout:stat: stdout 167 167 2024-04-26T19:57:57.255 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial keys... 2024-04-26T19:57:58.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQBFByxmKA+BLBAAevJvdCr0nI1G/8mnx/Dvmg== 2024-04-26T19:57:59.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQBGByxmBLanNBAAMDTkSU/No+psoG/Y/mISaw== 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQBHByxmDQDxOhAAqhKJfBkiSYZuiaiJrWfitw== 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial monmap... 2024-04-26T19:58:01.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T19:58:01.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool for a [v2:172.21.15.131:3300,v1:172.21.15.131:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = quincy 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: set fsid to 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:58:01.938 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mon... 2024-04-26T19:58:03.105 INFO:teuthology.orchestra.run.smithi131.stdout:create mon.a on 2024-04-26T19:58:03.514 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T19:58:03.764 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T19:58:04.074 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2027e7ca-0407-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-2027e7ca-0407-11ef-bc93-c7b262605968.target. 2024-04-26T19:58:04.074 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2027e7ca-0407-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-2027e7ca-0407-11ef-bc93-c7b262605968.target. 2024-04-26T19:58:04.487 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a 2024-04-26T19:58:04.487 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T19:58:04.782 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2027e7ca-0407-11ef-bc93-c7b262605968.target.wants/ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2027e7ca-0407-11ef-bc93-c7b262605968@.service. 2024-04-26T19:58:05.792 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T19:58:05.792 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-04-26T19:58:05.806 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T19:58:05.806 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-04-26T19:58:05.806 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-04-26T19:58:05.806 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T19:58:05.807 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon to start... 2024-04-26T19:58:05.807 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon... 2024-04-26T19:58:05.946 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:05 smithi131 ceph-mon[27554]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T19:58:07.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T19:58:07.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: monmap epoch 1 2024-04-26T19:58:07.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:07.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: last_changed 2024-04-26T19:58:01.203692+0000 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: min_mon_release 19 (squid) 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: election_strategy: 1 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: fsmap 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: mgrmap e1: no daemons active 2024-04-26T19:58:07.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:06 smithi131 ceph-mon[27554]: from='client.? 172.21.15.131:0/1950577506' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T19:58:07.187 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T19:58:07.188 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout id: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:07.188 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T19:58:07.188 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:07.188 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout services: 2024-04-26T19:58:07.189 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.742421s) 2024-04-26T19:58:07.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout data: 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.418+0000 7f7d87000700 1 Processor -- start 2024-04-26T19:58:07.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.418+0000 7f7d87000700 1 -- start start 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.419+0000 7f7d87000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8010b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.419+0000 7f7d87000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8010b7a0 con 0x7f7d8010ae40 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.420+0000 7f7d7fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8010b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.420+0000 7f7d7fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8010b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56680/0 (socket says 172.21.15.131:56680) 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.420+0000 7f7d7fe00700 1 -- 172.21.15.131:0/360856591 learned_addr learned my addr 172.21.15.131:0/360856591 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.421+0000 7f7d7fe00700 1 -- 172.21.15.131:0/360856591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8010bfa0 con 0x7f7d8010ae40 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.421+0000 7f7d7fe00700 1 --2- 172.21.15.131:0/360856591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8010b260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d700098a0 tx=0x7f7d70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e792f8c1263d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.421+0000 7f7d7ea00700 1 -- 172.21.15.131:0/360856591 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d70004160 con 0x7f7d8010ae40 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.421+0000 7f7d7ea00700 1 -- 172.21.15.131:0/360856591 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d70004300 con 0x7f7d8010ae40 2024-04-26T19:58:07.196 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.421+0000 7f7d7ea00700 1 -- 172.21.15.131:0/360856591 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d700036a0 con 0x7f7d8010ae40 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.422+0000 7f7d87000700 1 -- 172.21.15.131:0/360856591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 msgr2=0x7f7d8010b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.422+0000 7f7d87000700 1 --2- 172.21.15.131:0/360856591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8010b260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d700098a0 tx=0x7f7d70009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.422+0000 7f7d87000700 1 -- 172.21.15.131:0/360856591 shutdown_connections 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.422+0000 7f7d87000700 1 --2- 172.21.15.131:0/360856591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8010b260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.422+0000 7f7d87000700 1 -- 172.21.15.131:0/360856591 >> 172.21.15.131:0/360856591 conn(0x7f7d8007e230 msgr2=0x7f7d8007e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.422+0000 7f7d87000700 1 -- 172.21.15.131:0/360856591 shutdown_connections 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.422+0000 7f7d87000700 1 -- 172.21.15.131:0/360856591 wait complete. 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.423+0000 7f7d87000700 1 Processor -- start 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.423+0000 7f7d87000700 1 -- start start 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.424+0000 7f7d87000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.424+0000 7f7d87000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8010c470 con 0x7f7d8010ae40 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.424+0000 7f7d7fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.424+0000 7f7d7fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56686/0 (socket says 172.21.15.131:56686) 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.424+0000 7f7d7fe00700 1 -- 172.21.15.131:0/1950577506 learned_addr learned my addr 172.21.15.131:0/1950577506 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.424+0000 7f7d7fe00700 1 -- 172.21.15.131:0/1950577506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8019ea60 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d7fe00700 1 --2- 172.21.15.131:0/1950577506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8019e520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009fb0 tx=0x7f7d70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d7cc00700 1 -- 172.21.15.131:0/1950577506 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d70003de0 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d7cc00700 1 -- 172.21.15.131:0/1950577506 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d70011630 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8019ecf0 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d7cc00700 1 -- 172.21.15.131:0/1950577506 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d70011930 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d801a19e0 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d7cc00700 1 -- 172.21.15.131:0/1950577506 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d70024480 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.425+0000 7f7d7cc00700 1 -- 172.21.15.131:0/1950577506 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d7001a7e0 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.426+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d80064a80 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.429+0000 7f7d7cc00700 1 -- 172.21.15.131:0/1950577506 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7d7001a9e0 con 0x7f7d8010ae40 2024-04-26T19:58:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.503+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7d8002e5e0 con 0x7f7d8010ae40 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.503+0000 7f7d7cc00700 1 -- 172.21.15.131:0/1950577506 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7d7001abc0 con 0x7f7d8010ae40 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.504+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 msgr2=0x7f7d8019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.504+0000 7f7d87000700 1 --2- 172.21.15.131:0/1950577506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8019e520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009fb0 tx=0x7f7d70004750 comp rx=0 tx=0).stop 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.505+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 shutdown_connections 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.505+0000 7f7d87000700 1 --2- 172.21.15.131:0/1950577506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8010ae40 0x7f7d8019e520 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.505+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 >> 172.21.15.131:0/1950577506 conn(0x7f7d8007e230 msgr2=0x7f7d80195040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.505+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 shutdown_connections 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:06.505+0000 7f7d87000700 1 -- 172.21.15.131:0/1950577506 wait complete. 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:mon is available 2024-04-26T19:58:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T19:58:08.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:07 smithi131 ceph-mon[27554]: from='client.? 172.21.15.131:0/2609290127' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T19:58:08.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:07 smithi131 ceph-mon[27554]: from='client.? 172.21.15.131:0/2609290127' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T19:58:08.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.785+0000 7f69cd200700 1 Processor -- start 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.785+0000 7f69cd200700 1 -- start start 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.785+0000 7f69cd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8107f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.785+0000 7f69cd200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8108550 con 0x7f69c8107b60 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.786+0000 7f69c5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:08.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.786+0000 7f69c5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56688/0 (socket says 172.21.15.131:56688) 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.786+0000 7f69c5e00700 1 -- 172.21.15.131:0/2657657487 learned_addr learned my addr 172.21.15.131:0/2657657487 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.786+0000 7f69c5e00700 1 -- 172.21.15.131:0/2657657487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c8108d60 con 0x7f69c8107b60 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.786+0000 7f69c5e00700 1 --2- 172.21.15.131:0/2657657487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8107f80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f69b80098a0 tx=0x7f69b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b735662e48bc1e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69c4a00700 1 -- 172.21.15.131:0/2657657487 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b8004160 con 0x7f69c8107b60 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69c4a00700 1 -- 172.21.15.131:0/2657657487 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b8004300 con 0x7f69c8107b60 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69c4a00700 1 -- 172.21.15.131:0/2657657487 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b80036a0 con 0x7f69c8107b60 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69cd200700 1 -- 172.21.15.131:0/2657657487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 msgr2=0x7f69c8107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69cd200700 1 --2- 172.21.15.131:0/2657657487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8107f80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f69b80098a0 tx=0x7f69b8009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69cd200700 1 -- 172.21.15.131:0/2657657487 shutdown_connections 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69cd200700 1 --2- 172.21.15.131:0/2657657487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8107f80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.787+0000 7f69cd200700 1 -- 172.21.15.131:0/2657657487 >> 172.21.15.131:0/2657657487 conn(0x7f69c8103120 msgr2=0x7f69c8105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.788+0000 7f69cd200700 1 -- 172.21.15.131:0/2657657487 shutdown_connections 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.788+0000 7f69cd200700 1 -- 172.21.15.131:0/2657657487 wait complete. 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.789+0000 7f69cd200700 1 Processor -- start 2024-04-26T19:58:08.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.789+0000 7f69cd200700 1 -- start start 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.789+0000 7f69cd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8190720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.789+0000 7f69cd200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8109290 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.789+0000 7f69c5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8190720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.789+0000 7f69c5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8190720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56696/0 (socket says 172.21.15.131:56696) 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.790+0000 7f69c5e00700 1 -- 172.21.15.131:0/2609290127 learned_addr learned my addr 172.21.15.131:0/2609290127 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.790+0000 7f69c5e00700 1 -- 172.21.15.131:0/2609290127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c8190c60 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.790+0000 7f69c5e00700 1 --2- 172.21.15.131:0/2609290127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8190720 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f69b8000c00 tx=0x7f69b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.790+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b8003db0 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.790+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b8011630 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.790+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b8011950 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.790+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c818ee50 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.791+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c818f390 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.791+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f69b8011af0 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.791+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f69b801c450 con 0x7f69c8107b60 2024-04-26T19:58:08.507 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.792+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c8064a80 con 0x7f69c8107b60 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.795+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f69b8032020 con 0x7f69c8107b60 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.871+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f69c81958b0 con 0x7f69c8107b60 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.880+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f69b8004260 con 0x7f69c8107b60 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.881+0000 7f69bea00700 1 -- 172.21.15.131:0/2609290127 <== mon.0 v2:172.21.15.131:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f69b801c6a0 con 0x7f69c8107b60 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.882+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 msgr2=0x7f69c8190720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.882+0000 7f69cd200700 1 --2- 172.21.15.131:0/2609290127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8190720 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f69b8000c00 tx=0x7f69b8004750 comp rx=0 tx=0).stop 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.883+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 shutdown_connections 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.883+0000 7f69cd200700 1 --2- 172.21.15.131:0/2609290127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8107b60 0x7f69c8190720 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.883+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 >> 172.21.15.131:0/2609290127 conn(0x7f69c8103120 msgr2=0x7f69c8192fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.883+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 shutdown_connections 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:07.883+0000 7f69cd200700 1 -- 172.21.15.131:0/2609290127 wait complete. 2024-04-26T19:58:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:Generating new minimal ceph.conf... 2024-04-26T19:58:09.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:09 smithi131 ceph-mon[27554]: from='client.? 172.21.15.131:0/3292008945' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:58:09.936 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.122+0000 7f043e800700 1 Processor -- start 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.122+0000 7f043e800700 1 -- start start 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.123+0000 7f043e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f043807d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.123+0000 7f043e800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f043807d930 con 0x7f043807ef00 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.123+0000 7f0437400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f043807d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.123+0000 7f0437400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f043807d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56702/0 (socket says 172.21.15.131:56702) 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.123+0000 7f0437400700 1 -- 172.21.15.131:0/2032860672 learned_addr learned my addr 172.21.15.131:0/2032860672 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.124+0000 7f0437400700 1 -- 172.21.15.131:0/2032860672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043807dab0 con 0x7f043807ef00 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.124+0000 7f0437400700 1 --2- 172.21.15.131:0/2032860672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f043807d360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f04280098a0 tx=0x7f0428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d0a7729200abc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:09.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.124+0000 7f0436a00700 1 -- 172.21.15.131:0/2032860672 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0428004160 con 0x7f043807ef00 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.124+0000 7f0436a00700 1 -- 172.21.15.131:0/2032860672 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0428004300 con 0x7f043807ef00 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.124+0000 7f0436a00700 1 -- 172.21.15.131:0/2032860672 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0428011af0 con 0x7f043807ef00 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.125+0000 7f043e800700 1 -- 172.21.15.131:0/2032860672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 msgr2=0x7f043807d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.125+0000 7f043e800700 1 --2- 172.21.15.131:0/2032860672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f043807d360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f04280098a0 tx=0x7f0428009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.125+0000 7f043e800700 1 -- 172.21.15.131:0/2032860672 shutdown_connections 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.125+0000 7f043e800700 1 --2- 172.21.15.131:0/2032860672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f043807d360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.125+0000 7f043e800700 1 -- 172.21.15.131:0/2032860672 >> 172.21.15.131:0/2032860672 conn(0x7f04381041b0 msgr2=0x7f0438106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.126+0000 7f043e800700 1 -- 172.21.15.131:0/2032860672 shutdown_connections 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.126+0000 7f043e800700 1 -- 172.21.15.131:0/2032860672 wait complete. 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.127+0000 7f043e800700 1 Processor -- start 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.127+0000 7f043e800700 1 -- start start 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.127+0000 7f043e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f04381a3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.127+0000 7f043e800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f043810ade0 con 0x7f043807ef00 2024-04-26T19:58:09.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.127+0000 7f0437400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f04381a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.127+0000 7f0437400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f04381a3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56710/0 (socket says 172.21.15.131:56710) 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.127+0000 7f0437400700 1 -- 172.21.15.131:0/3292008945 learned_addr learned my addr 172.21.15.131:0/3292008945 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.128+0000 7f0437400700 1 -- 172.21.15.131:0/3292008945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04381a3970 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.128+0000 7f0437400700 1 --2- 172.21.15.131:0/3292008945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f04381a3430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f0428011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.128+0000 7f0434c00700 1 -- 172.21.15.131:0/3292008945 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0428003b20 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.128+0000 7f0434c00700 1 -- 172.21.15.131:0/3292008945 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f042801a8c0 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.128+0000 7f0434c00700 1 -- 172.21.15.131:0/3292008945 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0428024940 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.129+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04381a3c00 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.129+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381a68f0 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.129+0000 7f0434c00700 1 -- 172.21.15.131:0/3292008945 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0428022020 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.129+0000 7f0434c00700 1 -- 172.21.15.131:0/3292008945 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f042801a430 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.130+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0438064a80 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.133+0000 7f0434c00700 1 -- 172.21.15.131:0/3292008945 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f042802dde0 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.210+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f043802e490 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.211+0000 7f0434c00700 1 -- 172.21.15.131:0/3292008945 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f042801a630 con 0x7f043807ef00 2024-04-26T19:58:09.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.212+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 msgr2=0x7f04381a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:09.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.212+0000 7f043e800700 1 --2- 172.21.15.131:0/3292008945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f04381a3430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f0428011b70 comp rx=0 tx=0).stop 2024-04-26T19:58:09.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.213+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 shutdown_connections 2024-04-26T19:58:09.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.213+0000 7f043e800700 1 --2- 172.21.15.131:0/3292008945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043807ef00 0x7f04381a3430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:09.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.213+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 >> 172.21.15.131:0/3292008945 conn(0x7f04381041b0 msgr2=0x7f0438199f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:09.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.213+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 shutdown_connections 2024-04-26T19:58:09.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:09.213+0000 7f043e800700 1 -- 172.21.15.131:0/3292008945 wait complete. 2024-04-26T19:58:09.940 INFO:teuthology.orchestra.run.smithi131.stdout:Restarting the monitor... 2024-04-26T19:58:10.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:09 smithi131 systemd[1]: Stopping Ceph mon.a for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T19:58:10.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a[27549]: 2024-04-26T19:58:10.398+0000 7f94dd600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T19:58:10.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a[27549]: 2024-04-26T19:58:10.398+0000 7f94dd600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T19:58:10.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:10 smithi131 podman[27797]: 2024-04-26 19:58:10.460146131 +0000 UTC m=+0.194752880 container died 21a8b9afa5f5d2507409b2fb9fcf8b9e0e5031b42bb6c93d9ced0d433cf369c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T19:58:11.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:10 smithi131 podman[27797]: 2024-04-26 19:58:10.867122991 +0000 UTC m=+0.601729794 container cleanup 21a8b9afa5f5d2507409b2fb9fcf8b9e0e5031b42bb6c93d9ced0d433cf369c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T19:58:11.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:10 smithi131 bash[27797]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a 2024-04-26T19:58:11.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:11 smithi131 podman[27811]: 2024-04-26 19:58:11.168128115 +0000 UTC m=+0.690873194 container remove 21a8b9afa5f5d2507409b2fb9fcf8b9e0e5031b42bb6c93d9ced0d433cf369c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T19:58:11.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:11 smithi131 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T19:58:11.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:11 smithi131 systemd[1]: Stopped Ceph mon.a for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T19:58:11.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:11 smithi131 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.763s CPU time. 2024-04-26T19:58:11.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:11 smithi131 systemd[1]: Starting Ceph mon.a for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T19:58:12.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:11 smithi131 podman[27900]: 2024-04-26 19:58:11.687771915 +0000 UTC m=+0.036129348 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:58:12.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:11 smithi131 podman[27900]: 2024-04-26 19:58:11.794062677 +0000 UTC m=+0.142420138 container create fc567c89abfaf9b6235ff933556da3adaa11a5bc6eb86eece13c9c822b48b8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T19:58:12.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 podman[27900]: 2024-04-26 19:58:12.072685164 +0000 UTC m=+0.421042685 container init fc567c89abfaf9b6235ff933556da3adaa11a5bc6eb86eece13c9c822b48b8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T19:58:12.173 INFO:teuthology.orchestra.run.smithi131.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T19:58:12.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 podman[27900]: 2024-04-26 19:58:12.084346924 +0000 UTC m=+0.432704355 container start fc567c89abfaf9b6235ff933556da3adaa11a5bc6eb86eece13c9c822b48b8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T19:58:12.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T19:58:12.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T19:58:12.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: pidfile_write: ignore empty --pid-file 2024-04-26T19:58:12.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: load: jerasure load: lrc 2024-04-26T19:58:12.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: RocksDB version: 7.9.2 2024-04-26T19:58:12.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Git sha 0 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: DB SUMMARY 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: DB Session ID: LZIDPJESNZQVRXS6ZNAC 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: CURRENT file: CURRENT 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103556 ; 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.error_if_exists: 0 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.create_if_missing: 0 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.paranoid_checks: 1 2024-04-26T19:58:12.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T19:58:12.383 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T19:58:12.383 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T19:58:12.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.env: 0x55b072061cc0 2024-04-26T19:58:12.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T19:58:12.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.info_log: 0x55b0ac26bdc0 2024-04-26T19:58:12.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T19:58:12.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.statistics: (nil) 2024-04-26T19:58:12.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.use_fsync: 0 2024-04-26T19:58:12.385 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_log_file_size: 0 2024-04-26T19:58:12.385 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T19:58:12.385 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T19:58:12.385 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T19:58:12.385 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T19:58:12.385 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.allow_fallocate: 1 2024-04-26T19:58:12.386 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T19:58:12.386 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T19:58:12.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.use_direct_reads: 0 2024-04-26T19:58:12.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T19:58:12.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T19:58:12.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.db_log_dir: 2024-04-26T19:58:12.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.wal_dir: 2024-04-26T19:58:12.388 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T19:58:12.388 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T19:58:12.388 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T19:58:12.388 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T19:58:12.388 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.write_buffer_manager: 0x55b0ac23d0e0 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T19:58:12.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.unordered_write: 0 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.row_cache: None 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.wal_filter: None 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.two_write_queues: 0 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.wal_compression: 0 2024-04-26T19:58:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.atomic_flush: 0 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.log_readahead_size: 0 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_background_jobs: 2 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_background_compactions: -1 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_subcompactions: 1 2024-04-26T19:58:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_open_files: -1 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T19:58:12.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_background_flushes: -1 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Compression algorithms supported: 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kZSTD supported: 0 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kXpressCompression supported: 0 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kZlibCompression supported: 1 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kSnappyCompression supported: 1 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kLZ4Compression supported: 1 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: kBZip2Compression supported: 0 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T19:58:12.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.merge_operator: 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_filter: None 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b0ac26bf80) 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_type: 0 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_index_type: 0 2024-04-26T19:58:12.394 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_shortening: 1 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: checksum: 4 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: no_block_cache: 0 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache: 0x55b0ac26f1f0 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_options: 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: capacity : 536870912 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_shard_bits : 4 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: strict_capacity_limit : 0 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_compressed: (nil) 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: persistent_cache: (nil) 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size: 4096 2024-04-26T19:58:12.395 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size_deviation: 10 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_restart_interval: 16 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_block_restart_interval: 1 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: metadata_block_size: 4096 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: partition_filters: 0 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: use_delta_encoding: 1 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: filter_policy: bloomfilter 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: whole_key_filtering: 1 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: verify_compression: 0 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: format_version: 5 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_index_compression: 1 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_align: 0 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: max_auto_readahead_size: 262144 2024-04-26T19:58:12.396 INFO:journalctl@ceph.mon.a.smithi131.stdout: prepopulate_block_cache: 0 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout: initial_auto_readahead_size: 8192 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression: NoCompression 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.num_levels: 7 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T19:58:12.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T19:58:12.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T19:58:12.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T19:58:12.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T19:58:12.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.inplace_update_support: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.bloom_locality: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.max_successive_merges: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.ttl: 2592000 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T19:58:12.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.enable_blob_files: false 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.min_blob_size: 0 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T19:58:12.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8bd09c26-5151-4a72-af98-a607ff7971d2 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714161492141011, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714161492142907, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98809, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96762, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13786, "raw_average_key_size": 49, "raw_value_size": 89245, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714161492, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8bd09c26-5151-4a72-af98-a607ff7971d2", "db_session_id": "LZIDPJESNZQVRXS6ZNAC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714161492143183, "job": 1, "event": "recovery_finished"} 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b0ac38c000 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: DB pointer 0x55b0ac378000 2024-04-26T19:58:12.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** DB Stats ** 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 98.35 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T19:58:12.405 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 98.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T19:58:12.406 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T19:58:12.406 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-04-26T19:58:12.406 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-04-26T19:58:12.406 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T19:58:12.406 INFO:journalctl@ceph.mon.a.smithi131.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T19:58:12.406 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 82.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T19:58:12.407 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-04-26T19:58:12.407 INFO:journalctl@ceph.mon.a.smithi131.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T19:58:12.407 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-04-26T19:58:12.407 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T19:58:12.407 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T19:58:12.407 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T19:58:12.408 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T19:58:12.408 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T19:58:12.408 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T19:58:12.408 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 14.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T19:58:12.408 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 14.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T19:58:12.408 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T19:58:12.408 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache BinnedLRUCache@0x55b0ac26f1f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-04-26T19:58:12.410 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T19:58:12.410 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-04-26T19:58:12.410 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T19:58:12.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: starting mon.a rank 0 at public addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] at bind addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:12.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???) e1 preinit fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:12.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???).mds e1 new map 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???).mds e1 print_map 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout: e1 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout: legacy client fscid: -1 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout: No filesystems configured 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 bash[27900]: fc567c89abfaf9b6235ff933556da3adaa11a5bc6eb86eece13c9c822b48b8c3 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 systemd[1]: Started Ceph mon.a for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T19:58:12.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: monmap epoch 1 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: last_changed 2024-04-26T19:58:01.203692+0000 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: min_mon_release 19 (squid) 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: election_strategy: 1 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: fsmap 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T19:58:12.412 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:12 smithi131 ceph-mon[27919]: mgrmap e1: no daemons active 2024-04-26T19:58:13.664 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.897+0000 7f3a15000700 1 Processor -- start 2024-04-26T19:58:13.664 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.897+0000 7f3a15000700 1 -- start start 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.898+0000 7f3a15000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1010a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.898+0000 7f3a15000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10108860 con 0x7f3a1010a320 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.898+0000 7f3a0de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1010a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.898+0000 7f3a0de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1010a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56716/0 (socket says 172.21.15.131:56716) 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.898+0000 7f3a0de00700 1 -- 172.21.15.131:0/120631414 learned_addr learned my addr 172.21.15.131:0/120631414 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.899+0000 7f3a0de00700 1 -- 172.21.15.131:0/120631414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a101089e0 con 0x7f3a1010a320 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.899+0000 7f3a0de00700 1 --2- 172.21.15.131:0/120631414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1010a740 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009e80 tx=0x7f3a0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa2aeb5c4710815 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.899+0000 7f3a0d400700 1 -- 172.21.15.131:0/120631414 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00017070 con 0x7f3a1010a320 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.899+0000 7f3a0d400700 1 -- 172.21.15.131:0/120631414 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a0000b8c0 con 0x7f3a1010a320 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.899+0000 7f3a0d400700 1 -- 172.21.15.131:0/120631414 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00004490 con 0x7f3a1010a320 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.900+0000 7f3a15000700 1 -- 172.21.15.131:0/120631414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 msgr2=0x7f3a1010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.900+0000 7f3a15000700 1 --2- 172.21.15.131:0/120631414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1010a740 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009e80 tx=0x7f3a0000b330 comp rx=0 tx=0).stop 2024-04-26T19:58:13.665 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.900+0000 7f3a15000700 1 -- 172.21.15.131:0/120631414 shutdown_connections 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.900+0000 7f3a15000700 1 --2- 172.21.15.131:0/120631414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1010a740 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.900+0000 7f3a15000700 1 -- 172.21.15.131:0/120631414 >> 172.21.15.131:0/120631414 conn(0x7f3a10104200 msgr2=0x7f3a10106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.900+0000 7f3a15000700 1 -- 172.21.15.131:0/120631414 shutdown_connections 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.900+0000 7f3a15000700 1 -- 172.21.15.131:0/120631414 wait complete. 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.901+0000 7f3a15000700 1 Processor -- start 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.902+0000 7f3a15000700 1 -- start start 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.902+0000 7f3a15000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.902+0000 7f3a15000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010d1f0 con 0x7f3a1010a320 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.902+0000 7f3a0de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.902+0000 7f3a0de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56722/0 (socket says 172.21.15.131:56722) 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.902+0000 7f3a0de00700 1 -- 172.21.15.131:0/4050710837 learned_addr learned my addr 172.21.15.131:0/4050710837 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:13.666 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.903+0000 7f3a0de00700 1 -- 172.21.15.131:0/4050710837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1019f670 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.903+0000 7f3a0de00700 1 --2- 172.21.15.131:0/4050710837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1019f130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3a00004a60 tx=0x7f3a0000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.903+0000 7f3a06a00700 1 -- 172.21.15.131:0/4050710837 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00020020 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.903+0000 7f3a06a00700 1 -- 172.21.15.131:0/4050710837 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a00003c60 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.903+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019f900 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.903+0000 7f3a06a00700 1 -- 172.21.15.131:0/4050710837 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00017040 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.903+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a101a25f0 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.904+0000 7f3a06a00700 1 -- 172.21.15.131:0/4050710837 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a00023020 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.904+0000 7f3a06a00700 1 -- 172.21.15.131:0/4050710837 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a0000f050 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.905+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a1010d370 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.908+0000 7f3a06a00700 1 -- 172.21.15.131:0/4050710837 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3a0002a020 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.983+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3a10076ec0 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.988+0000 7f3a06a00700 1 -- 172.21.15.131:0/4050710837 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3a10076ec0 con 0x7f3a1010a320 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.990+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 msgr2=0x7f3a1019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:13.667 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.990+0000 7f3a15000700 1 --2- 172.21.15.131:0/4050710837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1019f130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3a00004a60 tx=0x7f3a0000bd50 comp rx=0 tx=0).stop 2024-04-26T19:58:13.668 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.990+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 shutdown_connections 2024-04-26T19:58:13.668 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.990+0000 7f3a15000700 1 --2- 172.21.15.131:0/4050710837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a1010a320 0x7f3a1019f130 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:13.668 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.990+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 >> 172.21.15.131:0/4050710837 conn(0x7f3a10104200 msgr2=0x7f3a10104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:13.668 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.990+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 shutdown_connections 2024-04-26T19:58:13.668 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:12.990+0000 7f3a15000700 1 -- 172.21.15.131:0/4050710837 wait complete. 2024-04-26T19:58:13.668 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T19:58:13.689 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T19:58:13.690 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mgr... 2024-04-26T19:58:13.691 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T19:58:13.692 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T19:58:14.083 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.x 2024-04-26T19:58:14.083 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T19:58:14.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:13 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4050710837' entity='client.admin' 2024-04-26T19:58:14.340 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2027e7ca-0407-11ef-bc93-c7b262605968.target.wants/ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-2027e7ca-0407-11ef-bc93-c7b262605968@.service. 2024-04-26T19:58:14.748 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:14 smithi131 systemd[1]: Starting Ceph mgr.x for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T19:58:15.007 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:14 smithi131 podman[28131]: 2024-04-26 19:58:14.651575447 +0000 UTC m=+0.033538331 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:58:15.007 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:14 smithi131 podman[28131]: 2024-04-26 19:58:14.75024454 +0000 UTC m=+0.132207461 container create c98006765fd693dd1d58feb06380dd486d24644f57b108ee9b2170fd1b88999b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T19:58:15.007 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:14 smithi131 podman[28131]: 2024-04-26 19:58:14.995635099 +0000 UTC m=+0.377598030 container init c98006765fd693dd1d58feb06380dd486d24644f57b108ee9b2170fd1b88999b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T19:58:15.007 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:15 smithi131 podman[28131]: 2024-04-26 19:58:15.005730553 +0000 UTC m=+0.387693481 container start c98006765fd693dd1d58feb06380dd486d24644f57b108ee9b2170fd1b88999b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T19:58:15.119 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T19:58:15.119 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-04-26T19:58:15.137 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T19:58:15.137 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-04-26T19:58:15.137 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-04-26T19:58:15.137 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T19:58:15.166 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T19:58:15.166 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-04-26T19:58:15.186 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T19:58:15.186 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-04-26T19:58:15.187 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-04-26T19:58:15.187 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T19:58:15.187 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr to start... 2024-04-26T19:58:15.187 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr... 2024-04-26T19:58:15.335 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:15 smithi131 bash[28131]: c98006765fd693dd1d58feb06380dd486d24644f57b108ee9b2170fd1b88999b 2024-04-26T19:58:15.336 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:15 smithi131 systemd[1]: Started Ceph mgr.x for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T19:58:15.336 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:15.254+0000 7fc53fecd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:58:15.667 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:15.523+0000 7fc53fecd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:58:15.668 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:15.666+0000 7fc53fecd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:58:16.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:16 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/68383929' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T19:58:16.220 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:16 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:16.102+0000 7fc53fecd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:58:16.220 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:16 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:16.219+0000 7fc53fecd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:58:16.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:16.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:58:16.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T19:58:16.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T19:58:16.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T19:58:16.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T19:58:16.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T19:58:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:16.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T19:58:16.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:16.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T19:58:16.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T19:58:16.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T19:58:16.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:16.885 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:16.886 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T19:58:05.765390+0000", 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.031+0000 7ffa68000700 1 Processor -- start 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.032+0000 7ffa68000700 1 -- start start 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.032+0000 7ffa68000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6007d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.032+0000 7ffa68000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6007f050 con 0x7ffa6007d320 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.033+0000 7ffa65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.033+0000 7ffa65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6007d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57556/0 (socket says 172.21.15.131:57556) 2024-04-26T19:58:16.887 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.033+0000 7ffa65600700 1 -- 172.21.15.131:0/2578828133 learned_addr learned my addr 172.21.15.131:0/2578828133 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.033+0000 7ffa65600700 1 -- 172.21.15.131:0/2578828133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6007dc80 con 0x7ffa6007d320 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.034+0000 7ffa65600700 1 --2- 172.21.15.131:0/2578828133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6007d740 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098a0 tx=0x7ffa54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55a20456a9359be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.034+0000 7ffa5fe00700 1 -- 172.21.15.131:0/2578828133 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa54004160 con 0x7ffa6007d320 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.034+0000 7ffa5fe00700 1 -- 172.21.15.131:0/2578828133 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffa54004300 con 0x7ffa6007d320 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.034+0000 7ffa5fe00700 1 -- 172.21.15.131:0/2578828133 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa54011af0 con 0x7ffa6007d320 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.034+0000 7ffa68000700 1 -- 172.21.15.131:0/2578828133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 msgr2=0x7ffa6007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.035+0000 7ffa68000700 1 --2- 172.21.15.131:0/2578828133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6007d740 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098a0 tx=0x7ffa54009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.035+0000 7ffa68000700 1 -- 172.21.15.131:0/2578828133 shutdown_connections 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.035+0000 7ffa68000700 1 --2- 172.21.15.131:0/2578828133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6007d740 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.035+0000 7ffa68000700 1 -- 172.21.15.131:0/2578828133 >> 172.21.15.131:0/2578828133 conn(0x7ffa60104200 msgr2=0x7ffa60106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.035+0000 7ffa68000700 1 -- 172.21.15.131:0/2578828133 shutdown_connections 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.035+0000 7ffa68000700 1 -- 172.21.15.131:0/2578828133 wait complete. 2024-04-26T19:58:16.888 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.036+0000 7ffa68000700 1 Processor -- start 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.036+0000 7ffa68000700 1 -- start start 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.036+0000 7ffa68000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.036+0000 7ffa68000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6010f310 con 0x7ffa6007d320 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.037+0000 7ffa65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.037+0000 7ffa65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57564/0 (socket says 172.21.15.131:57564) 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.037+0000 7ffa65600700 1 -- 172.21.15.131:0/68383929 learned_addr learned my addr 172.21.15.131:0/68383929 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.037+0000 7ffa65600700 1 -- 172.21.15.131:0/68383929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6019f670 con 0x7ffa6007d320 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.037+0000 7ffa65600700 1 --2- 172.21.15.131:0/68383929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6019f130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffa54011ac0 tx=0x7ffa54011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.038+0000 7ffa5e000700 1 -- 172.21.15.131:0/68383929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa54003b20 con 0x7ffa6007d320 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.038+0000 7ffa5e000700 1 -- 172.21.15.131:0/68383929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffa5401a8c0 con 0x7ffa6007d320 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.038+0000 7ffa5e000700 1 -- 172.21.15.131:0/68383929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa540248d0 con 0x7ffa6007d320 2024-04-26T19:58:16.889 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.038+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6019f900 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.038+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601a25a0 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.038+0000 7ffa5e000700 1 -- 172.21.15.131:0/68383929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa54022020 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.038+0000 7ffa5e000700 1 -- 172.21.15.131:0/68383929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa5402dde0 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.039+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa60064a80 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.044+0000 7ffa5e000700 1 -- 172.21.15.131:0/68383929 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ffa54042ec0 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.125+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffa60076e80 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.126+0000 7ffa5e000700 1 -- 172.21.15.131:0/68383929 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ffa54031020 con 0x7ffa6007d320 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.127+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 msgr2=0x7ffa6019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.127+0000 7ffa68000700 1 --2- 172.21.15.131:0/68383929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6019f130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffa54011ac0 tx=0x7ffa54011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.127+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 shutdown_connections 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.127+0000 7ffa68000700 1 --2- 172.21.15.131:0/68383929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa6007d320 0x7ffa6019f130 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:16.890 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.127+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 >> 172.21.15.131:0/68383929 conn(0x7ffa60104200 msgr2=0x7ffa60104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:16.891 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.127+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 shutdown_connections 2024-04-26T19:58:16.891 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:16.127+0000 7ffa68000700 1 -- 172.21.15.131:0/68383929 wait complete. 2024-04-26T19:58:16.891 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (1/15)... 2024-04-26T19:58:18.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:17 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:17.815+0000 7fc53fecd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:58:18.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:18.042+0000 7fc53fecd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:58:18.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:18.257+0000 7fc53fecd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:58:19.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:18.639+0000 7fc53fecd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:58:19.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:18.760+0000 7fc53fecd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:58:19.300 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:19 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:19.031+0000 7fc53fecd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:58:19.577 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:19 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:19.298+0000 7fc53fecd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:58:19.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:19 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2033439223' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T19:58:20.245 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:20.116+0000 7fc53fecd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:58:20.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:20.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:58:20.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T19:58:20.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T19:58:20.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T19:58:20.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T19:58:20.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T19:58:05.765390+0000", 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.487+0000 7fea22000700 1 Processor -- start 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.488+0000 7fea22000700 1 -- start start 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.488+0000 7fea22000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c07d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:20.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.488+0000 7fea22000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c07f030 con 0x7fea1c07d300 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.488+0000 7fea1b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c07d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.489+0000 7fea1b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c07d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57570/0 (socket says 172.21.15.131:57570) 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.489+0000 7fea1b400700 1 -- 172.21.15.131:0/2673839207 learned_addr learned my addr 172.21.15.131:0/2673839207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.489+0000 7fea1b400700 1 -- 172.21.15.131:0/2673839207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c07dc60 con 0x7fea1c07d300 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.489+0000 7fea1b400700 1 --2- 172.21.15.131:0/2673839207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c07d720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0098a0 tx=0x7fea0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f29e3067154a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea1a000700 1 -- 172.21.15.131:0/2673839207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea0c004160 con 0x7fea1c07d300 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea1a000700 1 -- 172.21.15.131:0/2673839207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea0c004300 con 0x7fea1c07d300 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea1a000700 1 -- 172.21.15.131:0/2673839207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea0c011af0 con 0x7fea1c07d300 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea22000700 1 -- 172.21.15.131:0/2673839207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 msgr2=0x7fea1c07d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea22000700 1 --2- 172.21.15.131:0/2673839207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c07d720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0098a0 tx=0x7fea0c009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea22000700 1 -- 172.21.15.131:0/2673839207 shutdown_connections 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea22000700 1 --2- 172.21.15.131:0/2673839207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c07d720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:20.273 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.490+0000 7fea22000700 1 -- 172.21.15.131:0/2673839207 >> 172.21.15.131:0/2673839207 conn(0x7fea1c104200 msgr2=0x7fea1c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.491+0000 7fea22000700 1 -- 172.21.15.131:0/2673839207 shutdown_connections 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.491+0000 7fea22000700 1 -- 172.21.15.131:0/2673839207 wait complete. 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.492+0000 7fea22000700 1 Processor -- start 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.492+0000 7fea22000700 1 -- start start 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.492+0000 7fea22000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.492+0000 7fea22000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c10f330 con 0x7fea1c07d300 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.492+0000 7fea1b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.492+0000 7fea1b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57578/0 (socket says 172.21.15.131:57578) 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.492+0000 7fea1b400700 1 -- 172.21.15.131:0/2033439223 learned_addr learned my addr 172.21.15.131:0/2033439223 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.493+0000 7fea1b400700 1 -- 172.21.15.131:0/2033439223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c19f630 con 0x7fea1c07d300 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.493+0000 7fea1b400700 1 --2- 172.21.15.131:0/2033439223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c19f0f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea0c003f40 tx=0x7fea0c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.493+0000 7fea13e00700 1 -- 172.21.15.131:0/2033439223 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea0c0038b0 con 0x7fea1c07d300 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.493+0000 7fea13e00700 1 -- 172.21.15.131:0/2033439223 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea0c003a50 con 0x7fea1c07d300 2024-04-26T19:58:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.493+0000 7fea13e00700 1 -- 172.21.15.131:0/2033439223 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea0c01a5c0 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.493+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1c19f8c0 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.493+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1c1a25b0 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.494+0000 7fea13e00700 1 -- 172.21.15.131:0/2033439223 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea0c01ac00 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.494+0000 7fea13e00700 1 -- 172.21.15.131:0/2033439223 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fea0c019700 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.494+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1c064a80 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.497+0000 7fea13e00700 1 -- 172.21.15.131:0/2033439223 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fea0c0199a0 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.575+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fea1c02e490 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.576+0000 7fea13e00700 1 -- 172.21.15.131:0/2033439223 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fea0c019b80 con 0x7fea1c07d300 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.577+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 msgr2=0x7fea1c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.577+0000 7fea22000700 1 --2- 172.21.15.131:0/2033439223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c19f0f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea0c003f40 tx=0x7fea0c011890 comp rx=0 tx=0).stop 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.577+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 shutdown_connections 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.577+0000 7fea22000700 1 --2- 172.21.15.131:0/2033439223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c07d300 0x7fea1c19f0f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.577+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 >> 172.21.15.131:0/2033439223 conn(0x7fea1c104200 msgr2=0x7fea1c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:20.275 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.578+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 shutdown_connections 2024-04-26T19:58:20.276 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:19.578+0000 7fea22000700 1 -- 172.21.15.131:0/2033439223 wait complete. 2024-04-26T19:58:20.276 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (2/15)... 2024-04-26T19:58:21.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:20.667+0000 7fc53fecd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:58:21.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:20.797+0000 7fc53fecd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:58:21.506 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:21.060+0000 7fc53fecd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:58:22.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:21.719+0000 7fc53fecd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:58:22.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:21.843+0000 7fc53fecd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:58:22.423 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:22.161+0000 7fc53fecd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:58:22.424 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:22.348+0000 7fc53fecd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:58:22.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:22.477+0000 7fc53fecd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:58:22.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:22.631+0000 7fc53fecd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:58:23.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:22 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/856855658' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T19:58:23.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:23 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:23.019+0000 7fc53fecd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-04-26T19:58:23.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T19:58:23.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T19:58:23.627 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T19:58:23.628 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T19:58:05.765390+0000", 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.841+0000 7f8933400700 1 Processor -- start 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.841+0000 7f8933400700 1 -- start start 2024-04-26T19:58:23.629 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.842+0000 7f8933400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.842+0000 7f8933400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f892c07f050 con 0x7f892c07d320 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.842+0000 7f8930a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.842+0000 7f8930a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c07d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57584/0 (socket says 172.21.15.131:57584) 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.842+0000 7f8930a00700 1 -- 172.21.15.131:0/3788836474 learned_addr learned my addr 172.21.15.131:0/3788836474 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.843+0000 7f8930a00700 1 -- 172.21.15.131:0/3788836474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c07dc80 con 0x7f892c07d320 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.843+0000 7f8930a00700 1 --2- 172.21.15.131:0/3788836474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c07d740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f89200098a0 tx=0x7f8920009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e660e00e2c507e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.843+0000 7f892b400700 1 -- 172.21.15.131:0/3788836474 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8920004160 con 0x7f892c07d320 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.844+0000 7f892b400700 1 -- 172.21.15.131:0/3788836474 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8920004300 con 0x7f892c07d320 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.844+0000 7f892b400700 1 -- 172.21.15.131:0/3788836474 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8920011af0 con 0x7f892c07d320 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.844+0000 7f8933400700 1 -- 172.21.15.131:0/3788836474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 msgr2=0x7f892c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.844+0000 7f8933400700 1 --2- 172.21.15.131:0/3788836474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c07d740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f89200098a0 tx=0x7f8920009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.845+0000 7f8933400700 1 -- 172.21.15.131:0/3788836474 shutdown_connections 2024-04-26T19:58:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.845+0000 7f8933400700 1 --2- 172.21.15.131:0/3788836474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c07d740 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.845+0000 7f8933400700 1 -- 172.21.15.131:0/3788836474 >> 172.21.15.131:0/3788836474 conn(0x7f892c104200 msgr2=0x7f892c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.845+0000 7f8933400700 1 -- 172.21.15.131:0/3788836474 shutdown_connections 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.845+0000 7f8933400700 1 -- 172.21.15.131:0/3788836474 wait complete. 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.846+0000 7f8933400700 1 Processor -- start 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.846+0000 7f8933400700 1 -- start start 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.846+0000 7f8933400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.846+0000 7f8933400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f892c10f310 con 0x7f892c07d320 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.847+0000 7f8930a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.847+0000 7f8930a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57600/0 (socket says 172.21.15.131:57600) 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.847+0000 7f8930a00700 1 -- 172.21.15.131:0/856855658 learned_addr learned my addr 172.21.15.131:0/856855658 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.847+0000 7f8930a00700 1 -- 172.21.15.131:0/856855658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c19f670 con 0x7f892c07d320 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.847+0000 7f8930a00700 1 --2- 172.21.15.131:0/856855658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c19f130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8920011ac0 tx=0x7f8920011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.848+0000 7f8929600700 1 -- 172.21.15.131:0/856855658 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8920003b20 con 0x7f892c07d320 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.848+0000 7f8929600700 1 -- 172.21.15.131:0/856855658 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f892001a8c0 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.848+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c19f900 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.848+0000 7f8929600700 1 -- 172.21.15.131:0/856855658 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89200248d0 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.848+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c1a25a0 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.848+0000 7f8929600700 1 -- 172.21.15.131:0/856855658 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8920022020 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.848+0000 7f8929600700 1 -- 172.21.15.131:0/856855658 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f892002dde0 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.849+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c064a80 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.852+0000 7f8929600700 1 -- 172.21.15.131:0/856855658 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8920042ec0 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.932+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f892c076e80 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.933+0000 7f8929600700 1 -- 172.21.15.131:0/856855658 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f8920031020 con 0x7f892c07d320 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.934+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 msgr2=0x7f892c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.934+0000 7f8933400700 1 --2- 172.21.15.131:0/856855658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c19f130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8920011ac0 tx=0x7f8920011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.934+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 shutdown_connections 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.934+0000 7f8933400700 1 --2- 172.21.15.131:0/856855658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f892c07d320 0x7f892c19f130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:23.632 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.934+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 >> 172.21.15.131:0/856855658 conn(0x7f892c104200 msgr2=0x7f892c104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:23.633 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.934+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 shutdown_connections 2024-04-26T19:58:23.633 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:22.934+0000 7f8933400700 1 -- 172.21.15.131:0/856855658 wait complete. 2024-04-26T19:58:23.633 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (3/15)... 2024-04-26T19:58:24.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:23 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:23.785+0000 7fc53fecd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:58:24.506 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:24.067+0000 7fc53fecd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: Activating manager daemon x 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: mgrmap e2: x(active, starting, since 0.0122046s) 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T19:58:24.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: Manager daemon x is now available 2024-04-26T19:58:24.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T19:58:24.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T19:58:24.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' 2024-04-26T19:58:24.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:24 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' 2024-04-26T19:58:25.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:25 smithi131 ceph-mon[27919]: from='mgr.14100 172.21.15.131:0/1100959736' entity='mgr.x' 2024-04-26T19:58:25.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:25 smithi131 ceph-mon[27919]: mgrmap e3: x(active, since 1.02163s) 2024-04-26T19:58:27.223 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:27 smithi131 ceph-mon[27919]: mgrmap e4: x(active, since 2s) 2024-04-26T19:58:27.223 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:27 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3247012530' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T19:58:27.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T19:58:27.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T19:58:27.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T19:58:27.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-04-26T19:58:27.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T19:58:05.765390+0000", 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.242+0000 7fed0b400700 1 Processor -- start 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.242+0000 7fed0b400700 1 -- start start 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.243+0000 7fed0b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0407d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.243+0000 7fed0b400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed0407d930 con 0x7fed0407ef00 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.243+0000 7fed08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0407d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.243+0000 7fed08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0407d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35728/0 (socket says 172.21.15.131:35728) 2024-04-26T19:58:27.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.243+0000 7fed08a00700 1 -- 172.21.15.131:0/3412311514 learned_addr learned my addr 172.21.15.131:0/3412311514 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.243+0000 7fed08a00700 1 -- 172.21.15.131:0/3412311514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0407dab0 con 0x7fed0407ef00 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.244+0000 7fed08a00700 1 --2- 172.21.15.131:0/3412311514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0407d360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fecf40098a0 tx=0x7fecf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234a091f9612691a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.244+0000 7fed03e00700 1 -- 172.21.15.131:0/3412311514 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecf4004160 con 0x7fed0407ef00 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.244+0000 7fed03e00700 1 -- 172.21.15.131:0/3412311514 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fecf4004300 con 0x7fed0407ef00 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.244+0000 7fed03e00700 1 -- 172.21.15.131:0/3412311514 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecf4011af0 con 0x7fed0407ef00 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.245+0000 7fed0b400700 1 -- 172.21.15.131:0/3412311514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 msgr2=0x7fed0407d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.245+0000 7fed0b400700 1 --2- 172.21.15.131:0/3412311514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0407d360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fecf40098a0 tx=0x7fecf4009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.245+0000 7fed0b400700 1 -- 172.21.15.131:0/3412311514 shutdown_connections 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.245+0000 7fed0b400700 1 --2- 172.21.15.131:0/3412311514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0407d360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.245+0000 7fed0b400700 1 -- 172.21.15.131:0/3412311514 >> 172.21.15.131:0/3412311514 conn(0x7fed041041b0 msgr2=0x7fed04106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.245+0000 7fed0b400700 1 -- 172.21.15.131:0/3412311514 shutdown_connections 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.245+0000 7fed0b400700 1 -- 172.21.15.131:0/3412311514 wait complete. 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.246+0000 7fed0b400700 1 Processor -- start 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.247+0000 7fed0b400700 1 -- start start 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.247+0000 7fed0b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:27.528 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.247+0000 7fed0b400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410f1f0 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.247+0000 7fed08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.247+0000 7fed08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35744/0 (socket says 172.21.15.131:35744) 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.247+0000 7fed08a00700 1 -- 172.21.15.131:0/3247012530 learned_addr learned my addr 172.21.15.131:0/3247012530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.247+0000 7fed08a00700 1 -- 172.21.15.131:0/3247012530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0419f5d0 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.248+0000 7fed08a00700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0419f090 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fecf4003f40 tx=0x7fecf4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.248+0000 7fed02000700 1 -- 172.21.15.131:0/3247012530 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecf40038b0 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.248+0000 7fed02000700 1 -- 172.21.15.131:0/3247012530 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fecf4003a50 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.248+0000 7fed02000700 1 -- 172.21.15.131:0/3247012530 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecf401a5c0 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.248+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0419f860 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.248+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041a2500 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.249+0000 7fed02000700 1 -- 172.21.15.131:0/3247012530 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fecf4024430 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.249+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed04064a80 con 0x7fed0407ef00 2024-04-26T19:58:27.529 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.250+0000 7fed02000700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7fece8039560 0x7fece803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.250+0000 7fed02000700 1 -- 172.21.15.131:0/3247012530 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fecf404da60 con 0x7fed0407ef00 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.250+0000 7fecfbe00700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7fece8039560 0x7fece803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.251+0000 7fecfbe00700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7fece8039560 0x7fece803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fecec006fd0 tx=0x7fecec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.256+0000 7fed02000700 1 -- 172.21.15.131:0/3247012530 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fecf404d480 con 0x7fed0407ef00 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.571+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fed0402e490 con 0x7fed0407ef00 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.572+0000 7fed02000700 1 -- 172.21.15.131:0/3247012530 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fecf402d910 con 0x7fed0407ef00 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7fece8039560 msgr2=0x7fece803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7fece8039560 0x7fece803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fecec006fd0 tx=0x7fecec006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 msgr2=0x7fed0419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0419f090 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fecf4003f40 tx=0x7fecf4011890 comp rx=0 tx=0).stop 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 shutdown_connections 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7fece8039560 0x7fece803ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 --2- 172.21.15.131:0/3247012530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0407ef00 0x7fed0419f090 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:27.530 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.575+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 >> 172.21.15.131:0/3247012530 conn(0x7fed041041b0 msgr2=0x7fed04104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:27.531 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.576+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 shutdown_connections 2024-04-26T19:58:27.531 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:26.576+0000 7fed0b400700 1 -- 172.21.15.131:0/3247012530 wait complete. 2024-04-26T19:58:27.531 INFO:teuthology.orchestra.run.smithi131.stdout:mgr is available 2024-04-26T19:58:28.736 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:28 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3749422486' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T19:58:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T19:58:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T19:58:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T19:58:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T19:58:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.130+0000 7f99ee800700 1 Processor -- start 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.130+0000 7f99ee800700 1 -- start start 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.131+0000 7f99ee800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e807d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:29.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.131+0000 7f99ee800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99e807f010 con 0x7f99e807d2e0 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.131+0000 7f99e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e807d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.131+0000 7f99e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e807d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35758/0 (socket says 172.21.15.131:35758) 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.131+0000 7f99e7400700 1 -- 172.21.15.131:0/334010040 learned_addr learned my addr 172.21.15.131:0/334010040 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.132+0000 7f99e7400700 1 -- 172.21.15.131:0/334010040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e807dc40 con 0x7f99e807d2e0 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.132+0000 7f99e7400700 1 --2- 172.21.15.131:0/334010040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e807d700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99d80098a0 tx=0x7f99d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0f63c1a881e81f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.132+0000 7f99e6000700 1 -- 172.21.15.131:0/334010040 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99d8004160 con 0x7f99e807d2e0 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.132+0000 7f99e6000700 1 -- 172.21.15.131:0/334010040 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99d8004300 con 0x7f99e807d2e0 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.132+0000 7f99e6000700 1 -- 172.21.15.131:0/334010040 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99d8011af0 con 0x7f99e807d2e0 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.133+0000 7f99ee800700 1 -- 172.21.15.131:0/334010040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 msgr2=0x7f99e807d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.133+0000 7f99ee800700 1 --2- 172.21.15.131:0/334010040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e807d700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99d80098a0 tx=0x7f99d8009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.133+0000 7f99ee800700 1 -- 172.21.15.131:0/334010040 shutdown_connections 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.133+0000 7f99ee800700 1 --2- 172.21.15.131:0/334010040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e807d700 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.133+0000 7f99ee800700 1 -- 172.21.15.131:0/334010040 >> 172.21.15.131:0/334010040 conn(0x7f99e8104200 msgr2=0x7f99e8106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:29.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.133+0000 7f99ee800700 1 -- 172.21.15.131:0/334010040 shutdown_connections 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.133+0000 7f99ee800700 1 -- 172.21.15.131:0/334010040 wait complete. 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.134+0000 7f99ee800700 1 Processor -- start 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.134+0000 7f99ee800700 1 -- start start 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.135+0000 7f99ee800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.135+0000 7f99ee800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99e810f2f0 con 0x7f99e807d2e0 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.135+0000 7f99e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.135+0000 7f99e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35760/0 (socket says 172.21.15.131:35760) 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.135+0000 7f99e7400700 1 -- 172.21.15.131:0/3749422486 learned_addr learned my addr 172.21.15.131:0/3749422486 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.135+0000 7f99e7400700 1 -- 172.21.15.131:0/3749422486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e819f650 con 0x7f99e807d2e0 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.136+0000 7f99e7400700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e819f110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99d8011ac0 tx=0x7f99d8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.136+0000 7f99dfe00700 1 -- 172.21.15.131:0/3749422486 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99d8003b20 con 0x7f99e807d2e0 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.136+0000 7f99dfe00700 1 -- 172.21.15.131:0/3749422486 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99d801a8c0 con 0x7f99e807d2e0 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.136+0000 7f99dfe00700 1 -- 172.21.15.131:0/3749422486 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99d80248d0 con 0x7f99e807d2e0 2024-04-26T19:58:29.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.136+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e819f8e0 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.136+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e81a25d0 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.137+0000 7f99dfe00700 1 -- 172.21.15.131:0/3749422486 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f99d8022020 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.137+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e8064a80 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.138+0000 7f99dfe00700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f99cc0395b0 0x7f99cc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.138+0000 7f99dfe00700 1 -- 172.21.15.131:0/3749422486 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99d804d490 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.138+0000 7f99e6a00700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f99cc0395b0 0x7f99cc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.139+0000 7f99e6a00700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f99cc0395b0 0x7f99cc03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99d4006fd0 tx=0x7f99d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.143+0000 7f99dfe00700 1 -- 172.21.15.131:0/3749422486 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f99d801ab20 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.364+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f99e802dae0 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.366+0000 7f99dfe00700 1 -- 172.21.15.131:0/3749422486 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f99d804d1f0 con 0x7f99e807d2e0 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f99cc0395b0 msgr2=0x7f99cc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f99cc0395b0 0x7f99cc03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99d4006fd0 tx=0x7f99d4006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 msgr2=0x7f99e819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e819f110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99d8011ac0 tx=0x7f99d8011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:29.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 shutdown_connections 2024-04-26T19:58:29.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f99cc0395b0 0x7f99cc03ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:29.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 --2- 172.21.15.131:0/3749422486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99e807d2e0 0x7f99e819f110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:29.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 >> 172.21.15.131:0/3749422486 conn(0x7f99e8104200 msgr2=0x7f99e8104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:29.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 shutdown_connections 2024-04-26T19:58:29.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:28.369+0000 7f99ee800700 1 -- 172.21.15.131:0/3749422486 wait complete. 2024-04-26T19:58:29.059 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling cephadm module... 2024-04-26T19:58:30.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:30 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/363030815' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T19:58:30.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: ignoring --setuser ceph since I am not root 2024-04-26T19:58:30.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: ignoring --setgroup ceph since I am not root 2024-04-26T19:58:30.660 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:30.292+0000 7f1393b2e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:58:30.660 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:30.528+0000 7f1393b2e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:58:30.660 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:30.658+0000 7f1393b2e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:58:30.755 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.665+0000 7ff76a800700 1 Processor -- start 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.666+0000 7ff76a800700 1 -- start start 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.666+0000 7ff76a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76407d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.666+0000 7ff76a800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff76407f070 con 0x7ff76407d340 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.666+0000 7ff763400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76407d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.667+0000 7ff763400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76407d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35774/0 (socket says 172.21.15.131:35774) 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.667+0000 7ff763400700 1 -- 172.21.15.131:0/969529273 learned_addr learned my addr 172.21.15.131:0/969529273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.667+0000 7ff763400700 1 -- 172.21.15.131:0/969529273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76407dca0 con 0x7ff76407d340 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.667+0000 7ff763400700 1 --2- 172.21.15.131:0/969529273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76407d760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff7540098a0 tx=0x7ff754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78352059ef13f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.668+0000 7ff762000700 1 -- 172.21.15.131:0/969529273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff754004160 con 0x7ff76407d340 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.668+0000 7ff762000700 1 -- 172.21.15.131:0/969529273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff754004300 con 0x7ff76407d340 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.668+0000 7ff762000700 1 -- 172.21.15.131:0/969529273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff754011af0 con 0x7ff76407d340 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.668+0000 7ff76a800700 1 -- 172.21.15.131:0/969529273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 msgr2=0x7ff76407d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.668+0000 7ff76a800700 1 --2- 172.21.15.131:0/969529273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76407d760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff7540098a0 tx=0x7ff754009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.669+0000 7ff76a800700 1 -- 172.21.15.131:0/969529273 shutdown_connections 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.669+0000 7ff76a800700 1 --2- 172.21.15.131:0/969529273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76407d760 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.669+0000 7ff76a800700 1 -- 172.21.15.131:0/969529273 >> 172.21.15.131:0/969529273 conn(0x7ff764104200 msgr2=0x7ff764106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.669+0000 7ff76a800700 1 -- 172.21.15.131:0/969529273 shutdown_connections 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.669+0000 7ff76a800700 1 -- 172.21.15.131:0/969529273 wait complete. 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.670+0000 7ff76a800700 1 Processor -- start 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.670+0000 7ff76a800700 1 -- start start 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.670+0000 7ff76a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.670+0000 7ff76a800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff76410f2f0 con 0x7ff76407d340 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.671+0000 7ff763400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.671+0000 7ff763400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76419f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35786/0 (socket says 172.21.15.131:35786) 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.671+0000 7ff763400700 1 -- 172.21.15.131:0/363030815 learned_addr learned my addr 172.21.15.131:0/363030815 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.671+0000 7ff763400700 1 -- 172.21.15.131:0/363030815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76419f650 con 0x7ff76407d340 2024-04-26T19:58:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.671+0000 7ff763400700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76419f110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff754011ac0 tx=0x7ff754011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.672+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff754003b20 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.672+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff75401a8c0 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.672+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7540248d0 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.672+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76419f8e0 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.672+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7641a25d0 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.673+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff754022020 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.673+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff764064a80 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.674+0000 7ff75be00700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7ff7480395b0 0x7ff74803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.674+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff75404d490 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.674+0000 7ff762a00700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7ff7480395b0 0x7ff74803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.675+0000 7ff762a00700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7ff7480395b0 0x7ff74803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff750006fd0 tx=0x7ff750006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.679+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff75401ab20 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:29.964+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff76402e490 con 0x7ff76407d340 2024-04-26T19:58:30.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.018+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff75404d1f0 con 0x7ff76407d340 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.021+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7ff7480395b0 msgr2=0x7ff74803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.021+0000 7ff76a800700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7ff7480395b0 0x7ff74803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff750006fd0 tx=0x7ff750006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.021+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 msgr2=0x7ff76419f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.021+0000 7ff75be00700 1 -- 172.21.15.131:0/363030815 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff75402d3f0 con 0x7ff76407d340 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.021+0000 7ff76a800700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76419f110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff754011ac0 tx=0x7ff754011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.022+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 shutdown_connections 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.022+0000 7ff76a800700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7ff7480395b0 0x7ff74803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.022+0000 7ff76a800700 1 --2- 172.21.15.131:0/363030815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76407d340 0x7ff76419f110 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.022+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 >> 172.21.15.131:0/363030815 conn(0x7ff764104200 msgr2=0x7ff764104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.024+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 shutdown_connections 2024-04-26T19:58:30.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:30.024+0000 7ff76a800700 1 -- 172.21.15.131:0/363030815 wait complete. 2024-04-26T19:58:31.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:31 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/363030815' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T19:58:31.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:31 smithi131 ceph-mon[27919]: mgrmap e5: x(active, since 5s) 2024-04-26T19:58:31.179 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:31.061+0000 7f1393b2e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:58:31.506 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:31.177+0000 7f1393b2e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:58:32.426 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:32 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2982689045' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.397+0000 7f5071a00700 1 Processor -- start 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.398+0000 7f5071a00700 1 -- start start 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.398+0000 7f5071a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c109050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.398+0000 7f5071a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f506c109620 con 0x7f506c108c30 2024-04-26T19:58:32.443 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.398+0000 7f506aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.399+0000 7f506aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c109050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35814/0 (socket says 172.21.15.131:35814) 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.399+0000 7f506aa00700 1 -- 172.21.15.131:0/3664822059 learned_addr learned my addr 172.21.15.131:0/3664822059 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.399+0000 7f506aa00700 1 -- 172.21.15.131:0/3664822059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506c109e30 con 0x7f506c108c30 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.399+0000 7f506aa00700 1 --2- 172.21.15.131:0/3664822059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c109050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f505c0098a0 tx=0x7f505c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5526cdf0f89b389b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.399+0000 7f506a000700 1 -- 172.21.15.131:0/3664822059 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f505c004160 con 0x7f506c108c30 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.400+0000 7f506a000700 1 -- 172.21.15.131:0/3664822059 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f505c004300 con 0x7f506c108c30 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.400+0000 7f506a000700 1 -- 172.21.15.131:0/3664822059 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f505c011af0 con 0x7f506c108c30 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.400+0000 7f5071a00700 1 -- 172.21.15.131:0/3664822059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 msgr2=0x7f506c109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.400+0000 7f5071a00700 1 --2- 172.21.15.131:0/3664822059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c109050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f505c0098a0 tx=0x7f505c009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.400+0000 7f5071a00700 1 -- 172.21.15.131:0/3664822059 shutdown_connections 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.400+0000 7f5071a00700 1 --2- 172.21.15.131:0/3664822059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c109050 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.400+0000 7f5071a00700 1 -- 172.21.15.131:0/3664822059 >> 172.21.15.131:0/3664822059 conn(0x7f506c1041b0 msgr2=0x7f506c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.401+0000 7f5071a00700 1 -- 172.21.15.131:0/3664822059 shutdown_connections 2024-04-26T19:58:32.444 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.401+0000 7f5071a00700 1 -- 172.21.15.131:0/3664822059 wait complete. 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.402+0000 7f5071a00700 1 Processor -- start 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.402+0000 7f5071a00700 1 -- start start 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.402+0000 7f5071a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.402+0000 7f5071a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f506c10d1e0 con 0x7f506c108c30 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.402+0000 7f506aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.402+0000 7f506aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35816/0 (socket says 172.21.15.131:35816) 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.402+0000 7f506aa00700 1 -- 172.21.15.131:0/2982689045 learned_addr learned my addr 172.21.15.131:0/2982689045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.403+0000 7f506aa00700 1 -- 172.21.15.131:0/2982689045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506c19d390 con 0x7f506c108c30 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.403+0000 7f506aa00700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c19ce50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f505c000c00 tx=0x7f505c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.403+0000 7f5063400700 1 -- 172.21.15.131:0/2982689045 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f505c003b20 con 0x7f506c108c30 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.403+0000 7f5063400700 1 -- 172.21.15.131:0/2982689045 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f505c01a8c0 con 0x7f506c108c30 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.403+0000 7f5063400700 1 -- 172.21.15.131:0/2982689045 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f505c024940 con 0x7f506c108c30 2024-04-26T19:58:32.445 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.403+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506c19d620 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.403+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506c1a0310 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.405+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506c064a80 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.405+0000 7f5063400700 1 -- 172.21.15.131:0/2982689045 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f505c022020 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.405+0000 7f5063400700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 0x7f505003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.405+0000 7f5063400700 1 -- 172.21.15.131:0/2982689045 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f505c04d490 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.406+0000 7f5063e00700 1 -- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 msgr2=0x7f505003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2560715060 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.406+0000 7f5063e00700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 0x7f505003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.411+0000 7f5063400700 1 -- 172.21.15.131:0/2982689045 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f505c00be50 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.607+0000 7f5063e00700 1 -- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 msgr2=0x7f505003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2560715060 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.607+0000 7f5063e00700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 0x7f505003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.682+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f506c02e490 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.682+0000 7f5063400700 1 -- 172.21.15.131:0/2982689045 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f505c04d1f0 con 0x7f506c108c30 2024-04-26T19:58:32.446 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.685+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 msgr2=0x7f505003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.685+0000 7f5071a00700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 0x7f505003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.685+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 msgr2=0x7f506c19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.685+0000 7f5071a00700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c19ce50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f505c000c00 tx=0x7f505c011b70 comp rx=0 tx=0).stop 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.686+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 shutdown_connections 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.686+0000 7f5071a00700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f5050039600 0x7f505003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.686+0000 7f5071a00700 1 --2- 172.21.15.131:0/2982689045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c108c30 0x7f506c19ce50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.686+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 >> 172.21.15.131:0/2982689045 conn(0x7f506c1041b0 msgr2=0x7f506c104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.686+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 shutdown_connections 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:31.686+0000 7f5071a00700 1 -- 172.21.15.131:0/2982689045 wait complete. 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-04-26T19:58:32.447 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 5... 2024-04-26T19:58:33.000 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:32.761+0000 7f1393b2e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:58:33.001 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:32.993+0000 7f1393b2e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:58:33.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:33.212+0000 7f1393b2e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:58:33.976 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:33.586+0000 7f1393b2e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:58:33.976 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:33.710+0000 7f1393b2e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:58:34.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:33.974+0000 7f1393b2e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:58:34.756 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:34 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:34.259+0000 7f1393b2e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:58:35.349 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:35.075+0000 7f1393b2e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:58:36.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:35.627+0000 7f1393b2e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:58:36.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:35.754+0000 7f1393b2e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:58:36.506 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:36.017+0000 7f1393b2e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:58:37.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:36.660+0000 7f1393b2e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:58:37.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:36.781+0000 7f1393b2e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:58:37.401 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:37 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:37.096+0000 7f1393b2e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:58:37.401 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:37 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:37.274+0000 7f1393b2e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:58:37.401 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:37 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:37.399+0000 7f1393b2e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:58:37.756 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:37 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:37.546+0000 7f1393b2e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:58:38.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:37 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:37.915+0000 7f1393b2e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:58:38.940 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:38.657+0000 7f1393b2e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:58:39.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:58:38.938+0000 7f1393b2e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:58:39.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: Active manager daemon x restarted 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: Activating manager daemon x 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: mgrmap e6: x(active, starting, since 0.00492767s) 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: Manager daemon x is now available 2024-04-26T19:58:39.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:38 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:40.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: Found migration_current of "None". Setting to last migration. 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:40.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:40 smithi131 ceph-mon[27919]: mgrmap e7: x(active, since 1.01465s) 2024-04-26T19:58:40.674 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:32.999+0000 7f7c6b200700 1 Processor -- start 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:32.999+0000 7f7c6b200700 1 -- start start 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.000+0000 7f7c6b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c108e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.000+0000 7f7c6b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c109430 con 0x7f7c6c108a40 2024-04-26T19:58:40.682 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.000+0000 7f7c69e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c108e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.000+0000 7f7c69e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c108e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35830/0 (socket says 172.21.15.131:35830) 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.000+0000 7f7c69e00700 1 -- 172.21.15.131:0/1210010542 learned_addr learned my addr 172.21.15.131:0/1210010542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.001+0000 7f7c69e00700 1 -- 172.21.15.131:0/1210010542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c109c40 con 0x7f7c6c108a40 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.001+0000 7f7c69e00700 1 --2- 172.21.15.131:0/1210010542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c108e60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0098a0 tx=0x7f7c5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949ec04332488201 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.001+0000 7f7c68a00700 1 -- 172.21.15.131:0/1210010542 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c5c004160 con 0x7f7c6c108a40 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.002+0000 7f7c68a00700 1 -- 172.21.15.131:0/1210010542 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7c5c004300 con 0x7f7c6c108a40 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.002+0000 7f7c68a00700 1 -- 172.21.15.131:0/1210010542 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c5c011af0 con 0x7f7c6c108a40 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.002+0000 7f7c6b200700 1 -- 172.21.15.131:0/1210010542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 msgr2=0x7f7c6c108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.002+0000 7f7c6b200700 1 --2- 172.21.15.131:0/1210010542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c108e60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0098a0 tx=0x7f7c5c009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.002+0000 7f7c6b200700 1 -- 172.21.15.131:0/1210010542 shutdown_connections 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.003+0000 7f7c6b200700 1 --2- 172.21.15.131:0/1210010542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c108e60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.003+0000 7f7c6b200700 1 -- 172.21.15.131:0/1210010542 >> 172.21.15.131:0/1210010542 conn(0x7f7c6c103fc0 msgr2=0x7f7c6c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:40.683 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.003+0000 7f7c6b200700 1 -- 172.21.15.131:0/1210010542 shutdown_connections 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.003+0000 7f7c6b200700 1 -- 172.21.15.131:0/1210010542 wait complete. 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.004+0000 7f7c6b200700 1 Processor -- start 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.004+0000 7f7c6b200700 1 -- start start 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.004+0000 7f7c6b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.004+0000 7f7c6b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c10cff0 con 0x7f7c6c108a40 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.004+0000 7f7c69e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.004+0000 7f7c69e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35846/0 (socket says 172.21.15.131:35846) 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.004+0000 7f7c69e00700 1 -- 172.21.15.131:0/2390615210 learned_addr learned my addr 172.21.15.131:0/2390615210 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.005+0000 7f7c69e00700 1 -- 172.21.15.131:0/2390615210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c19f410 con 0x7f7c6c108a40 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.005+0000 7f7c69e00700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c19eed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.005+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c5c003b20 con 0x7f7c6c108a40 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.005+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7c5c01a8c0 con 0x7f7c6c108a40 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.005+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c19f6a0 con 0x7f7c6c108a40 2024-04-26T19:58:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.005+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c5c024940 con 0x7f7c6c108a40 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.006+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c1a2390 con 0x7f7c6c108a40 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.007+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7c5c022020 con 0x7f7c6c108a40 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.007+0000 7f7c62a00700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.007+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 --> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c5003c1d0 con 0x7f7c50039600 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.007+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c5c04d620 con 0x7f7c6c108a40 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.007+0000 7f7c69400700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 msgr2=0x7f7c5003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2560715060 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.007+0000 7f7c69400700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.208+0000 7f7c69400700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 msgr2=0x7f7c5003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2560715060 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.208+0000 7f7c69400700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.609+0000 7f7c69400700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 msgr2=0x7f7c5003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2560715060 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:33.609+0000 7f7c69400700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:34.410+0000 7f7c69400700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 msgr2=0x7f7c5003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2560715060 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:34.410+0000 7f7c69400700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T19:58:40.685 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:36.011+0000 7f7c69400700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 msgr2=0x7f7c5003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2560715060 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:36.011+0000 7f7c69400700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:38.946+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f7c5c01aae0 con 0x7f7c6c108a40 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:38.946+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 msgr2=0x7f7c5003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:38.946+0000 7f7c62a00700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.952+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f7c5c04f0d0 con 0x7f7c6c108a40 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.953+0000 7f7c62a00700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7c5003cef0 0x7f7c5003f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.953+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c5003c1d0 con 0x7f7c5003cef0 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.954+0000 7f7c69400700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7c5003cef0 0x7f7c5003f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.956+0000 7f7c69400700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7c5003cef0 0x7f7c5003f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c58003a10 tx=0x7f7c580092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.958+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7c5003c1d0 con 0x7f7c5003cef0 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.967+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7c6c10ce40 con 0x7f7c5003cef0 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c62a00700 1 -- 172.21.15.131:0/2390615210 <== mgr.14120 v2:172.21.15.131:6800/3789992830 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7c6c10ce40 con 0x7f7c5003cef0 2024-04-26T19:58:40.686 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7c5003cef0 msgr2=0x7f7c5003f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7c5003cef0 0x7f7c5003f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c58003a10 tx=0x7f7c580092b0 comp rx=0 tx=0).stop 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 msgr2=0x7f7c6c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c19eed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c011b70 comp rx=0 tx=0).stop 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 shutdown_connections 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7c5003cef0 0x7f7c5003f2e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c6c108a40 0x7f7c6c19eed0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 --2- 172.21.15.131:0/2390615210 >> [v2:172.21.15.131:6800/2560715060,v1:172.21.15.131:6801/2560715060] conn(0x7f7c50039600 0x7f7c5003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.968+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 >> 172.21.15.131:0/2390615210 conn(0x7f7c6c103fc0 msgr2=0x7f7c6c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.969+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 shutdown_connections 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:39.969+0000 7f7c6b200700 1 -- 172.21.15.131:0/2390615210 wait complete. 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 5 is available 2024-04-26T19:58:40.687 INFO:teuthology.orchestra.run.smithi131.stdout:Setting orchestrator backend to cephadm... 2024-04-26T19:58:41.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:41 smithi131 ceph-mon[27919]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T19:58:41.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:41 smithi131 ceph-mon[27919]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T19:58:41.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:41 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:42.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:42 smithi131 ceph-mon[27919]: [26/Apr/2024:19:58:40] ENGINE Bus STARTING 2024-04-26T19:58:42.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:42 smithi131 ceph-mon[27919]: [26/Apr/2024:19:58:40] ENGINE Serving on https://172.21.15.131:7150 2024-04-26T19:58:42.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:42 smithi131 ceph-mon[27919]: [26/Apr/2024:19:58:40] ENGINE Serving on http://172.21.15.131:8765 2024-04-26T19:58:42.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:42 smithi131 ceph-mon[27919]: [26/Apr/2024:19:58:40] ENGINE Bus STARTED 2024-04-26T19:58:42.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:42 smithi131 ceph-mon[27919]: mgrmap e8: x(active, since 2s) 2024-04-26T19:58:42.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:42 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:42.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:42 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:42.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.308+0000 7f79f9a00700 1 Processor -- start 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.308+0000 7f79f9a00700 1 -- start start 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.309+0000 7f79f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.309+0000 7f79f9a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79f407d8d0 con 0x7f79f407eea0 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.309+0000 7f79f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.309+0000 7f79f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52356/0 (socket says 172.21.15.131:52356) 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.309+0000 7f79f3e00700 1 -- 172.21.15.131:0/1498792784 learned_addr learned my addr 172.21.15.131:0/1498792784 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.309+0000 7f79f3e00700 1 -- 172.21.15.131:0/1498792784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f407da50 con 0x7f79f407eea0 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.310+0000 7f79f3e00700 1 --2- 172.21.15.131:0/1498792784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f407d300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f79e40098a0 tx=0x7f79e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a962bc6a7cf7b139 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.310+0000 7f79f2a00700 1 -- 172.21.15.131:0/1498792784 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79e4004160 con 0x7f79f407eea0 2024-04-26T19:58:42.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.310+0000 7f79f2a00700 1 -- 172.21.15.131:0/1498792784 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f79e4004300 con 0x7f79f407eea0 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.310+0000 7f79f2a00700 1 -- 172.21.15.131:0/1498792784 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79e4011af0 con 0x7f79f407eea0 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.311+0000 7f79f9a00700 1 -- 172.21.15.131:0/1498792784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 msgr2=0x7f79f407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.311+0000 7f79f9a00700 1 --2- 172.21.15.131:0/1498792784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f407d300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f79e40098a0 tx=0x7f79e4009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.311+0000 7f79f9a00700 1 -- 172.21.15.131:0/1498792784 shutdown_connections 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.311+0000 7f79f9a00700 1 --2- 172.21.15.131:0/1498792784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f407d300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.311+0000 7f79f9a00700 1 -- 172.21.15.131:0/1498792784 >> 172.21.15.131:0/1498792784 conn(0x7f79f4103fc0 msgr2=0x7f79f4106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.311+0000 7f79f9a00700 1 -- 172.21.15.131:0/1498792784 shutdown_connections 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.311+0000 7f79f9a00700 1 -- 172.21.15.131:0/1498792784 wait complete. 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.312+0000 7f79f9a00700 1 Processor -- start 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.312+0000 7f79f9a00700 1 -- start start 2024-04-26T19:58:42.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.312+0000 7f79f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.312+0000 7f79f9a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410f010 con 0x7f79f407eea0 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52372/0 (socket says 172.21.15.131:52372) 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f3e00700 1 -- 172.21.15.131:0/2557856781 learned_addr learned my addr 172.21.15.131:0/2557856781 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f3e00700 1 -- 172.21.15.131:0/2557856781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f419f3c0 con 0x7f79f407eea0 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f3e00700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f419ee80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79e4011ac0 tx=0x7f79e4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f0c00700 1 -- 172.21.15.131:0/2557856781 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79e4003b20 con 0x7f79f407eea0 2024-04-26T19:58:42.270 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f0c00700 1 -- 172.21.15.131:0/2557856781 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f79e401a8c0 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f419f650 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.313+0000 7f79f0c00700 1 -- 172.21.15.131:0/2557856781 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79e40248d0 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.314+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f41a22f0 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.315+0000 7f79f0c00700 1 -- 172.21.15.131:0/2557856781 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f79e4022020 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.315+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f4064a80 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.315+0000 7f79f0c00700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f79d8039600 0x7f79d803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.315+0000 7f79f0c00700 1 -- 172.21.15.131:0/2557856781 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f79e404d900 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.315+0000 7f79f3400700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f79d8039600 0x7f79d803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.316+0000 7f79f3400700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f79d8039600 0x7f79d803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006fd0 tx=0x7f79e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.321+0000 7f79f0c00700 1 -- 172.21.15.131:0/2557856781 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f79e400f2e0 con 0x7f79f407eea0 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.564+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f79f402e500 con 0x7f79d8039600 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.574+0000 7f79f0c00700 1 -- 172.21.15.131:0/2557856781 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f79f402e500 con 0x7f79d8039600 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f79d8039600 msgr2=0x7f79d803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:42.271 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f79d8039600 0x7f79d803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006fd0 tx=0x7f79e0006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 msgr2=0x7f79f419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f419ee80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79e4011ac0 tx=0x7f79e4011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 shutdown_connections 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f79d8039600 0x7f79d803bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 --2- 172.21.15.131:0/2557856781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79f407eea0 0x7f79f419ee80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 >> 172.21.15.131:0/2557856781 conn(0x7f79f4103fc0 msgr2=0x7f79f4104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 shutdown_connections 2024-04-26T19:58:42.272 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:41.577+0000 7f79f9a00700 1 -- 172.21.15.131:0/2557856781 wait complete. 2024-04-26T19:58:43.155 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:43 smithi131 ceph-mon[27919]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.889+0000 7f9527200700 1 Processor -- start 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.889+0000 7f9527200700 1 -- start start 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.889+0000 7f9527200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.889+0000 7f9527200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95200834b0 con 0x7f952010fcf0 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.890+0000 7f951fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.890+0000 7f951fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52384/0 (socket says 172.21.15.131:52384) 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.890+0000 7f951fe00700 1 -- 172.21.15.131:0/1843069566 learned_addr learned my addr 172.21.15.131:0/1843069566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.890+0000 7f951fe00700 1 -- 172.21.15.131:0/1843069566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9520112620 con 0x7f952010fcf0 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.891+0000 7f951fe00700 1 --2- 172.21.15.131:0/1843069566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201120e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95100098a0 tx=0x7f9510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfc9391ea87c535 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:43.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.891+0000 7f951ea00700 1 -- 172.21.15.131:0/1843069566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9510004160 con 0x7f952010fcf0 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.891+0000 7f951ea00700 1 -- 172.21.15.131:0/1843069566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9510004300 con 0x7f952010fcf0 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.891+0000 7f951ea00700 1 -- 172.21.15.131:0/1843069566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9510011af0 con 0x7f952010fcf0 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.892+0000 7f9527200700 1 -- 172.21.15.131:0/1843069566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 msgr2=0x7f95201120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.892+0000 7f9527200700 1 --2- 172.21.15.131:0/1843069566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201120e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95100098a0 tx=0x7f9510009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.892+0000 7f9527200700 1 -- 172.21.15.131:0/1843069566 shutdown_connections 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.892+0000 7f9527200700 1 --2- 172.21.15.131:0/1843069566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201120e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.892+0000 7f9527200700 1 -- 172.21.15.131:0/1843069566 >> 172.21.15.131:0/1843069566 conn(0x7f952007e210 msgr2=0x7f9520080650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.892+0000 7f9527200700 1 -- 172.21.15.131:0/1843069566 shutdown_connections 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.892+0000 7f9527200700 1 -- 172.21.15.131:0/1843069566 wait complete. 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.893+0000 7f9527200700 1 Processor -- start 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.893+0000 7f9527200700 1 -- start start 2024-04-26T19:58:43.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.894+0000 7f9527200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.894+0000 7f9527200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95201135e0 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.894+0000 7f951fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.894+0000 7f951fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201a34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52396/0 (socket says 172.21.15.131:52396) 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.894+0000 7f951fe00700 1 -- 172.21.15.131:0/1480124834 learned_addr learned my addr 172.21.15.131:0/1480124834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.894+0000 7f951fe00700 1 -- 172.21.15.131:0/1480124834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95201a39e0 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.894+0000 7f951fe00700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201a34a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9510011ac0 tx=0x7f9510011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.895+0000 7f951cc00700 1 -- 172.21.15.131:0/1480124834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9510003970 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.895+0000 7f951cc00700 1 -- 172.21.15.131:0/1480124834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9510003b10 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.895+0000 7f951cc00700 1 -- 172.21.15.131:0/1480124834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9510024610 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.895+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95201a3c70 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.895+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95201a6960 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.896+0000 7f951cc00700 1 -- 172.21.15.131:0/1480124834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f951001a430 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.896+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9520064a80 con 0x7f952010fcf0 2024-04-26T19:58:43.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.896+0000 7f951cc00700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f9504039600 0x7f950403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.897+0000 7f951cc00700 1 -- 172.21.15.131:0/1480124834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f951004d9a0 con 0x7f952010fcf0 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.897+0000 7f951f400700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f9504039600 0x7f950403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.897+0000 7f951f400700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f9504039600 0x7f950403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f950c006fd0 tx=0x7f950c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:42.902+0000 7f951cc00700 1 -- 172.21.15.131:0/1480124834 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9510016260 con 0x7f952010fcf0 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.132+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f952002e500 con 0x7f9504039600 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.134+0000 7f951cc00700 1 -- 172.21.15.131:0/1480124834 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f952002e500 con 0x7f9504039600 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.136+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f9504039600 msgr2=0x7f950403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.136+0000 7f9527200700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f9504039600 0x7f950403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f950c006fd0 tx=0x7f950c006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.136+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 msgr2=0x7f95201a34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.136+0000 7f9527200700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201a34a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9510011ac0 tx=0x7f9510011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.137+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 shutdown_connections 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.137+0000 7f9527200700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f9504039600 0x7f950403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.137+0000 7f9527200700 1 --2- 172.21.15.131:0/1480124834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952010fcf0 0x7f95201a34a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.137+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 >> 172.21.15.131:0/1480124834 conn(0x7f952007e210 msgr2=0x7f952007ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:43.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.137+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 shutdown_connections 2024-04-26T19:58:43.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:43.137+0000 7f9527200700 1 -- 172.21.15.131:0/1480124834 wait complete. 2024-04-26T19:58:43.772 INFO:teuthology.orchestra.run.smithi131.stdout:Generating ssh key... 2024-04-26T19:58:44.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-mon[27919]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:58:45.257 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: Generating public/private rsa key pair. 2024-04-26T19:58:45.257 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: Your identification has been saved in /tmp/tmp0lo4f57u/key. 2024-04-26T19:58:45.257 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: Your public key has been saved in /tmp/tmp0lo4f57u/key.pub. 2024-04-26T19:58:45.257 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: The key fingerprint is: 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: SHA256:tMBibIZIGQgx2FAzPvDMdKMBtJVsIKcqSJu6Y6JNeCk ceph-2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: The key's randomart image is: 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: +---[RSA 3072]----+ 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |&@%.+ | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |=&+% o | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |ooX * o . | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |+ o= . o . | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |+o S | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |o. . | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |E + | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |oB | 2024-04-26T19:58:45.258 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: |*.. | 2024-04-26T19:58:45.259 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:58:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: +----[SHA256]-----+ 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.368+0000 7f7af2600700 1 Processor -- start 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.368+0000 7f7af2600700 1 -- start start 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.369+0000 7f7af2600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.369+0000 7f7af2600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec07d8d0 con 0x7f7aec07eea0 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.369+0000 7f7aeb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.369+0000 7f7aeb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45622/0 (socket says 172.21.15.131:45622) 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.369+0000 7f7aeb400700 1 -- 172.21.15.131:0/2727848506 learned_addr learned my addr 172.21.15.131:0/2727848506 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.369+0000 7f7aeb400700 1 -- 172.21.15.131:0/2727848506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec07da50 con 0x7f7aec07eea0 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.370+0000 7f7aeb400700 1 --2- 172.21.15.131:0/2727848506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec07d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0098a0 tx=0x7f7adc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe47d08a40b35c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.370+0000 7f7aea000700 1 -- 172.21.15.131:0/2727848506 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc004160 con 0x7f7aec07eea0 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.370+0000 7f7aea000700 1 -- 172.21.15.131:0/2727848506 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7adc004300 con 0x7f7aec07eea0 2024-04-26T19:58:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.370+0000 7f7aea000700 1 -- 172.21.15.131:0/2727848506 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc011af0 con 0x7f7aec07eea0 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.371+0000 7f7af2600700 1 -- 172.21.15.131:0/2727848506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 msgr2=0x7f7aec07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.371+0000 7f7af2600700 1 --2- 172.21.15.131:0/2727848506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec07d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0098a0 tx=0x7f7adc009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.371+0000 7f7af2600700 1 -- 172.21.15.131:0/2727848506 shutdown_connections 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.371+0000 7f7af2600700 1 --2- 172.21.15.131:0/2727848506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec07d300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.371+0000 7f7af2600700 1 -- 172.21.15.131:0/2727848506 >> 172.21.15.131:0/2727848506 conn(0x7f7aec1041b0 msgr2=0x7f7aec106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.371+0000 7f7af2600700 1 -- 172.21.15.131:0/2727848506 shutdown_connections 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.371+0000 7f7af2600700 1 -- 172.21.15.131:0/2727848506 wait complete. 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.372+0000 7f7af2600700 1 Processor -- start 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.372+0000 7f7af2600700 1 -- start start 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.373+0000 7f7af2600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.373+0000 7f7af2600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec10f200 con 0x7f7aec07eea0 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.373+0000 7f7aeb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.373+0000 7f7aeb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45638/0 (socket says 172.21.15.131:45638) 2024-04-26T19:58:45.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.373+0000 7f7aeb400700 1 -- 172.21.15.131:0/4168888959 learned_addr learned my addr 172.21.15.131:0/4168888959 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.373+0000 7f7aeb400700 1 -- 172.21.15.131:0/4168888959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec19f580 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.374+0000 7f7aeb400700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec19f040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7adc011ac0 tx=0x7f7adc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.374+0000 7f7ae3e00700 1 -- 172.21.15.131:0/4168888959 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc003b20 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.374+0000 7f7ae3e00700 1 -- 172.21.15.131:0/4168888959 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7adc01a8c0 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.374+0000 7f7ae3e00700 1 -- 172.21.15.131:0/4168888959 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc0248d0 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.374+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec19f810 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.374+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1a24b0 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.375+0000 7f7ae3e00700 1 -- 172.21.15.131:0/4168888959 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7adc022020 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.375+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec064a80 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.376+0000 7f7ae3e00700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7ad0039600 0x7f7ad003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.376+0000 7f7ae3e00700 1 -- 172.21.15.131:0/4168888959 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7adc04d880 con 0x7f7aec07eea0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.376+0000 7f7aeaa00700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7ad0039600 0x7f7ad003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:45.555 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.377+0000 7f7aeaa00700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7ad0039600 0x7f7ad003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8006fd0 tx=0x7f7ad8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.381+0000 7f7ae3e00700 1 -- 172.21.15.131:0/4168888959 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7adc00f2e0 con 0x7f7aec07eea0 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.607+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7aec02e500 con 0x7f7ad0039600 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.863+0000 7f7ae3e00700 1 -- 172.21.15.131:0/4168888959 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7aec02e500 con 0x7f7ad0039600 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7ad0039600 msgr2=0x7f7ad003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7ad0039600 0x7f7ad003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8006fd0 tx=0x7f7ad8006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 msgr2=0x7f7aec19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec19f040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7adc011ac0 tx=0x7f7adc011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 shutdown_connections 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f7ad0039600 0x7f7ad003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 --2- 172.21.15.131:0/4168888959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec07eea0 0x7f7aec19f040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 >> 172.21.15.131:0/4168888959 conn(0x7f7aec1041b0 msgr2=0x7f7aec104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.866+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 shutdown_connections 2024-04-26T19:58:45.556 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:44.867+0000 7f7af2600700 1 -- 172.21.15.131:0/4168888959 wait complete. 2024-04-26T19:58:46.122 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:45 smithi131 ceph-mon[27919]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:58:46.122 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:45 smithi131 ceph-mon[27919]: Generating ssh key... 2024-04-26T19:58:46.122 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:45 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:46.122 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:45 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq4IlVHkzONz4kZkrM5HLt/kLV3jQa+K/vUs40z+Q6VVntMQzS0rLv8pNscyoUVyaQMY/VW8rx31kA+r9k6vmmPhx4+qx4FNQG7MSH8qFde6NYPXhyzofsuz7IQhRhTI9PiOnsNqlkxzc7VDtgve7caP7I51+3bD+xZ6EUBQPf+Aa3ynanBv0zut3hM68I9/v7wXCAB2h4SyVBHrJKV+fIObGbWY/rPs20FTz3jiLr7Zo5sFDTMx639gY4IUSCqXPHHiyF/5ng287jTXJoVTVzuuy+utz1VG0Z7cJizGYoyzGl5OIQWxUwUvwdXrJy5SU/1rYybcRNxeemejoQ/52eaXwL+dor0BKJ1h3mPoSecihvQMnsytYRDFYuHtedlYY9RRzBbo9HFmFU8+kOWYCBzfHAmcEnJi/b1rkT01w09rQi6ZEI1F6qRlennuuxqRM+UfhW5iJQYOQD4bFx3N42KC+FI1czeWh4+bcnAdaVDvlYWk6/Mcnk5fs8T3fjMd0= ceph-2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:58:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.157+0000 7fdbfe800700 1 Processor -- start 2024-04-26T19:58:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.158+0000 7fdbfe800700 1 -- start start 2024-04-26T19:58:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.158+0000 7fdbfe800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf810bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.158+0000 7fdbfe800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf807ced0 con 0x7fdbf81096d0 2024-04-26T19:58:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.159+0000 7fdbf7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf810bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.159+0000 7fdbf7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf810bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45640/0 (socket says 172.21.15.131:45640) 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.159+0000 7fdbf7400700 1 -- 172.21.15.131:0/212697575 learned_addr learned my addr 172.21.15.131:0/212697575 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.159+0000 7fdbf7400700 1 -- 172.21.15.131:0/212697575 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf810c000 con 0x7fdbf81096d0 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.159+0000 7fdbf7400700 1 --2- 172.21.15.131:0/212697575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf810bac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80098a0 tx=0x7fdbe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965a14e78ebee72f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.160+0000 7fdbf6000700 1 -- 172.21.15.131:0/212697575 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe8004160 con 0x7fdbf81096d0 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.160+0000 7fdbf6000700 1 -- 172.21.15.131:0/212697575 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdbe8004300 con 0x7fdbf81096d0 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.160+0000 7fdbf6000700 1 -- 172.21.15.131:0/212697575 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe8011af0 con 0x7fdbf81096d0 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.160+0000 7fdbfe800700 1 -- 172.21.15.131:0/212697575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 msgr2=0x7fdbf810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.160+0000 7fdbfe800700 1 --2- 172.21.15.131:0/212697575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf810bac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80098a0 tx=0x7fdbe8009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.161+0000 7fdbfe800700 1 -- 172.21.15.131:0/212697575 shutdown_connections 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.161+0000 7fdbfe800700 1 --2- 172.21.15.131:0/212697575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf810bac0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.161+0000 7fdbfe800700 1 -- 172.21.15.131:0/212697575 >> 172.21.15.131:0/212697575 conn(0x7fdbf8103120 msgr2=0x7fdbf8105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.161+0000 7fdbfe800700 1 -- 172.21.15.131:0/212697575 shutdown_connections 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.161+0000 7fdbfe800700 1 -- 172.21.15.131:0/212697575 wait complete. 2024-04-26T19:58:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.162+0000 7fdbfe800700 1 Processor -- start 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.162+0000 7fdbfe800700 1 -- start start 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.162+0000 7fdbfe800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.162+0000 7fdbfe800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810cfd0 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.162+0000 7fdbf7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.162+0000 7fdbf7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45646/0 (socket says 172.21.15.131:45646) 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.162+0000 7fdbf7400700 1 -- 172.21.15.131:0/4101587764 learned_addr learned my addr 172.21.15.131:0/4101587764 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.163+0000 7fdbf7400700 1 -- 172.21.15.131:0/4101587764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf819f590 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.163+0000 7fdbf7400700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf819f050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8011ac0 tx=0x7fdbe8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.163+0000 7fdbefe00700 1 -- 172.21.15.131:0/4101587764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe8003b20 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.163+0000 7fdbefe00700 1 -- 172.21.15.131:0/4101587764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdbe801a8c0 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.163+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf819f820 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.163+0000 7fdbefe00700 1 -- 172.21.15.131:0/4101587764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe80248d0 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.163+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81a2510 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.165+0000 7fdbefe00700 1 -- 172.21.15.131:0/4101587764 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdbe8022020 con 0x7fdbf81096d0 2024-04-26T19:58:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.165+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf8064a80 con 0x7fdbf81096d0 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.165+0000 7fdbefe00700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fdbdc039600 0x7fdbdc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.165+0000 7fdbefe00700 1 -- 172.21.15.131:0/4101587764 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdbe804d830 con 0x7fdbf81096d0 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.165+0000 7fdbf6a00700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fdbdc039600 0x7fdbdc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.166+0000 7fdbf6a00700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fdbdc039600 0x7fdbdc03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4006fd0 tx=0x7fdbe4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.171+0000 7fdbefe00700 1 -- 172.21.15.131:0/4101587764 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdbe801ab20 con 0x7fdbf81096d0 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.397+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fdbf802e500 con 0x7fdbdc039600 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.399+0000 7fdbefe00700 1 -- 172.21.15.131:0/4101587764 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fdbf802e500 con 0x7fdbdc039600 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.402+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fdbdc039600 msgr2=0x7fdbdc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.402+0000 7fdbfe800700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fdbdc039600 0x7fdbdc03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4006fd0 tx=0x7fdbe4006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.402+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 msgr2=0x7fdbf819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.402+0000 7fdbfe800700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf819f050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8011ac0 tx=0x7fdbe8011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.403+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 shutdown_connections 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.403+0000 7fdbfe800700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fdbdc039600 0x7fdbdc03bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.403+0000 7fdbfe800700 1 --2- 172.21.15.131:0/4101587764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbf81096d0 0x7fdbf819f050 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:47.060 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.403+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 >> 172.21.15.131:0/4101587764 conn(0x7fdbf8103120 msgr2=0x7fdbf8103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:47.060 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.403+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 shutdown_connections 2024-04-26T19:58:47.060 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:46.403+0000 7fdbfe800700 1 -- 172.21.15.131:0/4101587764 wait complete. 2024-04-26T19:58:47.060 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T19:58:47.060 INFO:teuthology.orchestra.run.smithi131.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T19:58:47.064 INFO:teuthology.orchestra.run.smithi131.stdout:Adding host smithi131... 2024-04-26T19:58:47.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:47 smithi131 ceph-mon[27919]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:58:48.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:48 smithi131 ceph-mon[27919]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:58:49.468 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:49 smithi131 ceph-mon[27919]: Deploying cephadm binary to smithi131 2024-04-26T19:58:54.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Added host 'smithi131' with addr '172.21.15.131' 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.687+0000 7f55afc00700 1 Processor -- start 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.687+0000 7f55afc00700 1 -- start start 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.688+0000 7f55afc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.688+0000 7f55afc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55a80834b0 con 0x7f55a810fcf0 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.688+0000 7f55ad200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.688+0000 7f55ad200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45658/0 (socket says 172.21.15.131:45658) 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.688+0000 7f55ad200700 1 -- 172.21.15.131:0/699791172 learned_addr learned my addr 172.21.15.131:0/699791172 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.689+0000 7f55ad200700 1 -- 172.21.15.131:0/699791172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a8112620 con 0x7f55a810fcf0 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.689+0000 7f55ad200700 1 --2- 172.21.15.131:0/699791172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81120e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f559c0098a0 tx=0x7f559c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dafe99fb967df5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.689+0000 7f55a7400700 1 -- 172.21.15.131:0/699791172 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f559c004160 con 0x7f55a810fcf0 2024-04-26T19:58:54.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.689+0000 7f55a7400700 1 -- 172.21.15.131:0/699791172 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f559c004300 con 0x7f55a810fcf0 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.689+0000 7f55a7400700 1 -- 172.21.15.131:0/699791172 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f559c011af0 con 0x7f55a810fcf0 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.690+0000 7f55afc00700 1 -- 172.21.15.131:0/699791172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 msgr2=0x7f55a81120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.690+0000 7f55afc00700 1 --2- 172.21.15.131:0/699791172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81120e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f559c0098a0 tx=0x7f559c009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.690+0000 7f55afc00700 1 -- 172.21.15.131:0/699791172 shutdown_connections 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.690+0000 7f55afc00700 1 --2- 172.21.15.131:0/699791172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81120e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.690+0000 7f55afc00700 1 -- 172.21.15.131:0/699791172 >> 172.21.15.131:0/699791172 conn(0x7f55a807e210 msgr2=0x7f55a8080650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.690+0000 7f55afc00700 1 -- 172.21.15.131:0/699791172 shutdown_connections 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.690+0000 7f55afc00700 1 -- 172.21.15.131:0/699791172 wait complete. 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.691+0000 7f55afc00700 1 Processor -- start 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.691+0000 7f55afc00700 1 -- start start 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.692+0000 7f55afc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.692+0000 7f55afc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81135e0 con 0x7f55a810fcf0 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.692+0000 7f55ad200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.692+0000 7f55ad200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45668/0 (socket says 172.21.15.131:45668) 2024-04-26T19:58:54.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.692+0000 7f55ad200700 1 -- 172.21.15.131:0/928242717 learned_addr learned my addr 172.21.15.131:0/928242717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.692+0000 7f55ad200700 1 -- 172.21.15.131:0/928242717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a81a39d0 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.693+0000 7f55ad200700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81a3490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f559c009fb0 tx=0x7f559c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.693+0000 7f55a5600700 1 -- 172.21.15.131:0/928242717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f559c0036a0 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.693+0000 7f55a5600700 1 -- 172.21.15.131:0/928242717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f559c01a430 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.693+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a81a3c60 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.693+0000 7f55a5600700 1 -- 172.21.15.131:0/928242717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f559c019550 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.693+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a81a6950 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.694+0000 7f55a5600700 1 -- 172.21.15.131:0/928242717 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f559c003840 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.694+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a8064a80 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.695+0000 7f55a5600700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f55900395b0 0x7f559003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.695+0000 7f55a5600700 1 -- 172.21.15.131:0/928242717 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f559c04d940 con 0x7f55a810fcf0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.695+0000 7f55a7e00700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f55900395b0 0x7f559003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.696+0000 7f55a7e00700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f55900395b0 0x7f559003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5594006fd0 tx=0x7f5594006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:54.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.701+0000 7f55a5600700 1 -- 172.21.15.131:0/928242717 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f559c0172b0 con 0x7f55a810fcf0 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:47.931+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}) -- 0x7f55a802e500 con 0x7f55900395b0 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.038+0000 7f55a5600700 1 -- 172.21.15.131:0/928242717 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f55a802e500 con 0x7f55900395b0 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.042+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f55900395b0 msgr2=0x7f559003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.042+0000 7f55afc00700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f55900395b0 0x7f559003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5594006fd0 tx=0x7f5594006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 msgr2=0x7f55a81a3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81a3490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f559c009fb0 tx=0x7f559c011940 comp rx=0 tx=0).stop 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 shutdown_connections 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f55900395b0 0x7f559003ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 --2- 172.21.15.131:0/928242717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a810fcf0 0x7f55a81a3490 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 >> 172.21.15.131:0/928242717 conn(0x7f55a807e210 msgr2=0x7f55a807ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 shutdown_connections 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:54.043+0000 7f55afc00700 1 -- 172.21.15.131:0/928242717 wait complete. 2024-04-26T19:58:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mon service... 2024-04-26T19:58:55.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:55 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:55.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:55 smithi131 ceph-mon[27919]: Added host smithi131 2024-04-26T19:58:55.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:55 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:57.007 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T19:58:57.007 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.468+0000 7ff358000700 1 Processor -- start 2024-04-26T19:58:57.007 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.468+0000 7ff358000700 1 -- start start 2024-04-26T19:58:57.007 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.469+0000 7ff358000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff350107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.469+0000 7ff358000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff35007ce50 con 0x7ff350107560 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.469+0000 7ff355600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff350107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.470+0000 7ff355600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff350107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53350/0 (socket says 172.21.15.131:53350) 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.470+0000 7ff355600700 1 -- 172.21.15.131:0/758070876 learned_addr learned my addr 172.21.15.131:0/758070876 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.470+0000 7ff355600700 1 -- 172.21.15.131:0/758070876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff350107ec0 con 0x7ff350107560 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.471+0000 7ff355600700 1 --2- 172.21.15.131:0/758070876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff350107980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3440098a0 tx=0x7ff344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48664181c11584cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.471+0000 7ff34fe00700 1 -- 172.21.15.131:0/758070876 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff344004160 con 0x7ff350107560 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.471+0000 7ff34fe00700 1 -- 172.21.15.131:0/758070876 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff344004300 con 0x7ff350107560 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.471+0000 7ff34fe00700 1 -- 172.21.15.131:0/758070876 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff344011af0 con 0x7ff350107560 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.472+0000 7ff358000700 1 -- 172.21.15.131:0/758070876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 msgr2=0x7ff350107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:57.009 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.472+0000 7ff358000700 1 --2- 172.21.15.131:0/758070876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff350107980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3440098a0 tx=0x7ff344009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:57.010 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.472+0000 7ff358000700 1 -- 172.21.15.131:0/758070876 shutdown_connections 2024-04-26T19:58:57.010 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.472+0000 7ff358000700 1 --2- 172.21.15.131:0/758070876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff350107980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:57.010 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.472+0000 7ff358000700 1 -- 172.21.15.131:0/758070876 >> 172.21.15.131:0/758070876 conn(0x7ff350103120 msgr2=0x7ff350105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:57.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.472+0000 7ff358000700 1 -- 172.21.15.131:0/758070876 shutdown_connections 2024-04-26T19:58:57.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.472+0000 7ff358000700 1 -- 172.21.15.131:0/758070876 wait complete. 2024-04-26T19:58:57.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.473+0000 7ff358000700 1 Processor -- start 2024-04-26T19:58:57.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.473+0000 7ff358000700 1 -- start start 2024-04-26T19:58:57.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.473+0000 7ff358000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff35019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:57.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.473+0000 7ff358000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff350108ef0 con 0x7ff350107560 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.474+0000 7ff355600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff35019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.474+0000 7ff355600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff35019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53354/0 (socket says 172.21.15.131:53354) 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.474+0000 7ff355600700 1 -- 172.21.15.131:0/4188877008 learned_addr learned my addr 172.21.15.131:0/4188877008 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.474+0000 7ff355600700 1 -- 172.21.15.131:0/4188877008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35019f630 con 0x7ff350107560 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.474+0000 7ff355600700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff35019f0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff344011ac0 tx=0x7ff344011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.475+0000 7ff34e000700 1 -- 172.21.15.131:0/4188877008 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff344003970 con 0x7ff350107560 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.475+0000 7ff34e000700 1 -- 172.21.15.131:0/4188877008 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff344003b10 con 0x7ff350107560 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.475+0000 7ff34e000700 1 -- 172.21.15.131:0/4188877008 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff344024610 con 0x7ff350107560 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.475+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35019f8c0 con 0x7ff350107560 2024-04-26T19:58:57.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.475+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3501a25b0 con 0x7ff350107560 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.476+0000 7ff34e000700 1 -- 172.21.15.131:0/4188877008 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff34401a9e0 con 0x7ff350107560 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.476+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff350064a80 con 0x7ff350107560 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.476+0000 7ff34e000700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7ff338039600 0x7ff33803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.476+0000 7ff34e000700 1 -- 172.21.15.131:0/4188877008 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff34404d8c0 con 0x7ff350107560 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.477+0000 7ff354c00700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7ff338039600 0x7ff33803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.477+0000 7ff354c00700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7ff338039600 0x7ff33803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff33c006fd0 tx=0x7ff33c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.482+0000 7ff34e000700 1 -- 172.21.15.131:0/4188877008 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff34400b3e0 con 0x7ff350107560 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.751+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff35002e500 con 0x7ff338039600 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.760+0000 7ff34e000700 1 -- 172.21.15.131:0/4188877008 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff35002e500 con 0x7ff338039600 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7ff338039600 msgr2=0x7ff33803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7ff338039600 0x7ff33803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff33c006fd0 tx=0x7ff33c006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 msgr2=0x7ff35019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff35019f0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff344011ac0 tx=0x7ff344011ba0 comp rx=0 tx=0).stop 2024-04-26T19:58:57.016 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 shutdown_connections 2024-04-26T19:58:57.017 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7ff338039600 0x7ff33803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:57.017 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 --2- 172.21.15.131:0/4188877008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff350107560 0x7ff35019f0f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:57.017 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 >> 172.21.15.131:0/4188877008 conn(0x7ff350103120 msgr2=0x7ff3501959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:57.017 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 shutdown_connections 2024-04-26T19:58:57.017 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:55.763+0000 7ff358000700 1 -- 172.21.15.131:0/4188877008 wait complete. 2024-04-26T19:58:57.017 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mgr service... 2024-04-26T19:58:57.127 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:56 smithi131 ceph-mon[27919]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:58:57.128 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:56 smithi131 ceph-mon[27919]: Saving service mon spec with placement count:5 2024-04-26T19:58:57.128 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:56 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:58 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:58 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:58 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:58:59.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T19:58:59.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.105+0000 7f3ea8800700 1 Processor -- start 2024-04-26T19:58:59.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.105+0000 7f3ea8800700 1 -- start start 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.106+0000 7f3ea8800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea010a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.106+0000 7f3ea8800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0108860 con 0x7f3ea010a320 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.106+0000 7f3ea5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea010a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.106+0000 7f3ea5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea010a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53358/0 (socket says 172.21.15.131:53358) 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.106+0000 7f3ea5e00700 1 -- 172.21.15.131:0/4294732486 learned_addr learned my addr 172.21.15.131:0/4294732486 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.107+0000 7f3ea5e00700 1 -- 172.21.15.131:0/4294732486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01089e0 con 0x7f3ea010a320 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.107+0000 7f3ea5e00700 1 --2- 172.21.15.131:0/4294732486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea010a740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098a0 tx=0x7f3e94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddbe0aa25889e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.107+0000 7f3ea4a00700 1 -- 172.21.15.131:0/4294732486 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94004160 con 0x7f3ea010a320 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.107+0000 7f3ea4a00700 1 -- 172.21.15.131:0/4294732486 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e94004300 con 0x7f3ea010a320 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.107+0000 7f3ea4a00700 1 -- 172.21.15.131:0/4294732486 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94011a20 con 0x7f3ea010a320 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.108+0000 7f3ea8800700 1 -- 172.21.15.131:0/4294732486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 msgr2=0x7f3ea010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.108+0000 7f3ea8800700 1 --2- 172.21.15.131:0/4294732486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea010a740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098a0 tx=0x7f3e94009bb0 comp rx=0 tx=0).stop 2024-04-26T19:58:59.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.108+0000 7f3ea8800700 1 -- 172.21.15.131:0/4294732486 shutdown_connections 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.108+0000 7f3ea8800700 1 --2- 172.21.15.131:0/4294732486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea010a740 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.108+0000 7f3ea8800700 1 -- 172.21.15.131:0/4294732486 >> 172.21.15.131:0/4294732486 conn(0x7f3ea0104200 msgr2=0x7f3ea0106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.108+0000 7f3ea8800700 1 -- 172.21.15.131:0/4294732486 shutdown_connections 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.108+0000 7f3ea8800700 1 -- 172.21.15.131:0/4294732486 wait complete. 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.110+0000 7f3ea8800700 1 Processor -- start 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.110+0000 7f3ea8800700 1 -- start start 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.110+0000 7f3ea8800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.110+0000 7f3ea8800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010d1f0 con 0x7f3ea010a320 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.110+0000 7f3ea5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.110+0000 7f3ea5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53374/0 (socket says 172.21.15.131:53374) 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.110+0000 7f3ea5e00700 1 -- 172.21.15.131:0/3299112070 learned_addr learned my addr 172.21.15.131:0/3299112070 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.111+0000 7f3ea5e00700 1 -- 172.21.15.131:0/3299112070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea019d4a0 con 0x7f3ea010a320 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.111+0000 7f3ea5e00700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea019cf60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009520 tx=0x7f3e94011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.111+0000 7f3e9aa00700 1 -- 172.21.15.131:0/3299112070 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e940039e0 con 0x7f3ea010a320 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.111+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea019d730 con 0x7f3ea010a320 2024-04-26T19:58:59.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.111+0000 7f3e9aa00700 1 -- 172.21.15.131:0/3299112070 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9401b8f0 con 0x7f3ea010a320 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.111+0000 7f3e9aa00700 1 -- 172.21.15.131:0/3299112070 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94019770 con 0x7f3ea010a320 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.112+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01a03d0 con 0x7f3ea010a320 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.113+0000 7f3e9aa00700 1 -- 172.21.15.131:0/3299112070 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3e94019a20 con 0x7f3ea010a320 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.113+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea010d370 con 0x7f3ea010a320 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.113+0000 7f3e9aa00700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f3e88039600 0x7f3e8803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.113+0000 7f3e9aa00700 1 -- 172.21.15.131:0/3299112070 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e9404df70 con 0x7f3ea010a320 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.114+0000 7f3ea5400700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f3e88039600 0x7f3e8803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.114+0000 7f3ea5400700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f3e88039600 0x7f3e8803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.119+0000 7f3e9aa00700 1 -- 172.21.15.131:0/3299112070 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3e9401f030 con 0x7f3ea010a320 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.365+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3ea0063860 con 0x7f3e88039600 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.371+0000 7f3e9aa00700 1 -- 172.21.15.131:0/3299112070 <== mgr.14120 v2:172.21.15.131:6800/3789992830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3ea0063860 con 0x7f3e88039600 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.374+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f3e88039600 msgr2=0x7f3e8803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.374+0000 7f3ea8800700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f3e88039600 0x7f3e8803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90006e40 comp rx=0 tx=0).stop 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.374+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 msgr2=0x7f3ea019cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:58:59.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.374+0000 7f3ea8800700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea019cf60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009520 tx=0x7f3e94011aa0 comp rx=0 tx=0).stop 2024-04-26T19:58:59.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.375+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 shutdown_connections 2024-04-26T19:58:59.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.375+0000 7f3ea8800700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f3e88039600 0x7f3e8803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:59.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.375+0000 7f3ea8800700 1 --2- 172.21.15.131:0/3299112070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ea010a320 0x7f3ea019cf60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:58:59.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.375+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 >> 172.21.15.131:0/3299112070 conn(0x7f3ea0104200 msgr2=0x7f3ea0104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:58:59.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.375+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 shutdown_connections 2024-04-26T19:58:59.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:58.375+0000 7f3ea8800700 1 -- 172.21.15.131:0/3299112070 wait complete. 2024-04-26T19:58:59.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:59 smithi131 ceph-mon[27919]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:58:59.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:59 smithi131 ceph-mon[27919]: Saving service mgr spec with placement count:2 2024-04-26T19:58:59.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:58:59 smithi131 ceph-mon[27919]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:01.202 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.720+0000 7f4a2fc00700 1 Processor -- start 2024-04-26T19:59:01.202 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.720+0000 7f4a2fc00700 1 -- start start 2024-04-26T19:59:01.202 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.721+0000 7f4a2fc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2810b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:01.202 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.721+0000 7f4a2fc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2810b7a0 con 0x7f4a2810ae40 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.721+0000 7f4a2d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2810b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.721+0000 7f4a2d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2810b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53382/0 (socket says 172.21.15.131:53382) 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.721+0000 7f4a2d200700 1 -- 172.21.15.131:0/2991708230 learned_addr learned my addr 172.21.15.131:0/2991708230 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.722+0000 7f4a2d200700 1 -- 172.21.15.131:0/2991708230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2810bfa0 con 0x7f4a2810ae40 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.722+0000 7f4a2d200700 1 --2- 172.21.15.131:0/2991708230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2810b260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0098a0 tx=0x7f4a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a7bf21574e80d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.722+0000 7f4a27400700 1 -- 172.21.15.131:0/2991708230 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a1c004160 con 0x7f4a2810ae40 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.722+0000 7f4a27400700 1 -- 172.21.15.131:0/2991708230 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c004300 con 0x7f4a2810ae40 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.722+0000 7f4a27400700 1 -- 172.21.15.131:0/2991708230 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a1c011a20 con 0x7f4a2810ae40 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.723+0000 7f4a2fc00700 1 -- 172.21.15.131:0/2991708230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 msgr2=0x7f4a2810b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.723+0000 7f4a2fc00700 1 --2- 172.21.15.131:0/2991708230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2810b260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0098a0 tx=0x7f4a1c009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.723+0000 7f4a2fc00700 1 -- 172.21.15.131:0/2991708230 shutdown_connections 2024-04-26T19:59:01.203 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.723+0000 7f4a2fc00700 1 --2- 172.21.15.131:0/2991708230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2810b260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.723+0000 7f4a2fc00700 1 -- 172.21.15.131:0/2991708230 >> 172.21.15.131:0/2991708230 conn(0x7f4a2807e210 msgr2=0x7f4a2807e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.724+0000 7f4a2fc00700 1 -- 172.21.15.131:0/2991708230 shutdown_connections 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.724+0000 7f4a2fc00700 1 -- 172.21.15.131:0/2991708230 wait complete. 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.725+0000 7f4a2fc00700 1 Processor -- start 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.725+0000 7f4a2fc00700 1 -- start start 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.725+0000 7f4a2fc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.725+0000 7f4a2fc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2810f3e0 con 0x7f4a2810ae40 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.725+0000 7f4a2d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.725+0000 7f4a2d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53386/0 (socket says 172.21.15.131:53386) 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.725+0000 7f4a2d200700 1 -- 172.21.15.131:0/3669196447 learned_addr learned my addr 172.21.15.131:0/3669196447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.726+0000 7f4a2d200700 1 -- 172.21.15.131:0/3669196447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2819f580 con 0x7f4a2810ae40 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.726+0000 7f4a2d200700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2819f040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c004660 tx=0x7f4a1c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.726+0000 7f4a25600700 1 -- 172.21.15.131:0/3669196447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a1c003aa0 con 0x7f4a2810ae40 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.726+0000 7f4a25600700 1 -- 172.21.15.131:0/3669196447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c01b8f0 con 0x7f4a2810ae40 2024-04-26T19:59:01.204 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.726+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2819f810 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.726+0000 7f4a25600700 1 -- 172.21.15.131:0/3669196447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a1c023900 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.727+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a281a2500 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.728+0000 7f4a25600700 1 -- 172.21.15.131:0/3669196447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4a1c023b70 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.728+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a2810f560 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.728+0000 7f4a25600700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f4a10039600 0x7f4a1003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.728+0000 7f4a25600700 1 -- 172.21.15.131:0/3669196447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4a1c04df50 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.729+0000 7f4a27e00700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f4a10039600 0x7f4a1003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.729+0000 7f4a27e00700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f4a10039600 0x7f4a1003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006fd0 tx=0x7f4a14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.734+0000 7f4a25600700 1 -- 172.21.15.131:0/3669196447 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4a1c01f030 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.970+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4a2802e490 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.977+0000 7f4a25600700 1 -- 172.21.15.131:0/3669196447 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4a1c01bde0 con 0x7f4a2810ae40 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.980+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f4a10039600 msgr2=0x7f4a1003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.980+0000 7f4a2fc00700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f4a10039600 0x7f4a1003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006fd0 tx=0x7f4a14006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 msgr2=0x7f4a2819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:01.206 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2819f040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c004660 tx=0x7f4a1c011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:01.206 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 shutdown_connections 2024-04-26T19:59:01.206 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f4a10039600 0x7f4a1003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:01.206 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 --2- 172.21.15.131:0/3669196447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a2810ae40 0x7f4a2819f040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:01.206 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 >> 172.21.15.131:0/3669196447 conn(0x7f4a2807e210 msgr2=0x7f4a28108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:01.206 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 shutdown_connections 2024-04-26T19:59:01.206 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:58:59.981+0000 7f4a2fc00700 1 -- 172.21.15.131:0/3669196447 wait complete. 2024-04-26T19:59:01.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:00 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3669196447' entity='client.admin' 2024-04-26T19:59:02.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:01 smithi131 ceph-mon[27919]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:02.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:01 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.848+0000 7fb624c00700 1 Processor -- start 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.848+0000 7fb624c00700 1 -- start start 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.848+0000 7fb624c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62007d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.848+0000 7fb624c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb62007f070 con 0x7fb62007d340 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.849+0000 7fb61de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62007d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.849+0000 7fb61de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62007d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53402/0 (socket says 172.21.15.131:53402) 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.849+0000 7fb61de00700 1 -- 172.21.15.131:0/2580967432 learned_addr learned my addr 172.21.15.131:0/2580967432 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:02.795 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.849+0000 7fb61de00700 1 -- 172.21.15.131:0/2580967432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62007dca0 con 0x7fb62007d340 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.849+0000 7fb61de00700 1 --2- 172.21.15.131:0/2580967432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62007d760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098a0 tx=0x7fb610009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecb5b54168722c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.850+0000 7fb61ca00700 1 -- 172.21.15.131:0/2580967432 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb610004160 con 0x7fb62007d340 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.850+0000 7fb61ca00700 1 -- 172.21.15.131:0/2580967432 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb610004300 con 0x7fb62007d340 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.850+0000 7fb61ca00700 1 -- 172.21.15.131:0/2580967432 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb610011a20 con 0x7fb62007d340 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.851+0000 7fb624c00700 1 -- 172.21.15.131:0/2580967432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 msgr2=0x7fb62007d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.851+0000 7fb624c00700 1 --2- 172.21.15.131:0/2580967432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62007d760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098a0 tx=0x7fb610009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.851+0000 7fb624c00700 1 -- 172.21.15.131:0/2580967432 shutdown_connections 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.851+0000 7fb624c00700 1 --2- 172.21.15.131:0/2580967432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62007d760 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.851+0000 7fb624c00700 1 -- 172.21.15.131:0/2580967432 >> 172.21.15.131:0/2580967432 conn(0x7fb620104200 msgr2=0x7fb620106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.851+0000 7fb624c00700 1 -- 172.21.15.131:0/2580967432 shutdown_connections 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.851+0000 7fb624c00700 1 -- 172.21.15.131:0/2580967432 wait complete. 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.852+0000 7fb624c00700 1 Processor -- start 2024-04-26T19:59:02.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.852+0000 7fb624c00700 1 -- start start 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.852+0000 7fb624c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.852+0000 7fb624c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010f2e0 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.853+0000 7fb61de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.853+0000 7fb61de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53414/0 (socket says 172.21.15.131:53414) 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.853+0000 7fb61de00700 1 -- 172.21.15.131:0/3740938541 learned_addr learned my addr 172.21.15.131:0/3740938541 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.853+0000 7fb61de00700 1 -- 172.21.15.131:0/3740938541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62019f5d0 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.853+0000 7fb61de00700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62019f090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb610000c00 tx=0x7fb610011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.854+0000 7fb616a00700 1 -- 172.21.15.131:0/3740938541 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb610003aa0 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.854+0000 7fb616a00700 1 -- 172.21.15.131:0/3740938541 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61001b8f0 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.854+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62019f860 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.854+0000 7fb616a00700 1 -- 172.21.15.131:0/3740938541 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb610023870 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.854+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201a2550 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.855+0000 7fb616a00700 1 -- 172.21.15.131:0/3740938541 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb610023b70 con 0x7fb62007d340 2024-04-26T19:59:02.797 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.855+0000 7fb616a00700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fb6040395b0 0x7fb60403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.855+0000 7fb616a00700 1 -- 172.21.15.131:0/3740938541 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb61004e0c0 con 0x7fb62007d340 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.855+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb620064a80 con 0x7fb62007d340 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.856+0000 7fb61d400700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fb6040395b0 0x7fb60403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.856+0000 7fb61d400700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fb6040395b0 0x7fb60403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006fd0 tx=0x7fb60c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:01.861+0000 7fb616a00700 1 -- 172.21.15.131:0/3740938541 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb610031020 con 0x7fb62007d340 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.114+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb620076e90 con 0x7fb62007d340 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.118+0000 7fb616a00700 1 -- 172.21.15.131:0/3740938541 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb61001f030 con 0x7fb62007d340 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.121+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fb6040395b0 msgr2=0x7fb60403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.121+0000 7fb624c00700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fb6040395b0 0x7fb60403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006fd0 tx=0x7fb60c006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.121+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 msgr2=0x7fb62019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.122+0000 7fb624c00700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62019f090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb610000c00 tx=0x7fb610011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.122+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 shutdown_connections 2024-04-26T19:59:02.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.122+0000 7fb624c00700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fb6040395b0 0x7fb60403ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:02.799 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.122+0000 7fb624c00700 1 --2- 172.21.15.131:0/3740938541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb62007d340 0x7fb62019f090 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:02.799 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.122+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 >> 172.21.15.131:0/3740938541 conn(0x7fb620104200 msgr2=0x7fb620104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:02.799 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.122+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 shutdown_connections 2024-04-26T19:59:02.799 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:02.122+0000 7fb624c00700 1 -- 172.21.15.131:0/3740938541 wait complete. 2024-04-26T19:59:02.799 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling the dashboard module... 2024-04-26T19:59:03.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:03 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3740938541' entity='client.admin' 2024-04-26T19:59:04.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:04 smithi131 ceph-mon[27919]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:04.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:04 smithi131 ceph-mon[27919]: from='mgr.14120 172.21.15.131:0/578704296' entity='mgr.x' 2024-04-26T19:59:04.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:04 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/595757907' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T19:59:04.669 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: ignoring --setuser ceph since I am not root 2024-04-26T19:59:04.669 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: ignoring --setgroup ceph since I am not root 2024-04-26T19:59:04.669 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:04.668+0000 7f6406942200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:59:05.179 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:04.921+0000 7f6406942200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:59:05.179 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:05.057+0000 7f6406942200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:59:05.474 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:05 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/595757907' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T19:59:05.474 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:05 smithi131 ceph-mon[27919]: mgrmap e9: x(active, since 25s) 2024-04-26T19:59:05.575 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.397+0000 7fe82c200700 1 Processor -- start 2024-04-26T19:59:05.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.397+0000 7fe82c200700 1 -- start start 2024-04-26T19:59:05.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.397+0000 7fe82c200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82407d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:05.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.397+0000 7fe82c200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe82407f050 con 0x7fe82407d320 2024-04-26T19:59:05.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.398+0000 7fe829800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:05.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.398+0000 7fe829800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82407d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53428/0 (socket says 172.21.15.131:53428) 2024-04-26T19:59:05.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.398+0000 7fe829800700 1 -- 172.21.15.131:0/3337143827 learned_addr learned my addr 172.21.15.131:0/3337143827 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:05.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.398+0000 7fe829800700 1 -- 172.21.15.131:0/3337143827 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82407dc80 con 0x7fe82407d320 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.398+0000 7fe829800700 1 --2- 172.21.15.131:0/3337143827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82407d740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8180098a0 tx=0x7fe818009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15653bbcfcfd961 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.399+0000 7fe823e00700 1 -- 172.21.15.131:0/3337143827 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe818004160 con 0x7fe82407d320 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.399+0000 7fe823e00700 1 -- 172.21.15.131:0/3337143827 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe818004300 con 0x7fe82407d320 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.399+0000 7fe823e00700 1 -- 172.21.15.131:0/3337143827 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe818011a20 con 0x7fe82407d320 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.399+0000 7fe82c200700 1 -- 172.21.15.131:0/3337143827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 msgr2=0x7fe82407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.399+0000 7fe82c200700 1 --2- 172.21.15.131:0/3337143827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82407d740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8180098a0 tx=0x7fe818009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.399+0000 7fe82c200700 1 -- 172.21.15.131:0/3337143827 shutdown_connections 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.400+0000 7fe82c200700 1 --2- 172.21.15.131:0/3337143827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82407d740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.400+0000 7fe82c200700 1 -- 172.21.15.131:0/3337143827 >> 172.21.15.131:0/3337143827 conn(0x7fe824104200 msgr2=0x7fe824106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.400+0000 7fe82c200700 1 -- 172.21.15.131:0/3337143827 shutdown_connections 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.400+0000 7fe82c200700 1 -- 172.21.15.131:0/3337143827 wait complete. 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.401+0000 7fe82c200700 1 Processor -- start 2024-04-26T19:59:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.401+0000 7fe82c200700 1 -- start start 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.401+0000 7fe82c200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.401+0000 7fe82c200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410f300 con 0x7fe82407d320 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.401+0000 7fe829800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.402+0000 7fe829800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53440/0 (socket says 172.21.15.131:53440) 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.402+0000 7fe829800700 1 -- 172.21.15.131:0/595757907 learned_addr learned my addr 172.21.15.131:0/595757907 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.402+0000 7fe829800700 1 -- 172.21.15.131:0/595757907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82419f5f0 con 0x7fe82407d320 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.402+0000 7fe829800700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82419f0b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe818000c00 tx=0x7fe818011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.403+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe818003aa0 con 0x7fe82407d320 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.403+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe81801b8f0 con 0x7fe82407d320 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.403+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82419f880 con 0x7fe82407d320 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.403+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8241a2570 con 0x7fe82407d320 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.403+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe818023870 con 0x7fe82407d320 2024-04-26T19:59:05.583 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.404+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe81801b440 con 0x7fe82407d320 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.404+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe824064a80 con 0x7fe82407d320 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.405+0000 7fe822000700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fe80c039600 0x7fe80c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.405+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe81804dea0 con 0x7fe82407d320 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.405+0000 7fe828e00700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fe80c039600 0x7fe80c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.406+0000 7fe828e00700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fe80c039600 0x7fe80c03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe810006fd0 tx=0x7fe810006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.410+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe81801f030 con 0x7fe82407d320 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:03.693+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fe82402e490 con 0x7fe82407d320 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.349+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fe81801bd30 con 0x7fe82407d320 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.354+0000 7fe822000700 1 -- 172.21.15.131:0/595757907 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe818053020 con 0x7fe82407d320 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.355+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fe80c039600 msgr2=0x7fe80c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.355+0000 7fe82c200700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fe80c039600 0x7fe80c03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe810006fd0 tx=0x7fe810006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.355+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 msgr2=0x7fe82419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.355+0000 7fe82c200700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82419f0b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe818000c00 tx=0x7fe818011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.356+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 shutdown_connections 2024-04-26T19:59:05.585 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.356+0000 7fe82c200700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fe80c039600 0x7fe80c03bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:05.585 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.356+0000 7fe82c200700 1 --2- 172.21.15.131:0/595757907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe82407d320 0x7fe82419f0b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:05.585 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.356+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 >> 172.21.15.131:0/595757907 conn(0x7fe824104200 msgr2=0x7fe824104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:05.585 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.356+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 shutdown_connections 2024-04-26T19:59:05.585 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:04.356+0000 7fe82c200700 1 -- 172.21.15.131:0/595757907 wait complete. 2024-04-26T19:59:05.725 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:05.472+0000 7f6406942200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:59:05.725 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:05.590+0000 7f6406942200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:59:06.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:06 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2635628092' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T19:59:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:59:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T19:59:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T19:59:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T19:59:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T19:59:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:59:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.282+0000 7fee2de00700 1 Processor -- start 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.282+0000 7fee2de00700 1 -- start start 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.283+0000 7fee2de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2807d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.283+0000 7fee2de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee2807f050 con 0x7fee2807d320 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.283+0000 7fee26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.283+0000 7fee26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2807d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55456/0 (socket says 172.21.15.131:55456) 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.283+0000 7fee26a00700 1 -- 172.21.15.131:0/693468295 learned_addr learned my addr 172.21.15.131:0/693468295 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.284+0000 7fee26a00700 1 -- 172.21.15.131:0/693468295 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2807dc80 con 0x7fee2807d320 2024-04-26T19:59:07.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.284+0000 7fee26a00700 1 --2- 172.21.15.131:0/693468295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2807d740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee180098a0 tx=0x7fee18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45091c159e2ec95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.284+0000 7fee25600700 1 -- 172.21.15.131:0/693468295 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee18004160 con 0x7fee2807d320 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.284+0000 7fee25600700 1 -- 172.21.15.131:0/693468295 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee18004300 con 0x7fee2807d320 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.284+0000 7fee25600700 1 -- 172.21.15.131:0/693468295 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee18011a20 con 0x7fee2807d320 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.285+0000 7fee2de00700 1 -- 172.21.15.131:0/693468295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 msgr2=0x7fee2807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.285+0000 7fee2de00700 1 --2- 172.21.15.131:0/693468295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2807d740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee180098a0 tx=0x7fee18009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.285+0000 7fee2de00700 1 -- 172.21.15.131:0/693468295 shutdown_connections 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.285+0000 7fee2de00700 1 --2- 172.21.15.131:0/693468295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2807d740 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.285+0000 7fee2de00700 1 -- 172.21.15.131:0/693468295 >> 172.21.15.131:0/693468295 conn(0x7fee28104200 msgr2=0x7fee28106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.285+0000 7fee2de00700 1 -- 172.21.15.131:0/693468295 shutdown_connections 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.285+0000 7fee2de00700 1 -- 172.21.15.131:0/693468295 wait complete. 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.286+0000 7fee2de00700 1 Processor -- start 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.286+0000 7fee2de00700 1 -- start start 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.287+0000 7fee2de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:07.343 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.287+0000 7fee2de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee2810f300 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.287+0000 7fee26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.287+0000 7fee26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55464/0 (socket says 172.21.15.131:55464) 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.287+0000 7fee26a00700 1 -- 172.21.15.131:0/2635628092 learned_addr learned my addr 172.21.15.131:0/2635628092 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.287+0000 7fee26a00700 1 -- 172.21.15.131:0/2635628092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2819f5d0 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.288+0000 7fee26a00700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2819f090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee18000c00 tx=0x7fee18011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.288+0000 7fee2ca00700 1 -- 172.21.15.131:0/2635628092 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee18003aa0 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.288+0000 7fee2ca00700 1 -- 172.21.15.131:0/2635628092 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1801b8f0 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.288+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2819f860 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.288+0000 7fee2ca00700 1 -- 172.21.15.131:0/2635628092 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee18023870 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.288+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281a2550 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.289+0000 7fee2ca00700 1 -- 172.21.15.131:0/2635628092 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fee18023b20 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.289+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee28064a80 con 0x7fee2807d320 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.290+0000 7fee2ca00700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 0x7fee0c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:07.344 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.290+0000 7fee2ca00700 1 -- 172.21.15.131:0/2635628092 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fee1804ded0 con 0x7fee2807d320 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.290+0000 7fee26000700 1 -- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 msgr2=0x7fee0c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3789992830 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.290+0000 7fee26000700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 0x7fee0c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.295+0000 7fee2ca00700 1 -- 172.21.15.131:0/2635628092 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fee1801f030 con 0x7fee2807d320 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.491+0000 7fee26000700 1 -- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 msgr2=0x7fee0c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3789992830 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.491+0000 7fee26000700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 0x7fee0c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.573+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fee2802e490 con 0x7fee2807d320 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.574+0000 7fee2ca00700 1 -- 172.21.15.131:0/2635628092 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fee18016260 con 0x7fee2807d320 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 msgr2=0x7fee0c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 0x7fee0c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 msgr2=0x7fee2819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2819f090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee18000c00 tx=0x7fee18011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 shutdown_connections 2024-04-26T19:59:07.345 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7fee0c039660 0x7fee0c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:07.346 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 --2- 172.21.15.131:0/2635628092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2807d320 0x7fee2819f090 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:07.346 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.577+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 >> 172.21.15.131:0/2635628092 conn(0x7fee28104200 msgr2=0x7fee28104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:07.346 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.578+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 shutdown_connections 2024-04-26T19:59:07.346 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:06.578+0000 7fee2de00700 1 -- 172.21.15.131:0/2635628092 wait complete. 2024-04-26T19:59:07.346 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-04-26T19:59:07.346 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 9... 2024-04-26T19:59:07.490 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:07.221+0000 7f6406942200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:59:07.491 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:07.454+0000 7f6406942200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:59:07.756 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:07.680+0000 7f6406942200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:59:08.452 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:08.072+0000 7f6406942200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:59:08.452 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:08.195+0000 7f6406942200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:59:08.706 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:08.450+0000 7f6406942200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:59:08.706 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:08.704+0000 7f6406942200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:59:09.756 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:09.488+0000 7f6406942200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:59:10.432 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:10.034+0000 7f6406942200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:59:10.433 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:10.165+0000 7f6406942200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:59:10.433 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:10.431+0000 7f6406942200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:59:11.230 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:11.104+0000 7f6406942200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:59:11.506 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:11.228+0000 7f6406942200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:59:11.847 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:11.547+0000 7f6406942200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:59:11.847 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:11.719+0000 7f6406942200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:59:11.847 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:11.846+0000 7f6406942200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:59:12.256 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:11.996+0000 7f6406942200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:59:12.756 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:12 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:12.400+0000 7f6406942200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:59:13.442 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:13.153+0000 7f6406942200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:59:13.756 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T19:59:13.440+0000 7f6406942200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:59:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: Active manager daemon x restarted 2024-04-26T19:59:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: Activating manager daemon x 2024-04-26T19:59:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T19:59:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: mgrmap e10: x(active, starting, since 0.0078689s) 2024-04-26T19:59:13.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T19:59:13.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T19:59:13.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T19:59:13.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T19:59:13.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T19:59:13.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:13 smithi131 ceph-mon[27919]: Manager daemon x is now available 2024-04-26T19:59:14.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:14.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T19:59:14.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T19:59:14.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:14 smithi131 ceph-mon[27919]: mgrmap e11: x(active, since 1.01001s) 2024-04-26T19:59:15.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-04-26T19:59:15.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T19:59:15.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T19:59:15.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.945+0000 7f95df600700 1 Processor -- start 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.946+0000 7f95df600700 1 -- start start 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.946+0000 7f95df600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d810a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.946+0000 7f95df600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8108860 con 0x7f95d810a320 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.946+0000 7f95dcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d810a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.947+0000 7f95dcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d810a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55474/0 (socket says 172.21.15.131:55474) 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.947+0000 7f95dcc00700 1 -- 172.21.15.131:0/3681660199 learned_addr learned my addr 172.21.15.131:0/3681660199 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.947+0000 7f95dcc00700 1 -- 172.21.15.131:0/3681660199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d81089e0 con 0x7f95d810a320 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.947+0000 7f95dcc00700 1 --2- 172.21.15.131:0/3681660199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d810a740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0098a0 tx=0x7f95cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998fcbacf8ccd011 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.948+0000 7f95d7400700 1 -- 172.21.15.131:0/3681660199 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95cc004160 con 0x7f95d810a320 2024-04-26T19:59:15.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.948+0000 7f95d7400700 1 -- 172.21.15.131:0/3681660199 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95cc004300 con 0x7f95d810a320 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.948+0000 7f95d7400700 1 -- 172.21.15.131:0/3681660199 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95cc011a20 con 0x7f95d810a320 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.948+0000 7f95df600700 1 -- 172.21.15.131:0/3681660199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 msgr2=0x7f95d810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.948+0000 7f95df600700 1 --2- 172.21.15.131:0/3681660199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d810a740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0098a0 tx=0x7f95cc009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.949+0000 7f95df600700 1 -- 172.21.15.131:0/3681660199 shutdown_connections 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.949+0000 7f95df600700 1 --2- 172.21.15.131:0/3681660199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d810a740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.949+0000 7f95df600700 1 -- 172.21.15.131:0/3681660199 >> 172.21.15.131:0/3681660199 conn(0x7f95d8104200 msgr2=0x7f95d8106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.949+0000 7f95df600700 1 -- 172.21.15.131:0/3681660199 shutdown_connections 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.949+0000 7f95df600700 1 -- 172.21.15.131:0/3681660199 wait complete. 2024-04-26T19:59:15.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.950+0000 7f95df600700 1 Processor -- start 2024-04-26T19:59:15.107 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.950+0000 7f95df600700 1 -- start start 2024-04-26T19:59:15.107 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.951+0000 7f95df600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d807eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:15.107 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.951+0000 7f95df600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95d810d1f0 con 0x7f95d810a320 2024-04-26T19:59:15.107 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.951+0000 7f95dcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d807eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.951+0000 7f95dcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d807eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55480/0 (socket says 172.21.15.131:55480) 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.951+0000 7f95dcc00700 1 -- 172.21.15.131:0/2394366430 learned_addr learned my addr 172.21.15.131:0/2394366430 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.952+0000 7f95dcc00700 1 -- 172.21.15.131:0/2394366430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d807f080 con 0x7f95d810a320 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.952+0000 7f95dcc00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d807eb40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009520 tx=0x7f95cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.952+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95cc0039e0 con 0x7f95d810a320 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.952+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95cc01b8f0 con 0x7f95d810a320 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.952+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95cc019770 con 0x7f95d810a320 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.952+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d807d2c0 con 0x7f95d810a320 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.952+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d807d7a0 con 0x7f95d810a320 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.954+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f95cc01b440 con 0x7f95d810a320 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.954+0000 7f95d5600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.954+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 --> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f95c003c220 con 0x7f95c0039650 2024-04-26T19:59:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.954+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95cc04dfd0 con 0x7f95d810a320 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.954+0000 7f95d7e00700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 msgr2=0x7f95c003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3789992830 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:07.955+0000 7f95d7e00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:08.156+0000 7f95d7e00700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 msgr2=0x7f95c003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3789992830 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:08.156+0000 7f95d7e00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:08.557+0000 7f95d7e00700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 msgr2=0x7f95c003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3789992830 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:08.557+0000 7f95d7e00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:09.358+0000 7f95d7e00700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 msgr2=0x7f95c003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3789992830 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:09.358+0000 7f95d7e00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:10.960+0000 7f95d7e00700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 msgr2=0x7f95c003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3789992830 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:10.960+0000 7f95d7e00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:13.448+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f95cc019c90 con 0x7f95d810a320 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:13.448+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 msgr2=0x7f95c003bb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:13.448+0000 7f95d5600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.449+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f95cc04fa20 con 0x7f95d810a320 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.450+0000 7f95d5600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f95c003cfe0 0x7f95c003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:15.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.450+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f95c003c220 con 0x7f95c003cfe0 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.451+0000 7f95d7e00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f95c003cfe0 0x7f95c003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.452+0000 7f95d7e00700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f95c003cfe0 0x7f95c003f3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95c4003a10 tx=0x7f95c4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.453+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f95c003c220 con 0x7f95c003cfe0 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.462+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f95d810d040 con 0x7f95c003cfe0 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.462+0000 7f95d5600700 1 -- 172.21.15.131:0/2394366430 <== mgr.14152 v2:172.21.15.131:6800/2537375548 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f95d810d040 con 0x7f95c003cfe0 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f95c003cfe0 msgr2=0x7f95c003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f95c003cfe0 0x7f95c003f3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95c4003a10 tx=0x7f95c4009270 comp rx=0 tx=0).stop 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 msgr2=0x7f95d807eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d807eb40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009520 tx=0x7f95cc011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 shutdown_connections 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f95c003cfe0 0x7f95c003f3d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d810a320 0x7f95d807eb40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:15.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 --2- 172.21.15.131:0/2394366430 >> [v2:172.21.15.131:6800/3789992830,v1:172.21.15.131:6801/3789992830] conn(0x7f95c0039650 0x7f95c003bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:15.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 >> 172.21.15.131:0/2394366430 conn(0x7f95d8104200 msgr2=0x7f95d8104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:15.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 shutdown_connections 2024-04-26T19:59:15.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:14.463+0000 7f95df600700 1 -- 172.21.15.131:0/2394366430 wait complete. 2024-04-26T19:59:15.111 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 9 is available 2024-04-26T19:59:15.111 INFO:teuthology.orchestra.run.smithi131.stdout:Generating a dashboard self-signed certificate... 2024-04-26T19:59:15.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:15 smithi131 ceph-mon[27919]: [26/Apr/2024:19:59:14] ENGINE Bus STARTING 2024-04-26T19:59:15.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:15 smithi131 ceph-mon[27919]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T19:59:15.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:15 smithi131 ceph-mon[27919]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T19:59:15.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:15 smithi131 ceph-mon[27919]: [26/Apr/2024:19:59:14] ENGINE Serving on https://172.21.15.131:7150 2024-04-26T19:59:16.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:16 smithi131 ceph-mon[27919]: [26/Apr/2024:19:59:14] ENGINE Serving on http://172.21.15.131:8765 2024-04-26T19:59:16.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:16 smithi131 ceph-mon[27919]: [26/Apr/2024:19:59:14] ENGINE Bus STARTED 2024-04-26T19:59:16.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:16 smithi131 ceph-mon[27919]: mgrmap e12: x(active, since 2s) 2024-04-26T19:59:16.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:17.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T19:59:17.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.196+0000 7f0898200700 1 Processor -- start 2024-04-26T19:59:17.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.196+0000 7f0898200700 1 -- start start 2024-04-26T19:59:17.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.197+0000 7f0898200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f0890109060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:17.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.197+0000 7f0898200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0890109630 con 0x7f0890108c40 2024-04-26T19:59:17.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.197+0000 7f0895800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f0890109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.198+0000 7f0895800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f0890109060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59472/0 (socket says 172.21.15.131:59472) 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.198+0000 7f0895800700 1 -- 172.21.15.131:0/1993359206 learned_addr learned my addr 172.21.15.131:0/1993359206 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.198+0000 7f0895800700 1 -- 172.21.15.131:0/1993359206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0890109e40 con 0x7f0890108c40 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.198+0000 7f0895800700 1 --2- 172.21.15.131:0/1993359206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f0890109060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f08840098a0 tx=0x7f0884009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8354064df5b05ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.199+0000 7f088fe00700 1 -- 172.21.15.131:0/1993359206 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0884004160 con 0x7f0890108c40 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.199+0000 7f088fe00700 1 -- 172.21.15.131:0/1993359206 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0884004300 con 0x7f0890108c40 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.199+0000 7f088fe00700 1 -- 172.21.15.131:0/1993359206 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0884011a20 con 0x7f0890108c40 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.200+0000 7f0898200700 1 -- 172.21.15.131:0/1993359206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 msgr2=0x7f0890109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.200+0000 7f0898200700 1 --2- 172.21.15.131:0/1993359206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f0890109060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f08840098a0 tx=0x7f0884009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.200+0000 7f0898200700 1 -- 172.21.15.131:0/1993359206 shutdown_connections 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.200+0000 7f0898200700 1 --2- 172.21.15.131:0/1993359206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f0890109060 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.200+0000 7f0898200700 1 -- 172.21.15.131:0/1993359206 >> 172.21.15.131:0/1993359206 conn(0x7f0890104200 msgr2=0x7f0890106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.200+0000 7f0898200700 1 -- 172.21.15.131:0/1993359206 shutdown_connections 2024-04-26T19:59:17.303 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.200+0000 7f0898200700 1 -- 172.21.15.131:0/1993359206 wait complete. 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.202+0000 7f0898200700 1 Processor -- start 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.202+0000 7f0898200700 1 -- start start 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.202+0000 7f0898200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f089007eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.202+0000 7f0898200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f089010d1f0 con 0x7f0890108c40 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.203+0000 7f0895800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f089007eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.203+0000 7f0895800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f089007eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59476/0 (socket says 172.21.15.131:59476) 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.203+0000 7f0895800700 1 -- 172.21.15.131:0/3763322128 learned_addr learned my addr 172.21.15.131:0/3763322128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.203+0000 7f0895800700 1 -- 172.21.15.131:0/3763322128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089007f0d0 con 0x7f0890108c40 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.204+0000 7f0895800700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f089007eb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08840119f0 tx=0x7f0884011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.204+0000 7f088e000700 1 -- 172.21.15.131:0/3763322128 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08840039e0 con 0x7f0890108c40 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.204+0000 7f088e000700 1 -- 172.21.15.131:0/3763322128 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088401b8f0 con 0x7f0890108c40 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.204+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089007d2c0 con 0x7f0890108c40 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.204+0000 7f088e000700 1 -- 172.21.15.131:0/3763322128 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08840237a0 con 0x7f0890108c40 2024-04-26T19:59:17.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.204+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089007d800 con 0x7f0890108c40 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.206+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0890064a80 con 0x7f0890108c40 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.206+0000 7f088e000700 1 -- 172.21.15.131:0/3763322128 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f088401b440 con 0x7f0890108c40 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.207+0000 7f088e000700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0878039610 0x7f087803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.207+0000 7f088e000700 1 -- 172.21.15.131:0/3763322128 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f088404e0b0 con 0x7f0890108c40 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.207+0000 7f0894e00700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0878039610 0x7f087803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.208+0000 7f0894e00700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0878039610 0x7f087803bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f087c006fd0 tx=0x7f087c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.213+0000 7f088e000700 1 -- 172.21.15.131:0/3763322128 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f088401f030 con 0x7f0890108c40 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.450+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f08901965d0 con 0x7f0878039610 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.581+0000 7f088e000700 1 -- 172.21.15.131:0/3763322128 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f08901965d0 con 0x7f0878039610 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0878039610 msgr2=0x7f087803bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0878039610 0x7f087803bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f087c006fd0 tx=0x7f087c006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 msgr2=0x7f089007eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f089007eb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08840119f0 tx=0x7f0884011ad0 comp rx=0 tx=0).stop 2024-04-26T19:59:17.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 shutdown_connections 2024-04-26T19:59:17.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0878039610 0x7f087803bad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:17.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 --2- 172.21.15.131:0/3763322128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0890108c40 0x7f089007eb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:17.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 >> 172.21.15.131:0/3763322128 conn(0x7f0890104200 msgr2=0x7f0890104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:17.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 shutdown_connections 2024-04-26T19:59:17.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:16.585+0000 7f0898200700 1 -- 172.21.15.131:0/3763322128 wait complete. 2024-04-26T19:59:17.306 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial admin user... 2024-04-26T19:59:17.632 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:17 smithi131 ceph-mon[27919]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:59:17.632 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:17 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:17.632 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:17 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:19.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ErhVHLffaEe/Nur4u14EGOBZuyZzogMFeG/JOEnAEbT15PwZ5x.r2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714161558, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.935+0000 7fc6b1200700 1 Processor -- start 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.935+0000 7fc6b1200700 1 -- start start 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.936+0000 7fc6b1200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac1078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.936+0000 7fc6b1200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac07ced0 con 0x7fc6ac1074c0 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.936+0000 7fc6a9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac1078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.936+0000 7fc6a9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac1078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59482/0 (socket says 172.21.15.131:59482) 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.936+0000 7fc6a9e00700 1 -- 172.21.15.131:0/375719986 learned_addr learned my addr 172.21.15.131:0/375719986 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.937+0000 7fc6a9e00700 1 -- 172.21.15.131:0/375719986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac107e20 con 0x7fc6ac1074c0 2024-04-26T19:59:19.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.937+0000 7fc6a9e00700 1 --2- 172.21.15.131:0/375719986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac1078e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0098a0 tx=0x7fc69c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d54a84f4cd0546 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.937+0000 7fc6a8a00700 1 -- 172.21.15.131:0/375719986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c004160 con 0x7fc6ac1074c0 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.937+0000 7fc6a8a00700 1 -- 172.21.15.131:0/375719986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc69c004300 con 0x7fc6ac1074c0 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.937+0000 7fc6a8a00700 1 -- 172.21.15.131:0/375719986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c011a20 con 0x7fc6ac1074c0 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.938+0000 7fc6b1200700 1 -- 172.21.15.131:0/375719986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 msgr2=0x7fc6ac1078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.938+0000 7fc6b1200700 1 --2- 172.21.15.131:0/375719986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac1078e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0098a0 tx=0x7fc69c009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.938+0000 7fc6b1200700 1 -- 172.21.15.131:0/375719986 shutdown_connections 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.938+0000 7fc6b1200700 1 --2- 172.21.15.131:0/375719986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac1078e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.938+0000 7fc6b1200700 1 -- 172.21.15.131:0/375719986 >> 172.21.15.131:0/375719986 conn(0x7fc6ac103120 msgr2=0x7fc6ac105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.938+0000 7fc6b1200700 1 -- 172.21.15.131:0/375719986 shutdown_connections 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.939+0000 7fc6b1200700 1 -- 172.21.15.131:0/375719986 wait complete. 2024-04-26T19:59:19.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.940+0000 7fc6b1200700 1 Processor -- start 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.940+0000 7fc6b1200700 1 -- start start 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.940+0000 7fc6b1200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.940+0000 7fc6b1200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac108e90 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.940+0000 7fc6a9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.940+0000 7fc6a9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59496/0 (socket says 172.21.15.131:59496) 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.940+0000 7fc6a9e00700 1 -- 172.21.15.131:0/3313591845 learned_addr learned my addr 172.21.15.131:0/3313591845 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.941+0000 7fc6a9e00700 1 -- 172.21.15.131:0/3313591845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac19f530 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.941+0000 7fc6a9e00700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac19eff0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0116e0 tx=0x7fc69c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.941+0000 7fc6a2a00700 1 -- 172.21.15.131:0/3313591845 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c003a60 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.941+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac19f7c0 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.941+0000 7fc6a2a00700 1 -- 172.21.15.131:0/3313591845 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc69c01b8f0 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.941+0000 7fc6a2a00700 1 -- 172.21.15.131:0/3313591845 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c019860 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.941+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac1a24b0 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.943+0000 7fc6a2a00700 1 -- 172.21.15.131:0/3313591845 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc69c019b10 con 0x7fc6ac1074c0 2024-04-26T19:59:19.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.943+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac064a80 con 0x7fc6ac1074c0 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.943+0000 7fc6a2a00700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc690039610 0x7fc69003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.943+0000 7fc6a2a00700 1 -- 172.21.15.131:0/3313591845 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc69c04df70 con 0x7fc6ac1074c0 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.944+0000 7fc6a9400700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc690039610 0x7fc69003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.944+0000 7fc6a9400700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc690039610 0x7fc69003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc698006fd0 tx=0x7fc698006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:17.949+0000 7fc6a2a00700 1 -- 172.21.15.131:0/3313591845 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc69c01f030 con 0x7fc6ac1074c0 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.188+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc6ac197000 con 0x7fc690039610 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.422+0000 7fc6a2a00700 1 -- 172.21.15.131:0/3313591845 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc6ac197000 con 0x7fc690039610 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc690039610 msgr2=0x7fc69003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc690039610 0x7fc69003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc698006fd0 tx=0x7fc698006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 msgr2=0x7fc6ac19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac19eff0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0116e0 tx=0x7fc69c011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 shutdown_connections 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc690039610 0x7fc69003bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 --2- 172.21.15.131:0/3313591845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6ac1074c0 0x7fc6ac19eff0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:19.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 >> 172.21.15.131:0/3313591845 conn(0x7fc6ac103120 msgr2=0x7fc6ac1958e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:19.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 shutdown_connections 2024-04-26T19:59:19.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:18.425+0000 7fc6b1200700 1 -- 172.21.15.131:0/3313591845 wait complete. 2024-04-26T19:59:19.121 INFO:teuthology.orchestra.run.smithi131.stdout:Fetching dashboard port number... 2024-04-26T19:59:19.414 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:19.415 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:19 smithi131 ceph-mon[27919]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:59:19.415 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:20.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:20 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/253532327' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T19:59:20.907 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T19:59:20.907 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.844+0000 7f86cf800700 1 Processor -- start 2024-04-26T19:59:20.907 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.844+0000 7f86cf800700 1 -- start start 2024-04-26T19:59:20.907 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.844+0000 7f86cf800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c807d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:20.907 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.844+0000 7f86cf800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f86c807f010 con 0x7f86c807d2e0 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.845+0000 7f86cce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c807d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.845+0000 7f86cce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c807d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59508/0 (socket says 172.21.15.131:59508) 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.845+0000 7f86cce00700 1 -- 172.21.15.131:0/1115141192 learned_addr learned my addr 172.21.15.131:0/1115141192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.846+0000 7f86cce00700 1 -- 172.21.15.131:0/1115141192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c807dc40 con 0x7f86c807d2e0 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.846+0000 7f86cce00700 1 --2- 172.21.15.131:0/1115141192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c807d700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0098a0 tx=0x7f86bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c353249201c2a10c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.846+0000 7f86c7400700 1 -- 172.21.15.131:0/1115141192 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86bc004160 con 0x7f86c807d2e0 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.846+0000 7f86c7400700 1 -- 172.21.15.131:0/1115141192 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86bc004300 con 0x7f86c807d2e0 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.846+0000 7f86c7400700 1 -- 172.21.15.131:0/1115141192 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86bc011a20 con 0x7f86c807d2e0 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.847+0000 7f86cf800700 1 -- 172.21.15.131:0/1115141192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 msgr2=0x7f86c807d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:20.908 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.847+0000 7f86cf800700 1 --2- 172.21.15.131:0/1115141192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c807d700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0098a0 tx=0x7f86bc009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.847+0000 7f86cf800700 1 -- 172.21.15.131:0/1115141192 shutdown_connections 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.847+0000 7f86cf800700 1 --2- 172.21.15.131:0/1115141192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c807d700 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.847+0000 7f86cf800700 1 -- 172.21.15.131:0/1115141192 >> 172.21.15.131:0/1115141192 conn(0x7f86c8104200 msgr2=0x7f86c8106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.847+0000 7f86cf800700 1 -- 172.21.15.131:0/1115141192 shutdown_connections 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.847+0000 7f86cf800700 1 -- 172.21.15.131:0/1115141192 wait complete. 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.848+0000 7f86cf800700 1 Processor -- start 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.848+0000 7f86cf800700 1 -- start start 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.849+0000 7f86cf800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.849+0000 7f86cf800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f86c810f2e0 con 0x7f86c807d2e0 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.849+0000 7f86cce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.849+0000 7f86cce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59512/0 (socket says 172.21.15.131:59512) 2024-04-26T19:59:20.909 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.849+0000 7f86cce00700 1 -- 172.21.15.131:0/253532327 learned_addr learned my addr 172.21.15.131:0/253532327 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.849+0000 7f86cce00700 1 -- 172.21.15.131:0/253532327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c819f5f0 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.850+0000 7f86cce00700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c819f0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.850+0000 7f86c5600700 1 -- 172.21.15.131:0/253532327 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86bc003aa0 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.850+0000 7f86c5600700 1 -- 172.21.15.131:0/253532327 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86bc01b8f0 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.850+0000 7f86c5600700 1 -- 172.21.15.131:0/253532327 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86bc023870 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.850+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c819f880 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.851+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c81a2570 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.851+0000 7f86c5600700 1 -- 172.21.15.131:0/253532327 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f86bc023b20 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.852+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c8064a80 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.852+0000 7f86c5600700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86b0039660 0x7f86b003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.852+0000 7f86c5600700 1 -- 172.21.15.131:0/253532327 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f86bc04f9a0 con 0x7f86c807d2e0 2024-04-26T19:59:20.910 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.852+0000 7f86c7e00700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86b0039660 0x7f86b003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.853+0000 7f86c7e00700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86b0039660 0x7f86b003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86b8006fd0 tx=0x7f86b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:19.858+0000 7f86c5600700 1 -- 172.21.15.131:0/253532327 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f86bc057020 con 0x7f86c807d2e0 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.089+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f86c802e490 con 0x7f86c807d2e0 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.089+0000 7f86c5600700 1 -- 172.21.15.131:0/253532327 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f86bc01f030 con 0x7f86c807d2e0 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86b0039660 msgr2=0x7f86b003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86b0039660 0x7f86b003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86b8006fd0 tx=0x7f86b8006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 msgr2=0x7f86c819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c819f0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 shutdown_connections 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86b0039660 0x7f86b003bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 --2- 172.21.15.131:0/253532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86c807d2e0 0x7f86c819f0b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.092+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 >> 172.21.15.131:0/253532327 conn(0x7f86c8104200 msgr2=0x7f86c8104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.093+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 shutdown_connections 2024-04-26T19:59:20.911 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:20.093+0000 7f86cf800700 1 -- 172.21.15.131:0/253532327 wait complete. 2024-04-26T19:59:20.939 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T19:59:20.939 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-04-26T19:59:20.962 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T19:59:20.962 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-04-26T19:59:20.962 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-04-26T19:59:20.962 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T19:59:20.965 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph Dashboard is now available at: 2024-04-26T19:59:20.966 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:20.966 INFO:teuthology.orchestra.run.smithi131.stdout: URL: https://smithi131.front.sepia.ceph.com:8443/ 2024-04-26T19:59:20.966 INFO:teuthology.orchestra.run.smithi131.stdout: User: admin 2024-04-26T19:59:20.966 INFO:teuthology.orchestra.run.smithi131.stdout: Password: xnbanrcn73 2024-04-26T19:59:20.966 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:20.966 INFO:teuthology.orchestra.run.smithi131.stdout:Saving cluster configuration to /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config directory 2024-04-26T19:59:20.967 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling autotune for osd_memory_target 2024-04-26T19:59:21.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:21 smithi131 ceph-mon[27919]: mgrmap e13: x(active, since 6s) 2024-04-26T19:59:22.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.994+0000 7fc75bc00700 1 Processor -- start 2024-04-26T19:59:22.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.994+0000 7fc75bc00700 1 -- start start 2024-04-26T19:59:22.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.994+0000 7fc75bc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:22.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.994+0000 7fc75bc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7540834b0 con 0x7fc75410fcf0 2024-04-26T19:59:22.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.995+0000 7fc759200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:22.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.995+0000 7fc759200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59514/0 (socket says 172.21.15.131:59514) 2024-04-26T19:59:22.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.995+0000 7fc759200700 1 -- 172.21.15.131:0/598194128 learned_addr learned my addr 172.21.15.131:0/598194128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:22.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.995+0000 7fc759200700 1 -- 172.21.15.131:0/598194128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc754112620 con 0x7fc75410fcf0 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.996+0000 7fc759200700 1 --2- 172.21.15.131:0/598194128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541120e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7480098a0 tx=0x7fc748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1eda1152821a67b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.996+0000 7fc753400700 1 -- 172.21.15.131:0/598194128 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc748004160 con 0x7fc75410fcf0 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.996+0000 7fc753400700 1 -- 172.21.15.131:0/598194128 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc748004300 con 0x7fc75410fcf0 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.996+0000 7fc753400700 1 -- 172.21.15.131:0/598194128 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc748011a20 con 0x7fc75410fcf0 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.997+0000 7fc75bc00700 1 -- 172.21.15.131:0/598194128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 msgr2=0x7fc7541120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.997+0000 7fc75bc00700 1 --2- 172.21.15.131:0/598194128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541120e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7480098a0 tx=0x7fc748009bb0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.997+0000 7fc75bc00700 1 -- 172.21.15.131:0/598194128 shutdown_connections 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.997+0000 7fc75bc00700 1 --2- 172.21.15.131:0/598194128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541120e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.997+0000 7fc75bc00700 1 -- 172.21.15.131:0/598194128 >> 172.21.15.131:0/598194128 conn(0x7fc75407e210 msgr2=0x7fc754080650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.997+0000 7fc75bc00700 1 -- 172.21.15.131:0/598194128 shutdown_connections 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.997+0000 7fc75bc00700 1 -- 172.21.15.131:0/598194128 wait complete. 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.998+0000 7fc75bc00700 1 Processor -- start 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.998+0000 7fc75bc00700 1 -- start start 2024-04-26T19:59:22.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.998+0000 7fc75bc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.998+0000 7fc75bc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc754113620 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.999+0000 7fc759200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.999+0000 7fc759200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59528/0 (socket says 172.21.15.131:59528) 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.999+0000 7fc759200700 1 -- 172.21.15.131:0/2470254599 learned_addr learned my addr 172.21.15.131:0/2470254599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.999+0000 7fc759200700 1 -- 172.21.15.131:0/2470254599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7541a3960 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:21.999+0000 7fc759200700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541a3420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc748004660 tx=0x7fc748011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.000+0000 7fc751600700 1 -- 172.21.15.131:0/2470254599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc748003aa0 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.000+0000 7fc751600700 1 -- 172.21.15.131:0/2470254599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc74801b8f0 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.000+0000 7fc751600700 1 -- 172.21.15.131:0/2470254599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc748023900 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.000+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7541a3bf0 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.000+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7541a68e0 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.001+0000 7fc751600700 1 -- 172.21.15.131:0/2470254599 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fc74801baa0 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.001+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc754064a80 con 0x7fc75410fcf0 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.001+0000 7fc751600700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc73c039700 0x7fc73c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:22.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.001+0000 7fc751600700 1 -- 172.21.15.131:0/2470254599 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc74804e010 con 0x7fc75410fcf0 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.002+0000 7fc753e00700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc73c039700 0x7fc73c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.002+0000 7fc753e00700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc73c039700 0x7fc73c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.007+0000 7fc751600700 1 -- 172.21.15.131:0/2470254599 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc74801f030 con 0x7fc75410fcf0 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.233+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fc75402e490 con 0x7fc75410fcf0 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.234+0000 7fc751600700 1 -- 172.21.15.131:0/2470254599 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fc748018350 con 0x7fc75410fcf0 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc73c039700 msgr2=0x7fc73c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc73c039700 0x7fc73c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 msgr2=0x7fc7541a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541a3420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc748004660 tx=0x7fc748011aa0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 shutdown_connections 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc73c039700 0x7fc73c03bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 --2- 172.21.15.131:0/2470254599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc75410fcf0 0x7fc7541a3420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.879 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.238+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 >> 172.21.15.131:0/2470254599 conn(0x7fc75407e210 msgr2=0x7fc75407edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:22.879 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.239+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 shutdown_connections 2024-04-26T19:59:22.879 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:22.239+0000 7fc75bc00700 1 -- 172.21.15.131:0/2470254599 wait complete. 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.466+0000 7fd6fc400700 1 Processor -- start 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.467+0000 7fd6fc400700 1 -- start start 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.467+0000 7fd6fc400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f410b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.467+0000 7fd6fc400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410b7a0 con 0x7fd6f410ae40 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.467+0000 7fd6f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f410b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.468+0000 7fd6f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f410b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59532/0 (socket says 172.21.15.131:59532) 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.468+0000 7fd6f9a00700 1 -- 172.21.15.131:0/3802494327 learned_addr learned my addr 172.21.15.131:0/3802494327 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.468+0000 7fd6f9a00700 1 -- 172.21.15.131:0/3802494327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f410bfa0 con 0x7fd6f410ae40 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.468+0000 7fd6f9a00700 1 --2- 172.21.15.131:0/3802494327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f410b260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e80 tx=0x7fd6e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f6d9e7d8df25b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6f3e00700 1 -- 172.21.15.131:0/3802494327 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e4017070 con 0x7fd6f410ae40 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6f3e00700 1 -- 172.21.15.131:0/3802494327 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e400b8c0 con 0x7fd6f410ae40 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6f3e00700 1 -- 172.21.15.131:0/3802494327 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e4004490 con 0x7fd6f410ae40 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6fc400700 1 -- 172.21.15.131:0/3802494327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 msgr2=0x7fd6f410b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6fc400700 1 --2- 172.21.15.131:0/3802494327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f410b260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e80 tx=0x7fd6e400b330 comp rx=0 tx=0).stop 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6fc400700 1 -- 172.21.15.131:0/3802494327 shutdown_connections 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6fc400700 1 --2- 172.21.15.131:0/3802494327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f410b260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.469+0000 7fd6fc400700 1 -- 172.21.15.131:0/3802494327 >> 172.21.15.131:0/3802494327 conn(0x7fd6f407e230 msgr2=0x7fd6f407e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.470+0000 7fd6fc400700 1 -- 172.21.15.131:0/3802494327 shutdown_connections 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.470+0000 7fd6fc400700 1 -- 172.21.15.131:0/3802494327 wait complete. 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.471+0000 7fd6fc400700 1 Processor -- start 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.471+0000 7fd6fc400700 1 -- start start 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.471+0000 7fd6fc400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.471+0000 7fd6fc400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410f3e0 con 0x7fd6f410ae40 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.471+0000 7fd6f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.471+0000 7fd6f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59536/0 (socket says 172.21.15.131:59536) 2024-04-26T19:59:24.503 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.471+0000 7fd6f9a00700 1 -- 172.21.15.131:0/161490236 learned_addr learned my addr 172.21.15.131:0/161490236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:24.503 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.472+0000 7fd6f9a00700 1 -- 172.21.15.131:0/161490236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f419f310 con 0x7fd6f410ae40 2024-04-26T19:59:24.503 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.472+0000 7fd6f9a00700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f419edd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4015040 tx=0x7fd6e400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.503 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.472+0000 7fd6f2000700 1 -- 172.21.15.131:0/161490236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e4017040 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.472+0000 7fd6f2000700 1 -- 172.21.15.131:0/161490236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e4022070 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.472+0000 7fd6f2000700 1 -- 172.21.15.131:0/161490236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e4012430 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.472+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f419f5a0 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.472+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f41a2290 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.473+0000 7fd6f2000700 1 -- 172.21.15.131:0/161490236 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd6e4012670 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.474+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f410f560 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.474+0000 7fd6f2000700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd6d80396a0 0x7fd6d803bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.474+0000 7fd6f2000700 1 -- 172.21.15.131:0/161490236 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd6e400f050 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.474+0000 7fd6f9000700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd6d80396a0 0x7fd6d803bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.475+0000 7fd6f9000700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd6d80396a0 0x7fd6d803bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.480+0000 7fd6f2000700 1 -- 172.21.15.131:0/161490236 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd6e401c9d0 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.824+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fd6f41a2840 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.830+0000 7fd6f2000700 1 -- 172.21.15.131:0/161490236 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fd6e402c200 con 0x7fd6f410ae40 2024-04-26T19:59:24.504 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd6d80396a0 msgr2=0x7fd6d803bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd6d80396a0 0x7fd6d803bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 msgr2=0x7fd6f419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f419edd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4015040 tx=0x7fd6e400bd50 comp rx=0 tx=0).stop 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 shutdown_connections 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd6d80396a0 0x7fd6d803bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 --2- 172.21.15.131:0/161490236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6f410ae40 0x7fd6f419edd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 >> 172.21.15.131:0/161490236 conn(0x7fd6f407e230 msgr2=0x7fd6f41084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 shutdown_connections 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-04-26T19:59:23.833+0000 7fd6fc400700 1 -- 172.21.15.131:0/161490236 wait complete. 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T19:59:24.505 INFO:teuthology.orchestra.run.smithi131.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: ceph telemetry on 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout:For more information see: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T19:59:24.507 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:24.507 INFO:teuthology.orchestra.run.smithi131.stdout:Bootstrap complete. 2024-04-26T19:59:24.589 INFO:tasks.cephadm:Fetching config... 2024-04-26T19:59:24.590 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:59:24.590 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T19:59:24.638 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T19:59:24.638 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:59:24.638 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T19:59:24.724 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T19:59:24.724 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:59:24.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T19:59:24.833 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T19:59:24.833 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T19:59:24.833 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T19:59:24.922 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T19:59:24.922 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq4IlVHkzONz4kZkrM5HLt/kLV3jQa+K/vUs40z+Q6VVntMQzS0rLv8pNscyoUVyaQMY/VW8rx31kA+r9k6vmmPhx4+qx4FNQG7MSH8qFde6NYPXhyzofsuz7IQhRhTI9PiOnsNqlkxzc7VDtgve7caP7I51+3bD+xZ6EUBQPf+Aa3ynanBv0zut3hM68I9/v7wXCAB2h4SyVBHrJKV+fIObGbWY/rPs20FTz3jiLr7Zo5sFDTMx639gY4IUSCqXPHHiyF/5ng287jTXJoVTVzuuy+utz1VG0Z7cJizGYoyzGl5OIQWxUwUvwdXrJy5SU/1rYybcRNxeemejoQ/52eaXwL+dor0BKJ1h3mPoSecihvQMnsytYRDFYuHtedlYY9RRzBbo9HFmFU8+kOWYCBzfHAmcEnJi/b1rkT01w09rQi6ZEI1F6qRlennuuxqRM+UfhW5iJQYOQD4bFx3N42KC+FI1czeWh4+bcnAdaVDvlYWk6/Mcnk5fs8T3fjMd0= ceph-2027e7ca-0407-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T19:59:25.065 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq4IlVHkzONz4kZkrM5HLt/kLV3jQa+K/vUs40z+Q6VVntMQzS0rLv8pNscyoUVyaQMY/VW8rx31kA+r9k6vmmPhx4+qx4FNQG7MSH8qFde6NYPXhyzofsuz7IQhRhTI9PiOnsNqlkxzc7VDtgve7caP7I51+3bD+xZ6EUBQPf+Aa3ynanBv0zut3hM68I9/v7wXCAB2h4SyVBHrJKV+fIObGbWY/rPs20FTz3jiLr7Zo5sFDTMx639gY4IUSCqXPHHiyF/5ng287jTXJoVTVzuuy+utz1VG0Z7cJizGYoyzGl5OIQWxUwUvwdXrJy5SU/1rYybcRNxeemejoQ/52eaXwL+dor0BKJ1h3mPoSecihvQMnsytYRDFYuHtedlYY9RRzBbo9HFmFU8+kOWYCBzfHAmcEnJi/b1rkT01w09rQi6ZEI1F6qRlennuuxqRM+UfhW5iJQYOQD4bFx3N42KC+FI1czeWh4+bcnAdaVDvlYWk6/Mcnk5fs8T3fjMd0= ceph-2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:59:25.095 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:24 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/161490236' entity='client.admin' 2024-04-26T19:59:25.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq4IlVHkzONz4kZkrM5HLt/kLV3jQa+K/vUs40z+Q6VVntMQzS0rLv8pNscyoUVyaQMY/VW8rx31kA+r9k6vmmPhx4+qx4FNQG7MSH8qFde6NYPXhyzofsuz7IQhRhTI9PiOnsNqlkxzc7VDtgve7caP7I51+3bD+xZ6EUBQPf+Aa3ynanBv0zut3hM68I9/v7wXCAB2h4SyVBHrJKV+fIObGbWY/rPs20FTz3jiLr7Zo5sFDTMx639gY4IUSCqXPHHiyF/5ng287jTXJoVTVzuuy+utz1VG0Z7cJizGYoyzGl5OIQWxUwUvwdXrJy5SU/1rYybcRNxeemejoQ/52eaXwL+dor0BKJ1h3mPoSecihvQMnsytYRDFYuHtedlYY9RRzBbo9HFmFU8+kOWYCBzfHAmcEnJi/b1rkT01w09rQi6ZEI1F6qRlennuuxqRM+UfhW5iJQYOQD4bFx3N42KC+FI1czeWh4+bcnAdaVDvlYWk6/Mcnk5fs8T3fjMd0= ceph-2027e7ca-0407-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T19:59:25.148 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq4IlVHkzONz4kZkrM5HLt/kLV3jQa+K/vUs40z+Q6VVntMQzS0rLv8pNscyoUVyaQMY/VW8rx31kA+r9k6vmmPhx4+qx4FNQG7MSH8qFde6NYPXhyzofsuz7IQhRhTI9PiOnsNqlkxzc7VDtgve7caP7I51+3bD+xZ6EUBQPf+Aa3ynanBv0zut3hM68I9/v7wXCAB2h4SyVBHrJKV+fIObGbWY/rPs20FTz3jiLr7Zo5sFDTMx639gY4IUSCqXPHHiyF/5ng287jTXJoVTVzuuy+utz1VG0Z7cJizGYoyzGl5OIQWxUwUvwdXrJy5SU/1rYybcRNxeemejoQ/52eaXwL+dor0BKJ1h3mPoSecihvQMnsytYRDFYuHtedlYY9RRzBbo9HFmFU8+kOWYCBzfHAmcEnJi/b1rkT01w09rQi6ZEI1F6qRlennuuxqRM+UfhW5iJQYOQD4bFx3N42KC+FI1czeWh4+bcnAdaVDvlYWk6/Mcnk5fs8T3fjMd0= ceph-2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:59:25.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq4IlVHkzONz4kZkrM5HLt/kLV3jQa+K/vUs40z+Q6VVntMQzS0rLv8pNscyoUVyaQMY/VW8rx31kA+r9k6vmmPhx4+qx4FNQG7MSH8qFde6NYPXhyzofsuz7IQhRhTI9PiOnsNqlkxzc7VDtgve7caP7I51+3bD+xZ6EUBQPf+Aa3ynanBv0zut3hM68I9/v7wXCAB2h4SyVBHrJKV+fIObGbWY/rPs20FTz3jiLr7Zo5sFDTMx639gY4IUSCqXPHHiyF/5ng287jTXJoVTVzuuy+utz1VG0Z7cJizGYoyzGl5OIQWxUwUvwdXrJy5SU/1rYybcRNxeemejoQ/52eaXwL+dor0BKJ1h3mPoSecihvQMnsytYRDFYuHtedlYY9RRzBbo9HFmFU8+kOWYCBzfHAmcEnJi/b1rkT01w09rQi6ZEI1F6qRlennuuxqRM+UfhW5iJQYOQD4bFx3N42KC+FI1czeWh4+bcnAdaVDvlYWk6/Mcnk5fs8T3fjMd0= ceph-2027e7ca-0407-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T19:59:25.199 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq4IlVHkzONz4kZkrM5HLt/kLV3jQa+K/vUs40z+Q6VVntMQzS0rLv8pNscyoUVyaQMY/VW8rx31kA+r9k6vmmPhx4+qx4FNQG7MSH8qFde6NYPXhyzofsuz7IQhRhTI9PiOnsNqlkxzc7VDtgve7caP7I51+3bD+xZ6EUBQPf+Aa3ynanBv0zut3hM68I9/v7wXCAB2h4SyVBHrJKV+fIObGbWY/rPs20FTz3jiLr7Zo5sFDTMx639gY4IUSCqXPHHiyF/5ng287jTXJoVTVzuuy+utz1VG0Z7cJizGYoyzGl5OIQWxUwUvwdXrJy5SU/1rYybcRNxeemejoQ/52eaXwL+dor0BKJ1h3mPoSecihvQMnsytYRDFYuHtedlYY9RRzBbo9HFmFU8+kOWYCBzfHAmcEnJi/b1rkT01w09rQi6ZEI1F6qRlennuuxqRM+UfhW5iJQYOQD4bFx3N42KC+FI1czeWh4+bcnAdaVDvlYWk6/Mcnk5fs8T3fjMd0= ceph-2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T19:59:25.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T19:59:25.628 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T19:59:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.856+0000 7fed31800700 1 -- 172.21.15.131:0/1093068702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 msgr2=0x7fed2c105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.856+0000 7fed31800700 1 --2- 172.21.15.131:0/1093068702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c105290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009910 tx=0x7fed1c009c20 comp rx=0 tx=0).stop 2024-04-26T19:59:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.856+0000 7fed31800700 1 -- 172.21.15.131:0/1093068702 shutdown_connections 2024-04-26T19:59:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.856+0000 7fed31800700 1 --2- 172.21.15.131:0/1093068702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c105290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.856+0000 7fed31800700 1 -- 172.21.15.131:0/1093068702 >> 172.21.15.131:0/1093068702 conn(0x7fed2c100480 msgr2=0x7fed2c1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.856+0000 7fed31800700 1 -- 172.21.15.131:0/1093068702 shutdown_connections 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.856+0000 7fed31800700 1 -- 172.21.15.131:0/1093068702 wait complete. 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.857+0000 7fed31800700 1 Processor -- start 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.857+0000 7fed31800700 1 -- start start 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.858+0000 7fed31800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.858+0000 7fed31800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10ad70 con 0x7fed2c104e70 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.858+0000 7fed2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.858+0000 7fed2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54156/0 (socket says 172.21.15.131:54156) 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.858+0000 7fed2be00700 1 -- 172.21.15.131:0/4247701515 learned_addr learned my addr 172.21.15.131:0/4247701515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.858+0000 7fed2be00700 1 -- 172.21.15.131:0/4247701515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c19aa40 con 0x7fed2c104e70 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.859+0000 7fed2be00700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c19a500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005950 tx=0x7fed1c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.859+0000 7fed28c00700 1 -- 172.21.15.131:0/4247701515 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1c026070 con 0x7fed2c104e70 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.859+0000 7fed28c00700 1 -- 172.21.15.131:0/4247701515 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c00bc00 con 0x7fed2c104e70 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.859+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c19acd0 con 0x7fed2c104e70 2024-04-26T19:59:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.859+0000 7fed28c00700 1 -- 172.21.15.131:0/4247701515 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1c003a10 con 0x7fed2c104e70 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.859+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c19d9c0 con 0x7fed2c104e70 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.860+0000 7fed28c00700 1 -- 172.21.15.131:0/4247701515 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fed1c029070 con 0x7fed2c104e70 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.860+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c10b4f0 con 0x7fed2c104e70 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.861+0000 7fed28c00700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed100396b0 0x7fed1003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.861+0000 7fed28c00700 1 -- 172.21.15.131:0/4247701515 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fed1c01e070 con 0x7fed2c104e70 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.861+0000 7fed2b400700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed100396b0 0x7fed1003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.862+0000 7fed2b400700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed100396b0 0x7fed1003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed18006fd0 tx=0x7fed18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:27.867+0000 7fed28c00700 1 -- 172.21.15.131:0/4247701515 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fed1c013070 con 0x7fed2c104e70 2024-04-26T19:59:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.123+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fed2c02e5a0 con 0x7fed2c104e70 2024-04-26T19:59:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.127+0000 7fed28c00700 1 -- 172.21.15.131:0/4247701515 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fed1c030020 con 0x7fed2c104e70 2024-04-26T19:59:28.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.130+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed100396b0 msgr2=0x7fed1003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:28.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.130+0000 7fed31800700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed100396b0 0x7fed1003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed18006fd0 tx=0x7fed18006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 msgr2=0x7fed2c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c19a500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005950 tx=0x7fed1c004fa0 comp rx=0 tx=0).stop 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 shutdown_connections 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed100396b0 0x7fed1003bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 --2- 172.21.15.131:0/4247701515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2c104e70 0x7fed2c19a500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 >> 172.21.15.131:0/4247701515 conn(0x7fed2c100480 msgr2=0x7fed2c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 shutdown_connections 2024-04-26T19:59:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:28.131+0000 7fed31800700 1 -- 172.21.15.131:0/4247701515 wait complete. 2024-04-26T19:59:28.828 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T19:59:28.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T19:59:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T19:59:29.502 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:29 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4247701515' entity='client.admin' 2024-04-26T19:59:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.827+0000 7fec13200700 1 -- 172.21.15.131:0/3510172445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 msgr2=0x7fec0c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.827+0000 7fec13200700 1 --2- 172.21.15.131:0/3510172445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c105d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7febfc009910 tx=0x7febfc009c20 comp rx=0 tx=0).stop 2024-04-26T19:59:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.827+0000 7fec13200700 1 -- 172.21.15.131:0/3510172445 shutdown_connections 2024-04-26T19:59:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.827+0000 7fec13200700 1 --2- 172.21.15.131:0/3510172445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c105d70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.827+0000 7fec13200700 1 -- 172.21.15.131:0/3510172445 >> 172.21.15.131:0/3510172445 conn(0x7fec0c0fd390 msgr2=0x7fec0c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.827+0000 7fec13200700 1 -- 172.21.15.131:0/3510172445 shutdown_connections 2024-04-26T19:59:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.827+0000 7fec13200700 1 -- 172.21.15.131:0/3510172445 wait complete. 2024-04-26T19:59:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.828+0000 7fec13200700 1 Processor -- start 2024-04-26T19:59:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.828+0000 7fec13200700 1 -- start start 2024-04-26T19:59:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.828+0000 7fec13200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.828+0000 7fec13200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c108d60 con 0x7fec0c103980 2024-04-26T19:59:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.829+0000 7fec0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.829+0000 7fec0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54166/0 (socket says 172.21.15.131:54166) 2024-04-26T19:59:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.829+0000 7fec0be00700 1 -- 172.21.15.131:0/2618877730 learned_addr learned my addr 172.21.15.131:0/2618877730 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.829+0000 7fec0be00700 1 -- 172.21.15.131:0/2618877730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c103740 con 0x7fec0c103980 2024-04-26T19:59:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.829+0000 7fec0be00700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c103200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7febfc005190 tx=0x7febfc004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.830+0000 7fec09600700 1 -- 172.21.15.131:0/2618877730 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febfc027070 con 0x7fec0c103980 2024-04-26T19:59:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.830+0000 7fec09600700 1 -- 172.21.15.131:0/2618877730 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc022470 con 0x7fec0c103980 2024-04-26T19:59:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.830+0000 7fec09600700 1 -- 172.21.15.131:0/2618877730 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febfc00f5f0 con 0x7fec0c103980 2024-04-26T19:59:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.830+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c101930 con 0x7fec0c103980 2024-04-26T19:59:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.830+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c101e70 con 0x7fec0c103980 2024-04-26T19:59:31.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.831+0000 7fec09600700 1 -- 172.21.15.131:0/2618877730 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7febfc02a070 con 0x7fec0c103980 2024-04-26T19:59:31.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.831+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0c1094e0 con 0x7fec0c103980 2024-04-26T19:59:31.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.831+0000 7fec09600700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7febf0039650 0x7febf003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:31.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.832+0000 7fec09600700 1 -- 172.21.15.131:0/2618877730 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7febfc01e030 con 0x7fec0c103980 2024-04-26T19:59:31.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.832+0000 7fec03e00700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7febf0039650 0x7febf003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.832+0000 7fec03e00700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7febf0039650 0x7febf003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febf4006fd0 tx=0x7febf4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:31.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:31.837+0000 7fec09600700 1 -- 172.21.15.131:0/2618877730 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7febfc013070 con 0x7fec0c103980 2024-04-26T19:59:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.067+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fec0c1918e0 con 0x7febf0039650 2024-04-26T19:59:32.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.074+0000 7fec09600700 1 -- 172.21.15.131:0/2618877730 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fec0c1918e0 con 0x7febf0039650 2024-04-26T19:59:32.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.077+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7febf0039650 msgr2=0x7febf003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:32.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.077+0000 7fec13200700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7febf0039650 0x7febf003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febf4006fd0 tx=0x7febf4006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:32.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.077+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 msgr2=0x7fec0c103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:32.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.077+0000 7fec13200700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c103200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7febfc005190 tx=0x7febfc004f80 comp rx=0 tx=0).stop 2024-04-26T19:59:32.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.078+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 shutdown_connections 2024-04-26T19:59:32.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.078+0000 7fec13200700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7febf0039650 0x7febf003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:32.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.078+0000 7fec13200700 1 --2- 172.21.15.131:0/2618877730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec0c103980 0x7fec0c103200 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:32.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.078+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 >> 172.21.15.131:0/2618877730 conn(0x7fec0c0fd390 msgr2=0x7fec0c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:32.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.078+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 shutdown_connections 2024-04-26T19:59:32.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:32.078+0000 7fec13200700 1 -- 172.21.15.131:0/2618877730 wait complete. 2024-04-26T19:59:32.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:32.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:32.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:59:32.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:32.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:32.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:32.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:32.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:32.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:59:32.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:59:32.898 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-04-26T19:59:32.899 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:59:32.899 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-04-26T19:59:32.917 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T19:59:32.917 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T19:59:32.974 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-04-26T19:59:32.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch host add smithi148 2024-04-26T19:59:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T19:59:33.609 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:33 smithi131 ceph-mon[27919]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:59:33.609 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:33 smithi131 ceph-mon[27919]: Updating smithi131:/etc/ceph/ceph.conf 2024-04-26T19:59:34.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:34 smithi131 ceph-mon[27919]: Updating smithi131:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T19:59:34.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:34 smithi131 ceph-mon[27919]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-04-26T19:59:34.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:34 smithi131 ceph-mon[27919]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:35.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.478+0000 7ff313000700 1 -- 172.21.15.131:0/2778204831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 msgr2=0x7ff30c075c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:35.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.478+0000 7ff313000700 1 --2- 172.21.15.131:0/2778204831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c075c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00b0a0 tx=0x7ff2fc00b3b0 comp rx=0 tx=0).stop 2024-04-26T19:59:35.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.478+0000 7ff313000700 1 -- 172.21.15.131:0/2778204831 shutdown_connections 2024-04-26T19:59:35.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.478+0000 7ff313000700 1 --2- 172.21.15.131:0/2778204831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c075c30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:35.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.478+0000 7ff313000700 1 -- 172.21.15.131:0/2778204831 >> 172.21.15.131:0/2778204831 conn(0x7ff30c0fe910 msgr2=0x7ff30c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:35.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.478+0000 7ff313000700 1 -- 172.21.15.131:0/2778204831 shutdown_connections 2024-04-26T19:59:35.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.478+0000 7ff313000700 1 -- 172.21.15.131:0/2778204831 wait complete. 2024-04-26T19:59:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.479+0000 7ff313000700 1 Processor -- start 2024-04-26T19:59:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.479+0000 7ff313000700 1 -- start start 2024-04-26T19:59:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.480+0000 7ff313000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.480+0000 7ff313000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10f560 con 0x7ff30c075850 2024-04-26T19:59:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.480+0000 7ff30be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.480+0000 7ff30be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43684/0 (socket says 172.21.15.131:43684) 2024-04-26T19:59:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.480+0000 7ff30be00700 1 -- 172.21.15.131:0/4118703786 learned_addr learned my addr 172.21.15.131:0/4118703786 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:35.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.480+0000 7ff30be00700 1 -- 172.21.15.131:0/4118703786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c1a0bf0 con 0x7ff30c075850 2024-04-26T19:59:35.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.481+0000 7ff30be00700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c1a06b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00d010 tx=0x7ff2fc009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:35.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.481+0000 7ff309600700 1 -- 172.21.15.131:0/4118703786 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2fc021020 con 0x7ff30c075850 2024-04-26T19:59:35.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.481+0000 7ff309600700 1 -- 172.21.15.131:0/4118703786 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2fc004000 con 0x7ff30c075850 2024-04-26T19:59:35.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.481+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c1a0e80 con 0x7ff30c075850 2024-04-26T19:59:35.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.481+0000 7ff309600700 1 -- 172.21.15.131:0/4118703786 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2fc017040 con 0x7ff30c075850 2024-04-26T19:59:35.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.481+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c19a870 con 0x7ff30c075850 2024-04-26T19:59:35.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.482+0000 7ff309600700 1 -- 172.21.15.131:0/4118703786 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff2fc024020 con 0x7ff30c075850 2024-04-26T19:59:35.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.482+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c076f00 con 0x7ff30c075850 2024-04-26T19:59:35.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.483+0000 7ff309600700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2f00396f0 0x7ff2f003bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:35.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.483+0000 7ff309600700 1 -- 172.21.15.131:0/4118703786 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff2fc00f050 con 0x7ff30c075850 2024-04-26T19:59:35.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.483+0000 7ff303400700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2f00396f0 0x7ff2f003bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:35.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.484+0000 7ff303400700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2f00396f0 0x7ff2f003bbb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4006fd0 tx=0x7ff2f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:35.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.488+0000 7ff309600700 1 -- 172.21.15.131:0/4118703786 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2fc02b020 con 0x7ff30c075850 2024-04-26T19:59:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:35.721+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) -- 0x7ff30c19bda0 con 0x7ff2f00396f0 2024-04-26T19:59:35.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:35 smithi131 ceph-mon[27919]: Updating smithi131:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T19:59:35.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:35.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:35.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:36.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:36 smithi131 ceph-mon[27919]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:37.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:37 smithi131 ceph-mon[27919]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:59:37.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:37 smithi131 ceph-mon[27919]: Deploying cephadm binary to smithi148 2024-04-26T19:59:38.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:38 smithi131 ceph-mon[27919]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:40.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.402+0000 7ff309600700 1 -- 172.21.15.131:0/4118703786 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff30c19bda0 con 0x7ff2f00396f0 2024-04-26T19:59:40.403 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-04-26T19:59:40.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2f00396f0 msgr2=0x7ff2f003bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:40.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2f00396f0 0x7ff2f003bbb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4006fd0 tx=0x7ff2f4006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:40.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 msgr2=0x7ff30c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:40.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c1a06b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00d010 tx=0x7ff2fc009790 comp rx=0 tx=0).stop 2024-04-26T19:59:40.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 shutdown_connections 2024-04-26T19:59:40.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2f00396f0 0x7ff2f003bbb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 --2- 172.21.15.131:0/4118703786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff30c075850 0x7ff30c1a06b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.405+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 >> 172.21.15.131:0/4118703786 conn(0x7ff30c0fe910 msgr2=0x7ff30c1070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:40.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.406+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 shutdown_connections 2024-04-26T19:59:40.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:40.406+0000 7ff313000700 1 -- 172.21.15.131:0/4118703786 wait complete. 2024-04-26T19:59:40.734 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:40 smithi131 ceph-mon[27919]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:41.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T19:59:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T19:59:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:41 smithi131 ceph-mon[27919]: Added host smithi148 2024-04-26T19:59:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:42.686 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:42 smithi131 ceph-mon[27919]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.458+0000 7f9b5f800700 1 -- 172.21.15.131:0/3479112142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 msgr2=0x7f9b58103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.458+0000 7f9b5f800700 1 --2- 172.21.15.131:0/3479112142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b58103450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009910 tx=0x7f9b4c009c20 comp rx=0 tx=0).stop 2024-04-26T19:59:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.459+0000 7f9b5f800700 1 -- 172.21.15.131:0/3479112142 shutdown_connections 2024-04-26T19:59:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.459+0000 7f9b5f800700 1 --2- 172.21.15.131:0/3479112142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b58103450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.459+0000 7f9b5f800700 1 -- 172.21.15.131:0/3479112142 >> 172.21.15.131:0/3479112142 conn(0x7f9b580fe940 msgr2=0x7f9b58100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.459+0000 7f9b5f800700 1 -- 172.21.15.131:0/3479112142 shutdown_connections 2024-04-26T19:59:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.459+0000 7f9b5f800700 1 -- 172.21.15.131:0/3479112142 wait complete. 2024-04-26T19:59:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.460+0000 7f9b5f800700 1 Processor -- start 2024-04-26T19:59:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.460+0000 7f9b5f800700 1 -- start start 2024-04-26T19:59:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.460+0000 7f9b5f800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b581a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.460+0000 7f9b5f800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5810d2f0 con 0x7f9b58103070 2024-04-26T19:59:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.461+0000 7f9b5ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b581a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.461+0000 7f9b5ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b581a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43704/0 (socket says 172.21.15.131:43704) 2024-04-26T19:59:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.461+0000 7f9b5ce00700 1 -- 172.21.15.131:0/4217055765 learned_addr learned my addr 172.21.15.131:0/4217055765 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.461+0000 7f9b5ce00700 1 -- 172.21.15.131:0/4217055765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b581a0c40 con 0x7f9b58103070 2024-04-26T19:59:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.461+0000 7f9b5ce00700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b581a0700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0056d0 tx=0x7f9b4c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.462+0000 7f9b55600700 1 -- 172.21.15.131:0/4217055765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c026070 con 0x7f9b58103070 2024-04-26T19:59:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.462+0000 7f9b55600700 1 -- 172.21.15.131:0/4217055765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4c00bba0 con 0x7f9b58103070 2024-04-26T19:59:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.462+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581a0f30 con 0x7f9b58103070 2024-04-26T19:59:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.462+0000 7f9b55600700 1 -- 172.21.15.131:0/4217055765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c003920 con 0x7f9b58103070 2024-04-26T19:59:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.462+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5819a8c0 con 0x7f9b58103070 2024-04-26T19:59:43.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.463+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b58104550 con 0x7f9b58103070 2024-04-26T19:59:43.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.463+0000 7f9b55600700 1 -- 172.21.15.131:0/4217055765 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f9b4c029070 con 0x7f9b58103070 2024-04-26T19:59:43.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.463+0000 7f9b55600700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b400396b0 0x7f9b4003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.463+0000 7f9b55600700 1 -- 172.21.15.131:0/4217055765 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9b4c01e070 con 0x7f9b58103070 2024-04-26T19:59:43.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.464+0000 7f9b57e00700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b400396b0 0x7f9b4003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.464+0000 7f9b57e00700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b400396b0 0x7f9b4003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006fd0 tx=0x7f9b44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.469+0000 7f9b55600700 1 -- 172.21.15.131:0/4217055765 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9b4c013070 con 0x7f9b58103070 2024-04-26T19:59:43.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.702+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b5819bdf0 con 0x7f9b400396b0 2024-04-26T19:59:43.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.704+0000 7f9b55600700 1 -- 172.21.15.131:0/4217055765 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f9b5819bdf0 con 0x7f9b400396b0 2024-04-26T19:59:43.705 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:43.705 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b400396b0 msgr2=0x7f9b4003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b400396b0 0x7f9b4003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006fd0 tx=0x7f9b44006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 msgr2=0x7f9b581a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b581a0700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0056d0 tx=0x7f9b4c0057b0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 shutdown_connections 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b400396b0 0x7f9b4003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 --2- 172.21.15.131:0/4217055765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b58103070 0x7f9b581a0700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 >> 172.21.15.131:0/4217055765 conn(0x7f9b580fe940 msgr2=0x7f9b58075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:43.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 shutdown_connections 2024-04-26T19:59:43.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:43.707+0000 7f9b5f800700 1 -- 172.21.15.131:0/4217055765 wait complete. 2024-04-26T19:59:44.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:44.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:44 smithi131 ceph-mon[27919]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:44.461 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-04-26T19:59:44.462 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:59:44.462 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-04-26T19:59:44.480 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T19:59:44.480 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T19:59:44.537 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-04-26T19:59:44.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch host add smithi157 2024-04-26T19:59:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T19:59:45.471 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:45 smithi131 ceph-mon[27919]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T19:59:46.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:46 smithi131 ceph-mon[27919]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.120+0000 7fd441200700 1 -- 172.21.15.131:0/3762331251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 msgr2=0x7fd43c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.120+0000 7fd441200700 1 --2- 172.21.15.131:0/3762331251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c103280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009910 tx=0x7fd42c009c20 comp rx=0 tx=0).stop 2024-04-26T19:59:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.120+0000 7fd441200700 1 -- 172.21.15.131:0/3762331251 shutdown_connections 2024-04-26T19:59:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.120+0000 7fd441200700 1 --2- 172.21.15.131:0/3762331251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c103280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.120+0000 7fd441200700 1 -- 172.21.15.131:0/3762331251 >> 172.21.15.131:0/3762331251 conn(0x7fd43c0fe770 msgr2=0x7fd43c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.120+0000 7fd441200700 1 -- 172.21.15.131:0/3762331251 shutdown_connections 2024-04-26T19:59:47.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.120+0000 7fd441200700 1 -- 172.21.15.131:0/3762331251 wait complete. 2024-04-26T19:59:47.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.121+0000 7fd441200700 1 Processor -- start 2024-04-26T19:59:47.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.121+0000 7fd441200700 1 -- start start 2024-04-26T19:59:47.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.122+0000 7fd441200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.122+0000 7fd441200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c10d120 con 0x7fd43c102ea0 2024-04-26T19:59:47.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.122+0000 7fd43b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.122+0000 7fd43b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56616/0 (socket says 172.21.15.131:56616) 2024-04-26T19:59:47.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.122+0000 7fd43b400700 1 -- 172.21.15.131:0/2070820666 learned_addr learned my addr 172.21.15.131:0/2070820666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:47.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.123+0000 7fd43b400700 1 -- 172.21.15.131:0/2070820666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c077610 con 0x7fd43c102ea0 2024-04-26T19:59:47.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.123+0000 7fd43b400700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c075720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0056d0 tx=0x7fd42c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:47.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.123+0000 7fd433e00700 1 -- 172.21.15.131:0/2070820666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c026070 con 0x7fd43c102ea0 2024-04-26T19:59:47.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.123+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c075cc0 con 0x7fd43c102ea0 2024-04-26T19:59:47.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.123+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c076180 con 0x7fd43c102ea0 2024-04-26T19:59:47.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.124+0000 7fd433e00700 1 -- 172.21.15.131:0/2070820666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42c00b6f0 con 0x7fd43c102ea0 2024-04-26T19:59:47.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.124+0000 7fd433e00700 1 -- 172.21.15.131:0/2070820666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c030060 con 0x7fd43c102ea0 2024-04-26T19:59:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.125+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c005330 con 0x7fd43c102ea0 2024-04-26T19:59:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.125+0000 7fd433e00700 1 -- 172.21.15.131:0/2070820666 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd42c029070 con 0x7fd43c102ea0 2024-04-26T19:59:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.126+0000 7fd433e00700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd420039630 0x7fd42003baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.126+0000 7fd433e00700 1 -- 172.21.15.131:0/2070820666 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd42c01e070 con 0x7fd43c102ea0 2024-04-26T19:59:47.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.126+0000 7fd43aa00700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd420039630 0x7fd42003baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.127+0000 7fd43aa00700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd420039630 0x7fd42003baf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd428006fd0 tx=0x7fd428006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:47.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.133+0000 7fd433e00700 1 -- 172.21.15.131:0/2070820666 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd42c013070 con 0x7fd43c102ea0 2024-04-26T19:59:47.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:47.366+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) -- 0x7fd41c000bc0 con 0x7fd420039630 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:59:47.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:47 smithi131 ceph-mon[27919]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: Updating smithi148:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: Updating smithi148:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:48 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:49.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:49 smithi131 ceph-mon[27919]: Deploying cephadm binary to smithi157 2024-04-26T19:59:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:50 smithi131 ceph-mon[27919]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.001+0000 7fd433e00700 1 -- 172.21.15.131:0/2070820666 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd41c000bc0 con 0x7fd420039630 2024-04-26T19:59:52.003 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.004+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd420039630 msgr2=0x7fd42003baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.004+0000 7fd441200700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd420039630 0x7fd42003baf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd428006fd0 tx=0x7fd428006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.004+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 msgr2=0x7fd43c075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.004+0000 7fd441200700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c075720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0056d0 tx=0x7fd42c0057b0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.005+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 shutdown_connections 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.005+0000 7fd441200700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd420039630 0x7fd42003baf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.005+0000 7fd441200700 1 --2- 172.21.15.131:0/2070820666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd43c102ea0 0x7fd43c075720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.005+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 >> 172.21.15.131:0/2070820666 conn(0x7fd43c0fe770 msgr2=0x7fd43c10aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.005+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 shutdown_connections 2024-04-26T19:59:52.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:52.005+0000 7fd441200700 1 -- 172.21.15.131:0/2070820666 wait complete. 2024-04-26T19:59:52.688 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:52 smithi131 ceph-mon[27919]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:52.688 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:52.688 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:52.741 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T19:59:53.185 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T19:59:53.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:53 smithi131 ceph-mon[27919]: Added host smithi157 2024-04-26T19:59:53.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:53.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:54.783 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:54 smithi131 ceph-mon[27919]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.106+0000 7f80b1e00700 1 -- 172.21.15.131:0/4153907071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 msgr2=0x7f80ac075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.106+0000 7f80b1e00700 1 --2- 172.21.15.131:0/4153907071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac075b20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f809c009910 tx=0x7f809c009c20 comp rx=0 tx=0).stop 2024-04-26T19:59:55.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.106+0000 7f80b1e00700 1 -- 172.21.15.131:0/4153907071 shutdown_connections 2024-04-26T19:59:55.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.106+0000 7f80b1e00700 1 --2- 172.21.15.131:0/4153907071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac075b20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.106+0000 7f80b1e00700 1 -- 172.21.15.131:0/4153907071 >> 172.21.15.131:0/4153907071 conn(0x7f80ac0fe940 msgr2=0x7f80ac100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:55.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.106+0000 7f80b1e00700 1 -- 172.21.15.131:0/4153907071 shutdown_connections 2024-04-26T19:59:55.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.106+0000 7f80b1e00700 1 -- 172.21.15.131:0/4153907071 wait complete. 2024-04-26T19:59:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.107+0000 7f80b1e00700 1 Processor -- start 2024-04-26T19:59:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.107+0000 7f80b1e00700 1 -- start start 2024-04-26T19:59:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.108+0000 7f80b1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.108+0000 7f80b1e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac10f550 con 0x7f80ac075740 2024-04-26T19:59:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.108+0000 7f80aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.108+0000 7f80aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39308/0 (socket says 172.21.15.131:39308) 2024-04-26T19:59:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.108+0000 7f80aaa00700 1 -- 172.21.15.131:0/3052449518 learned_addr learned my addr 172.21.15.131:0/3052449518 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.108+0000 7f80aaa00700 1 -- 172.21.15.131:0/3052449518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac1a0c70 con 0x7f80ac075740 2024-04-26T19:59:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.109+0000 7f80aaa00700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac1a0730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f809c006010 tx=0x7f809c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.109+0000 7f80b0a00700 1 -- 172.21.15.131:0/3052449518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f809c026070 con 0x7f80ac075740 2024-04-26T19:59:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.109+0000 7f80b0a00700 1 -- 172.21.15.131:0/3052449518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809c00bc00 con 0x7f80ac075740 2024-04-26T19:59:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.109+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ac1a0f00 con 0x7f80ac075740 2024-04-26T19:59:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.109+0000 7f80b0a00700 1 -- 172.21.15.131:0/3052449518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f809c003910 con 0x7f80ac075740 2024-04-26T19:59:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.109+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ac19a8f0 con 0x7f80ac075740 2024-04-26T19:59:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.110+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ac076e80 con 0x7f80ac075740 2024-04-26T19:59:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.110+0000 7f80b0a00700 1 -- 172.21.15.131:0/3052449518 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f809c029070 con 0x7f80ac075740 2024-04-26T19:59:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.111+0000 7f80b0a00700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8090039650 0x7f809003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.111+0000 7f80b0a00700 1 -- 172.21.15.131:0/3052449518 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f809c01e070 con 0x7f80ac075740 2024-04-26T19:59:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.111+0000 7f80aa000700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8090039650 0x7f809003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.112+0000 7f80aa000700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8090039650 0x7f809003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:55.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.116+0000 7f80b0a00700 1 -- 172.21.15.131:0/3052449518 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f809c013070 con 0x7f80ac075740 2024-04-26T19:59:55.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.348+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80ac19be20 con 0x7f8090039650 2024-04-26T19:59:55.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.350+0000 7f80b0a00700 1 -- 172.21.15.131:0/3052449518 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f80ac19be20 con 0x7f8090039650 2024-04-26T19:59:55.351 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T19:59:55.351 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8090039650 msgr2=0x7f809003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8090039650 0x7f809003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 msgr2=0x7f80ac1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac1a0730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f809c006010 tx=0x7f809c005780 comp rx=0 tx=0).stop 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 shutdown_connections 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8090039650 0x7f809003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 --2- 172.21.15.131:0/3052449518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80ac075740 0x7f80ac1a0730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.353+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 >> 172.21.15.131:0/3052449518 conn(0x7f80ac0fe940 msgr2=0x7f80ac107130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.354+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 shutdown_connections 2024-04-26T19:59:55.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:55.354+0000 7f80b1e00700 1 -- 172.21.15.131:0/3052449518 wait complete. 2024-04-26T19:59:56.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:56.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:55 smithi131 ceph-mon[27919]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T19:59:56.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:55 smithi131 ceph-mon[27919]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:56.099 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T19:59:56.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T19:59:56.534 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T19:59:58.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.521+0000 7f7da3600700 1 -- 172.21.15.131:0/1478642448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 msgr2=0x7f7d9c105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.521+0000 7f7da3600700 1 --2- 172.21.15.131:0/1478642448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c105650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009910 tx=0x7f7d90009c20 comp rx=0 tx=0).stop 2024-04-26T19:59:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.521+0000 7f7da3600700 1 -- 172.21.15.131:0/1478642448 shutdown_connections 2024-04-26T19:59:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.521+0000 7f7da3600700 1 --2- 172.21.15.131:0/1478642448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c105650 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.521+0000 7f7da3600700 1 -- 172.21.15.131:0/1478642448 >> 172.21.15.131:0/1478642448 conn(0x7f7d9c100b40 msgr2=0x7f7d9c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.521+0000 7f7da3600700 1 -- 172.21.15.131:0/1478642448 shutdown_connections 2024-04-26T19:59:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.522+0000 7f7da3600700 1 -- 172.21.15.131:0/1478642448 wait complete. 2024-04-26T19:59:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.522+0000 7f7da3600700 1 Processor -- start 2024-04-26T19:59:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.523+0000 7f7da3600700 1 -- start start 2024-04-26T19:59:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.523+0000 7f7da3600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.523+0000 7f7da3600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10f4f0 con 0x7f7d9c105270 2024-04-26T19:59:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.523+0000 7f7da0c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.523+0000 7f7da0c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39328/0 (socket says 172.21.15.131:39328) 2024-04-26T19:59:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.523+0000 7f7da0c00700 1 -- 172.21.15.131:0/1093978589 learned_addr learned my addr 172.21.15.131:0/1093978589 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T19:59:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.524+0000 7f7da0c00700 1 -- 172.21.15.131:0/1093978589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c1a0c30 con 0x7f7d9c105270 2024-04-26T19:59:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.524+0000 7f7da0c00700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c1a06f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006010 tx=0x7f7d90005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.524+0000 7f7d99600700 1 -- 172.21.15.131:0/1093978589 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90026070 con 0x7f7d9c105270 2024-04-26T19:59:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.524+0000 7f7d99600700 1 -- 172.21.15.131:0/1093978589 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9000bba0 con 0x7f7d9c105270 2024-04-26T19:59:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.524+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c1a0f20 con 0x7f7d9c105270 2024-04-26T19:59:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.524+0000 7f7d99600700 1 -- 172.21.15.131:0/1093978589 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90003910 con 0x7f7d9c105270 2024-04-26T19:59:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.525+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c19a8b0 con 0x7f7d9c105270 2024-04-26T19:59:58.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.526+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c106750 con 0x7f7d9c105270 2024-04-26T19:59:58.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.526+0000 7f7d99600700 1 -- 172.21.15.131:0/1093978589 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f7d90029070 con 0x7f7d9c105270 2024-04-26T19:59:58.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.526+0000 7f7d99600700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7d840396b0 0x7f7d8403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.527+0000 7f7d99600700 1 -- 172.21.15.131:0/1093978589 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7d9001e070 con 0x7f7d9c105270 2024-04-26T19:59:58.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.527+0000 7f7d9be00700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7d840396b0 0x7f7d8403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.528+0000 7f7d9be00700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7d840396b0 0x7f7d8403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7d88006fd0 tx=0x7f7d88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.532+0000 7f7d99600700 1 -- 172.21.15.131:0/1093978589 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7d90013070 con 0x7f7d9c105270 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:59:58.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:59:58.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:58.766+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7d9c19bde0 con 0x7f7d9c105270 2024-04-26T19:59:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.526+0000 7f7d99600700 1 -- 172.21.15.131:0/1093978589 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7d9000fe20 con 0x7f7d9c105270 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.530+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7d840396b0 msgr2=0x7f7d8403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.530+0000 7f7da3600700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7d840396b0 0x7f7d8403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7d88006fd0 tx=0x7f7d88006e40 comp rx=0 tx=0).stop 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.530+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 msgr2=0x7f7d9c1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.530+0000 7f7da3600700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c1a06f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006010 tx=0x7f7d90005780 comp rx=0 tx=0).stop 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.531+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 shutdown_connections 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.531+0000 7f7da3600700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7d840396b0 0x7f7d8403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.531+0000 7f7da3600700 1 --2- 172.21.15.131:0/1093978589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d9c105270 0x7f7d9c1a06f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.531+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 >> 172.21.15.131:0/1093978589 conn(0x7f7d9c100b40 msgr2=0x7f7d9c106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.531+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 shutdown_connections 2024-04-26T19:59:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T19:59:59.531+0000 7f7da3600700 1 -- 172.21.15.131:0/1093978589 wait complete. 2024-04-26T19:59:59.533 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-04-26T19:59:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:59 smithi131 ceph-mon[27919]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-26T19:59:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:59 smithi131 ceph-mon[27919]: Updating smithi157:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T19:59:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:59 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1093978589' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T19:59:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T19:59:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 19:59:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:00.239 INFO:tasks.cephadm:Adding mon.a on smithi131 2024-04-26T20:00:00.239 INFO:tasks.cephadm:Adding mon.b on smithi148 2024-04-26T20:00:00.239 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-04-26T20:00:00.239 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi131:172.21.15.131=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c' 2024-04-26T20:00:00.482 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:01.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:00 smithi131 ceph-mon[27919]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-26T20:00:01.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:00 smithi131 ceph-mon[27919]: Updating smithi157:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T20:00:01.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:00 smithi131 ceph-mon[27919]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:01.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:00 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1093978589' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T20:00:01.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:00 smithi131 ceph-mon[27919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T20:00:01.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 -- 172.21.15.157:0/4105189410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 msgr2=0x7f1508103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 --2- 172.21.15.157:0/4105189410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508103450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009910 tx=0x7f14f8009c20 comp rx=0 tx=0).stop 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 -- 172.21.15.157:0/4105189410 shutdown_connections 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 --2- 172.21.15.157:0/4105189410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508103450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 -- 172.21.15.157:0/4105189410 >> 172.21.15.157:0/4105189410 conn(0x7f15080fe940 msgr2=0x7f1508100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 -- 172.21.15.157:0/4105189410 shutdown_connections 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 -- 172.21.15.157:0/4105189410 wait complete. 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 Processor -- start 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.839+0000 7f1510688700 1 -- start start 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.840+0000 7f1510688700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.840+0000 7f1510688700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f150810d2f0 con 0x7f1508103070 2024-04-26T20:00:01.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.840+0000 7f150e424700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.840+0000 7f150e424700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:41324/0 (socket says 172.21.15.157:41324) 2024-04-26T20:00:01.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.840+0000 7f150e424700 1 -- 172.21.15.157:0/1080248303 learned_addr learned my addr 172.21.15.157:0/1080248303 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:00:01.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.841+0000 7f150e424700 1 -- 172.21.15.157:0/1080248303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1508077610 con 0x7f1508103070 2024-04-26T20:00:01.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.841+0000 7f150e424700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508075720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f14f80056d0 tx=0x7f14f80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.842+0000 7f14ff7fe700 1 -- 172.21.15.157:0/1080248303 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f8026070 con 0x7f1508103070 2024-04-26T20:00:01.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.842+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1508075cc0 con 0x7f1508103070 2024-04-26T20:00:01.843 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.842+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1508076180 con 0x7f1508103070 2024-04-26T20:00:01.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.842+0000 7f14ff7fe700 1 -- 172.21.15.157:0/1080248303 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f800bba0 con 0x7f1508103070 2024-04-26T20:00:01.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.842+0000 7f14ff7fe700 1 -- 172.21.15.157:0/1080248303 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f8030060 con 0x7f1508103070 2024-04-26T20:00:01.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.843+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec005330 con 0x7f1508103070 2024-04-26T20:00:01.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.843+0000 7f14ff7fe700 1 -- 172.21.15.157:0/1080248303 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f14f8029070 con 0x7f1508103070 2024-04-26T20:00:01.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.843+0000 7f14ff7fe700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14f40396b0 0x7f14f403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.843+0000 7f14ff7fe700 1 -- 172.21.15.157:0/1080248303 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f14f801e070 con 0x7f1508103070 2024-04-26T20:00:01.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.844+0000 7f150dc23700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14f40396b0 0x7f14f403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.845+0000 7f150dc23700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14f40396b0 0x7f14f403bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1504006fd0 tx=0x7f1504006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:01.858+0000 7f14ff7fe700 1 -- 172.21.15.157:0/1080248303 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f14f8013070 con 0x7f1508103070 2024-04-26T20:00:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:01 smithi131 ceph-mon[27919]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:02.072 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.071+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) -- 0x7f14ec000c80 con 0x7f14f40396b0 2024-04-26T20:00:02.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.086+0000 7f14ff7fe700 1 -- 172.21.15.157:0/1080248303 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f14ec000c80 con 0x7f14f40396b0 2024-04-26T20:00:02.087 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-04-26T20:00:02.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14f40396b0 msgr2=0x7f14f403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14f40396b0 0x7f14f403bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1504006fd0 tx=0x7f1504006e40 comp rx=0 tx=0).stop 2024-04-26T20:00:02.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 msgr2=0x7f1508075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508075720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f14f80056d0 tx=0x7f14f80057b0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 shutdown_connections 2024-04-26T20:00:02.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14f40396b0 0x7f14f403bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 --2- 172.21.15.157:0/1080248303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1508103070 0x7f1508075720 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 >> 172.21.15.157:0/1080248303 conn(0x7f15080fe940 msgr2=0x7f1508104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:02.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 shutdown_connections 2024-04-26T20:00:02.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:02.088+0000 7f1510688700 1 -- 172.21.15.157:0/1080248303 wait complete. 2024-04-26T20:00:02.763 DEBUG:teuthology.orchestra.run.smithi148:mon.b> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T20:00:02.766 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T20:00:02.768 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T20:00:02.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T20:00:03.484 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:03.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c;count:3 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:03.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:03 smithi131 ceph-mon[27919]: Deploying daemon mon.c on smithi157 2024-04-26T20:00:04.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:04 smithi131 ceph-mon[27919]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.476+0000 7f300e358700 1 -- 172.21.15.157:0/1304618620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 msgr2=0x7f3008102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.476+0000 7f300e358700 1 --2- 172.21.15.157:0/1304618620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f3008102340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009910 tx=0x7f2ff8009c20 comp rx=0 tx=0).stop 2024-04-26T20:00:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.476+0000 7f300e358700 1 -- 172.21.15.157:0/1304618620 shutdown_connections 2024-04-26T20:00:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.476+0000 7f300e358700 1 --2- 172.21.15.157:0/1304618620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f3008102340 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.476+0000 7f300e358700 1 -- 172.21.15.157:0/1304618620 >> 172.21.15.157:0/1304618620 conn(0x7f30080789f0 msgr2=0x7f30080ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.476+0000 7f300e358700 1 -- 172.21.15.157:0/1304618620 shutdown_connections 2024-04-26T20:00:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.476+0000 7f300e358700 1 -- 172.21.15.157:0/1304618620 wait complete. 2024-04-26T20:00:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.477+0000 7f300e358700 1 Processor -- start 2024-04-26T20:00:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.477+0000 7f300e358700 1 -- start start 2024-04-26T20:00:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.478+0000 7f300e358700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f30081996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.478+0000 7f300e358700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f300810c1e0 con 0x7f3008101f60 2024-04-26T20:00:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.478+0000 7f3007fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f30081996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.478+0000 7f3007fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f30081996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:41336/0 (socket says 172.21.15.157:41336) 2024-04-26T20:00:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.479+0000 7f3007fff700 1 -- 172.21.15.157:0/2562886383 learned_addr learned my addr 172.21.15.157:0/2562886383 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:00:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.479+0000 7f3007fff700 1 -- 172.21.15.157:0/2562886383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3008199be0 con 0x7f3008101f60 2024-04-26T20:00:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.479+0000 7f3007fff700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f30081996a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80056d0 tx=0x7f2ff80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.480+0000 7f30057fa700 1 -- 172.21.15.157:0/2562886383 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8026070 con 0x7f3008101f60 2024-04-26T20:00:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.480+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3008199ed0 con 0x7f3008101f60 2024-04-26T20:00:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.480+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300819a390 con 0x7f3008101f60 2024-04-26T20:00:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.480+0000 7f30057fa700 1 -- 172.21.15.157:0/2562886383 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff800bba0 con 0x7f3008101f60 2024-04-26T20:00:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.480+0000 7f30057fa700 1 -- 172.21.15.157:0/2562886383 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff80037f0 con 0x7f3008101f60 2024-04-26T20:00:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.481+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff4005330 con 0x7f3008101f60 2024-04-26T20:00:06.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.481+0000 7f30057fa700 1 -- 172.21.15.157:0/2562886383 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2ff8029070 con 0x7f3008101f60 2024-04-26T20:00:06.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.482+0000 7f30057fa700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ff0039650 0x7f2ff003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:06.482 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.482+0000 7f30057fa700 1 -- 172.21.15.157:0/2562886383 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2ff801e070 con 0x7f3008101f60 2024-04-26T20:00:06.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.483+0000 7f30077fe700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ff0039650 0x7f2ff003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.484+0000 7f30077fe700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ff0039650 0x7f2ff003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:06.498 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.497+0000 7f30057fa700 1 -- 172.21.15.157:0/2562886383 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ff8013070 con 0x7f3008101f60 2024-04-26T20:00:06.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.786+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2ff4005620 con 0x7f3008101f60 2024-04-26T20:00:06.788 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.787+0000 7f30057fa700 1 -- 172.21.15.157:0/2562886383 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f2ff80282c0 con 0x7f3008101f60 2024-04-26T20:00:06.789 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T20:00:06.789 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","modified":"2024-04-26T19:58:01.203692Z","created":"2024-04-26T19:58:01.203692Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ff0039650 msgr2=0x7f2ff003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ff0039650 0x7f2ff003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).stop 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 msgr2=0x7f30081996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f30081996a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80056d0 tx=0x7f2ff80057b0 comp rx=0 tx=0).stop 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 shutdown_connections 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ff0039650 0x7f2ff003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 --2- 172.21.15.157:0/2562886383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3008101f60 0x7f30081996a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 >> 172.21.15.157:0/2562886383 conn(0x7f30080789f0 msgr2=0x7f3008103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 shutdown_connections 2024-04-26T20:00:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:06.790+0000 7f300e358700 1 -- 172.21.15.157:0/2562886383 wait complete. 2024-04-26T20:00:06.792 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-26T20:00:07.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:06 smithi131 ceph-mon[27919]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:08.747 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T20:00:08.748 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T20:00:08.994 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:00:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:00:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: mon.a calling monitor election 2024-04-26T20:00:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: mon.c calling monitor election 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: monmap epoch 2 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: last_changed 2024-04-26T20:00:07.007870+0000 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: min_mon_release 19 (squid) 2024-04-26T20:00:12.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: election_strategy: 1 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: fsmap 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: mgrmap e13: x(active, since 58s) 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: overall HEALTH_OK 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:12.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.456+0000 7feaa29e2700 1 -- 172.21.15.157:0/2554588338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea840035c0 msgr2=0x7fea84005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.456+0000 7feaa29e2700 1 --2- 172.21.15.157:0/2554588338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea840035c0 0x7fea84005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea9c06c890 tx=0x7fea8c00ded0 comp rx=0 tx=0).stop 2024-04-26T20:00:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.456+0000 7feaa29e2700 1 -- 172.21.15.157:0/2554588338 shutdown_connections 2024-04-26T20:00:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.456+0000 7feaa29e2700 1 --2- 172.21.15.157:0/2554588338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea840035c0 0x7fea84005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.456+0000 7feaa29e2700 1 -- 172.21.15.157:0/2554588338 >> 172.21.15.157:0/2554588338 conn(0x7fea9c078490 msgr2=0x7fea9c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.456+0000 7feaa29e2700 1 -- 172.21.15.157:0/2554588338 shutdown_connections 2024-04-26T20:00:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.456+0000 7feaa29e2700 1 -- 172.21.15.157:0/2554588338 wait complete. 2024-04-26T20:00:13.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:13.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.457+0000 7feaa29e2700 1 Processor -- start 2024-04-26T20:00:13.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.457+0000 7feaa29e2700 1 -- start start 2024-04-26T20:00:13.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7feaa29e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 0x7fea9c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7feaa29e2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9c1a0d80 0x7fea9c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7feaa29e2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c108ca0 con 0x7fea9c1a0d80 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7feaa29e2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c108b20 con 0x7fea840035c0 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 0x7fea9c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 0x7fea9c1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55340/0 (socket says 172.21.15.157:55340) 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9bfff700 1 -- 172.21.15.157:0/4173309083 learned_addr learned my addr 172.21.15.157:0/4173309083 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9bfff700 1 -- 172.21.15.157:0/4173309083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 msgr2=0x7fea9c1a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9bfff700 1 -- 172.21.15.157:0/4173309083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 msgr2=0x7fea9c1a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9bfff700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 0x7fea9c1a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9bfff700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 0x7fea9c1a0840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:00:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.458+0000 7fea9b7fe700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9c1a0d80 0x7fea9c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.459+0000 7fea9b7fe700 1 -- 172.21.15.157:0/4173309083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 msgr2=0x7fea9c1a0840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:00:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.459+0000 7fea9b7fe700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 0x7fea9c1a0840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.459+0000 7fea9b7fe700 1 -- 172.21.15.157:0/4173309083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1a13c0 con 0x7fea9c1a0d80 2024-04-26T20:00:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.459+0000 7fea9b7fe700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9c1a0d80 0x7fea9c19aa00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea9000b4d0 tx=0x7fea9000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.460+0000 7fea997fa700 1 -- 172.21.15.157:0/4173309083 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fea90011630 con 0x7fea9c1a0d80 2024-04-26T20:00:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.460+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c19af40 con 0x7fea9c1a0d80 2024-04-26T20:00:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.460+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c19b4d0 con 0x7fea9c1a0d80 2024-04-26T20:00:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.460+0000 7fea997fa700 1 -- 172.21.15.157:0/4173309083 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea90011c80 con 0x7fea9c1a0d80 2024-04-26T20:00:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.460+0000 7fea997fa700 1 -- 172.21.15.157:0/4173309083 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fea9001adc0 con 0x7fea9c1a0d80 2024-04-26T20:00:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.461+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea84002160 con 0x7fea9c1a0d80 2024-04-26T20:00:13.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.461+0000 7fea997fa700 1 -- 172.21.15.157:0/4173309083 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fea90018020 con 0x7fea9c1a0d80 2024-04-26T20:00:13.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.462+0000 7fea997fa700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fea880393f0 0x7fea8803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:13.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.462+0000 7fea997fa700 1 -- 172.21.15.157:0/4173309083 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fea9004c1f0 con 0x7fea9c1a0d80 2024-04-26T20:00:13.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.462+0000 7fea9bfff700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fea880393f0 0x7fea8803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:13.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.463+0000 7fea9bfff700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fea880393f0 0x7fea8803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005d10 tx=0x7fea8c005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:13.480 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.479+0000 7fea997fa700 1 -- 172.21.15.157:0/4173309083 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fea9001f020 con 0x7fea9c1a0d80 2024-04-26T20:00:13.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:13.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:13.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:13.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:13 smithi131 ceph-mon[27919]: Deploying daemon mon.b on smithi148 2024-04-26T20:00:13.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:13.762 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.761+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fea84002c50 con 0x7fea9c1a0d80 2024-04-26T20:00:13.763 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.762+0000 7fea997fa700 1 -- 172.21.15.157:0/4173309083 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fea90010e80 con 0x7fea9c1a0d80 2024-04-26T20:00:13.763 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T20:00:13.763 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","modified":"2024-04-26T20:00:07.007870Z","created":"2024-04-26T19:58:01.203692Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.764+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fea880393f0 msgr2=0x7fea8803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.764+0000 7feaa29e2700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fea880393f0 0x7fea8803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005d10 tx=0x7fea8c005bc0 comp rx=0 tx=0).stop 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.764+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9c1a0d80 msgr2=0x7fea9c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.764+0000 7feaa29e2700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9c1a0d80 0x7fea9c19aa00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea9000b4d0 tx=0x7fea9000b890 comp rx=0 tx=0).stop 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.765+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 shutdown_connections 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.765+0000 7feaa29e2700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fea880393f0 0x7fea8803b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.765+0000 7feaa29e2700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea840035c0 0x7fea9c1a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.765+0000 7feaa29e2700 1 --2- 172.21.15.157:0/4173309083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9c1a0d80 0x7fea9c19aa00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.765+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 >> 172.21.15.157:0/4173309083 conn(0x7fea9c078490 msgr2=0x7fea9c10aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.765+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 shutdown_connections 2024-04-26T20:00:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:13.765+0000 7feaa29e2700 1 -- 172.21.15.157:0/4173309083 wait complete. 2024-04-26T20:00:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-04-26T20:00:14.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:14 smithi157 ceph-mon[32703]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:14.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:14 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/4173309083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T20:00:14.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:14 smithi131 ceph-mon[27919]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:14.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:14 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/4173309083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T20:00:14.507 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:00:14 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:00:14.007+0000 7f63cb800700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T20:00:15.332 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T20:00:15.332 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T20:00:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:00:21.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.404+0000 7f2af60ec700 1 -- 172.21.15.157:0/1273878805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2adc0035c0 msgr2=0x7f2adc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.404+0000 7f2af60ec700 1 --2- 172.21.15.157:0/1273878805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2adc0035c0 0x7f2adc005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2af006b710 tx=0x7f2ae000fe60 comp rx=0 tx=0).stop 2024-04-26T20:00:21.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.404+0000 7f2af60ec700 1 -- 172.21.15.157:0/1273878805 shutdown_connections 2024-04-26T20:00:21.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.404+0000 7f2af60ec700 1 --2- 172.21.15.157:0/1273878805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2adc0035c0 0x7f2adc005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.404+0000 7f2af60ec700 1 -- 172.21.15.157:0/1273878805 >> 172.21.15.157:0/1273878805 conn(0x7f2af00fd910 msgr2=0x7f2af00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:21.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.404+0000 7f2af60ec700 1 -- 172.21.15.157:0/1273878805 shutdown_connections 2024-04-26T20:00:21.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.404+0000 7f2af60ec700 1 -- 172.21.15.157:0/1273878805 wait complete. 2024-04-26T20:00:21.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.405+0000 7f2af60ec700 1 Processor -- start 2024-04-26T20:00:21.407 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.405+0000 7f2af60ec700 1 -- start start 2024-04-26T20:00:21.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2af60ec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 0x7f2af0101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2af60ec700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 0x7f2af0102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2af60ec700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2af01a4d80 0x7f2af01a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2af60ec700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2af010ae20 con 0x7f2af01a4d80 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2af60ec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2af010aca0 con 0x7f2adc0035c0 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2af60ec700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2af010afa0 con 0x7f2af0102280 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2ae6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 0x7f2af0102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2ae6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 0x7f2af0102700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52178/0 (socket says 172.21.15.157:52178) 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2ae6ffd700 1 -- 172.21.15.157:0/667812435 learned_addr learned my addr 172.21.15.157:0/667812435 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2aef7fe700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 0x7f2af0101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2ae6ffd700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 msgr2=0x7f2af0102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2ae6ffd700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 msgr2=0x7f2af0102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T20:00:21.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2ae6ffd700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 0x7f2af0102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2ae6ffd700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 0x7f2af0102700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2aef7fe700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 msgr2=0x7f2af0101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2aef7fe700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 msgr2=0x7f2af0101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2aef7fe700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 0x7f2af0101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2aef7fe700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 0x7f2af0101d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.406+0000 7f2aeffff700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2af01a4d80 0x7f2af01a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.407+0000 7f2aeffff700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 msgr2=0x7f2af0102700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.407+0000 7f2aeffff700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 0x7f2af0102700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.407+0000 7f2aeffff700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 msgr2=0x7f2af0101d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.407+0000 7f2aeffff700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 0x7f2af0101d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.407+0000 7f2aeffff700 1 -- 172.21.15.157:0/667812435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af01a7840 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.407+0000 7f2aeffff700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2af01a4d80 0x7f2af01a7170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800bd50 tx=0x7f2ae800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.407+0000 7f2aed7fa700 1 -- 172.21.15.157:0/667812435 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8019070 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.408+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af01a7b30 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.408+0000 7f2aed7fa700 1 -- 172.21.15.157:0/667812435 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae800cd10 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.408+0000 7f2aed7fa700 1 -- 172.21.15.157:0/667812435 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8012700 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.408+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af01a8070 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.409+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af0064a80 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.409+0000 7f2aed7fa700 1 -- 172.21.15.157:0/667812435 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2ae801c070 con 0x7f2af01a4d80 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.409+0000 7f2aed7fa700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ad0039ae0 0x7f2ad003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.409+0000 7f2aed7fa700 1 -- 172.21.15.157:0/667812435 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2ae804c0d0 con 0x7f2af01a4d80 2024-04-26T20:00:21.414 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.410+0000 7f2aef7fe700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ad0039ae0 0x7f2ad003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.414 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.411+0000 7f2aef7fe700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ad0039ae0 0x7f2ad003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000fa10 tx=0x7f2ae0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.425+0000 7f2aed7fa700 1 -- 172.21.15.157:0/667812435 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ae8014440 con 0x7f2af01a4d80 2024-04-26T20:00:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: mon.c calling monitor election 2024-04-26T20:00:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: mon.a calling monitor election 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: monmap epoch 3 2024-04-26T20:00:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: last_changed 2024-04-26T20:00:16.149435+0000 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: min_mon_release 19 (squid) 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: election_strategy: 1 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: fsmap 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: mgrmap e13: x(active, since 67s) 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: overall HEALTH_OK 2024-04-26T20:00:21.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:21.483 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:21.483 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: mon.c calling monitor election 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: mon.a calling monitor election 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: monmap epoch 3 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: last_changed 2024-04-26T20:00:16.149435+0000 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: min_mon_release 19 (squid) 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: election_strategy: 1 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-26T20:00:21.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-26T20:00:21.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: fsmap 2024-04-26T20:00:21.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T20:00:21.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: mgrmap e13: x(active, since 67s) 2024-04-26T20:00:21.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: overall HEALTH_OK 2024-04-26T20:00:21.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:21.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:21.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:21.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.721+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2af01a83d0 con 0x7f2af01a4d80 2024-04-26T20:00:21.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.722+0000 7f2aed7fa700 1 -- 172.21.15.157:0/667812435 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f2ae80128a0 con 0x7f2af01a4d80 2024-04-26T20:00:21.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-26T20:00:21.723 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","modified":"2024-04-26T20:00:16.149435Z","created":"2024-04-26T19:58:01.203692Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T20:00:21.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ad0039ae0 msgr2=0x7f2ad003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ad0039ae0 0x7f2ad003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000fa10 tx=0x7f2ae0005a90 comp rx=0 tx=0).stop 2024-04-26T20:00:21.725 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2af01a4d80 msgr2=0x7f2af01a7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2af01a4d80 0x7f2af01a7170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800bd50 tx=0x7f2ae800be30 comp rx=0 tx=0).stop 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 shutdown_connections 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2af01a4d80 0x7f2af01a7170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2ad0039ae0 0x7f2ad003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2adc0035c0 0x7f2af0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 --2- 172.21.15.157:0/667812435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2af0102280 0x7f2af0102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 >> 172.21.15.157:0/667812435 conn(0x7f2af00fd910 msgr2=0x7f2af01079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 shutdown_connections 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:21.724+0000 7f2af60ec700 1 -- 172.21.15.157:0/667812435 wait complete. 2024-04-26T20:00:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-04-26T20:00:22.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:22.367 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:22 smithi157 ceph-mon[32703]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:22.367 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:22.367 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:22 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/667812435' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T20:00:22.367 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:22.368 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T20:00:22.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T20:00:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:22 smithi131 ceph-mon[27919]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:22 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/667812435' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T20:00:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:22.797 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:00:23.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:23.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: mon.b calling monitor election 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: mon.b calling monitor election 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: mon.c calling monitor election 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: mon.a calling monitor election 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: monmap epoch 3 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: last_changed 2024-04-26T20:00:16.149435+0000 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: min_mon_release 19 (squid) 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: election_strategy: 1 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: fsmap 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T20:00:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: mgrmap e13: x(active, since 69s) 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: overall HEALTH_OK 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: Updating smithi131:/etc/ceph/ceph.conf 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:24.002 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: mon.b calling monitor election 2024-04-26T20:00:24.002 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: mon.b calling monitor election 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: mon.c calling monitor election 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: mon.a calling monitor election 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: monmap epoch 3 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: last_changed 2024-04-26T20:00:16.149435+0000 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: min_mon_release 19 (squid) 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: election_strategy: 1 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: fsmap 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T20:00:24.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: mgrmap e13: x(active, since 69s) 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: overall HEALTH_OK 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: Updating smithi131:/etc/ceph/ceph.conf 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:24.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: mon.b calling monitor election 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: mon.b calling monitor election 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: mon.c calling monitor election 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: mon.a calling monitor election 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: monmap epoch 3 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: last_changed 2024-04-26T20:00:16.149435+0000 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: created 2024-04-26T19:58:01.203692+0000 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: min_mon_release 19 (squid) 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: election_strategy: 1 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: 0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-26T20:00:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: fsmap 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: mgrmap e13: x(active, since 69s) 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: overall HEALTH_OK 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: Updating smithi131:/etc/ceph/ceph.conf 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: Updating smithi148:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: Updating smithi157:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: Updating smithi131:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.049+0000 7f557b600700 1 -- 172.21.15.131:0/3722193371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 msgr2=0x7f5574075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.049+0000 7f557b600700 1 --2- 172.21.15.131:0/3722193371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f5574075ba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5568009910 tx=0x7f5568009c20 comp rx=0 tx=0).stop 2024-04-26T20:00:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.049+0000 7f557b600700 1 -- 172.21.15.131:0/3722193371 shutdown_connections 2024-04-26T20:00:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.049+0000 7f557b600700 1 --2- 172.21.15.131:0/3722193371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f5574075ba0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.049+0000 7f557b600700 1 -- 172.21.15.131:0/3722193371 >> 172.21.15.131:0/3722193371 conn(0x7f55740fe910 msgr2=0x7f5574100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.049+0000 7f557b600700 1 -- 172.21.15.131:0/3722193371 shutdown_connections 2024-04-26T20:00:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.049+0000 7f557b600700 1 -- 172.21.15.131:0/3722193371 wait complete. 2024-04-26T20:00:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.051+0000 7f557b600700 1 Processor -- start 2024-04-26T20:00:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.051+0000 7f557b600700 1 -- start start 2024-04-26T20:00:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f557b600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f557419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f557b600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557419f620 0x7f55741a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f557b600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55741a3ef0 0x7f55741a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f557b600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5574113950 con 0x7f55740757c0 2024-04-26T20:00:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f557b600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55741137d0 con 0x7f55741a3ef0 2024-04-26T20:00:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f557b600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5574113ad0 con 0x7f557419f620 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f5578c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f557419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f5578c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f557419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34484/0 (socket says 172.21.15.131:34484) 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.052+0000 7f5578c00700 1 -- 172.21.15.131:0/3888273896 learned_addr learned my addr 172.21.15.131:0/3888273896 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5579600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55741a3ef0 0x7f55741a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5573e00700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557419f620 0x7f55741a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5578c00700 1 -- 172.21.15.131:0/3888273896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557419f620 msgr2=0x7f55741a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5578c00700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557419f620 0x7f55741a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5578c00700 1 -- 172.21.15.131:0/3888273896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55741a3ef0 msgr2=0x7f55741a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5578c00700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55741a3ef0 0x7f55741a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5578c00700 1 -- 172.21.15.131:0/3888273896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55741abc60 con 0x7f55740757c0 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5573e00700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557419f620 0x7f55741a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5579600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55741a3ef0 0x7f55741a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.053+0000 7f5578c00700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f557419f0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5568005b40 tx=0x7f5568004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.054+0000 7f5571600700 1 -- 172.21.15.131:0/3888273896 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568027070 con 0x7f55740757c0 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.054+0000 7f5571600700 1 -- 172.21.15.131:0/3888273896 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556800be10 con 0x7f55740757c0 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.054+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741abe30 con 0x7f55740757c0 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.054+0000 7f5571600700 1 -- 172.21.15.131:0/3888273896 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568022e60 con 0x7f55740757c0 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.055+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55741ac230 con 0x7f55740757c0 2024-04-26T20:00:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.055+0000 7f5571600700 1 -- 172.21.15.131:0/3888273896 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f556802a070 con 0x7f55740757c0 2024-04-26T20:00:25.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.056+0000 7f5571600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f555c039ae0 0x7f555c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.056+0000 7f5571600700 1 -- 172.21.15.131:0/3888273896 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5568051a30 con 0x7f55740757c0 2024-04-26T20:00:25.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.056+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5574064a80 con 0x7f55740757c0 2024-04-26T20:00:25.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.057+0000 7f5573e00700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f555c039ae0 0x7f555c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.058+0000 7f5573e00700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f555c039ae0 0x7f555c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55600079b0 tx=0x7f5560008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.063+0000 7f5571600700 1 -- 172.21.15.131:0/3888273896 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5568013070 con 0x7f55740757c0 2024-04-26T20:00:25.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: Updating smithi148:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: Updating smithi157:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: Updating smithi131:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:25.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.302+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f55741ac460 con 0x7f55740757c0 2024-04-26T20:00:25.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.303+0000 7f5571600700 1 -- 172.21.15.131:0/3888273896 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f55680292c0 con 0x7f55740757c0 2024-04-26T20:00:25.304 INFO:teuthology.orchestra.run.smithi131.stdout:# minimal ceph.conf for 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:25.304 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-04-26T20:00:25.305 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:00:25.305 INFO:teuthology.orchestra.run.smithi131.stdout: mon_host = [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-04-26T20:00:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f555c039ae0 msgr2=0x7f555c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f555c039ae0 0x7f555c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55600079b0 tx=0x7f5560008040 comp rx=0 tx=0).stop 2024-04-26T20:00:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 msgr2=0x7f557419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f557419f0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5568005b40 tx=0x7f5568004f80 comp rx=0 tx=0).stop 2024-04-26T20:00:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 shutdown_connections 2024-04-26T20:00:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f555c039ae0 0x7f555c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55741a3ef0 0x7f55741a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55740757c0 0x7f557419f0e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 --2- 172.21.15.131:0/3888273896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557419f620 0x7f55741a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.307+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 >> 172.21.15.131:0/3888273896 conn(0x7f55740fe910 msgr2=0x7f55740fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.308+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 shutdown_connections 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:25.308+0000 7f557b600700 1 -- 172.21.15.131:0/3888273896 wait complete. 2024-04-26T20:00:25.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: Updating smithi148:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: Updating smithi157:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: Updating smithi131:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:25.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.042 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T20:00:26.042 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:00:26.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T20:00:26.109 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:00:26.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T20:00:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T20:00:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: Reconfiguring daemon mon.a on smithi131 2024-04-26T20:00:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3888273896' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:25 smithi131 ceph-mon[27919]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:26.219 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T20:00:26.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T20:00:26.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T20:00:26.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:26.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:26.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:26.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: Reconfiguring daemon mon.a on smithi131 2024-04-26T20:00:26.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3888273896' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:26.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:25 smithi157 ceph-mon[32703]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:26.246 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.246 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.246 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.246 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:26.246 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T20:00:26.247 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:26.247 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:26.247 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:26.247 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: Reconfiguring daemon mon.a on smithi131 2024-04-26T20:00:26.247 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3888273896' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:26.247 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:25 smithi148 ceph-mon[32281]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:26.252 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T20:00:26.252 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T20:00:26.318 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T20:00:26.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T20:00:26.345 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T20:00:26.345 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T20:00:26.409 INFO:tasks.cephadm:Adding mgr.x on smithi131 2024-04-26T20:00:26.409 INFO:tasks.cephadm:Adding mgr.y on smithi148 2024-04-26T20:00:26.409 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-04-26T20:00:26.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi131=x;smithi148=y;smithi157=z' 2024-04-26T20:00:26.678 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:00:28.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.164+0000 7f9a540e2700 1 -- 172.21.15.157:0/37985168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a4c103ab0 msgr2=0x7f9a4c105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.164+0000 7f9a540e2700 1 --2- 172.21.15.157:0/37985168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c105ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00b0a0 tx=0x7f9a3c00b3b0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.164+0000 7f9a540e2700 1 -- 172.21.15.157:0/37985168 shutdown_connections 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.164+0000 7f9a540e2700 1 --2- 172.21.15.157:0/37985168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c105ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.164+0000 7f9a540e2700 1 -- 172.21.15.157:0/37985168 >> 172.21.15.157:0/37985168 conn(0x7f9a4c0fd460 msgr2=0x7f9a4c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 -- 172.21.15.157:0/37985168 shutdown_connections 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 -- 172.21.15.157:0/37985168 wait complete. 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 Processor -- start 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 -- start start 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a4c19ade0 0x7f9a4c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c19b260 0x7f9a4c1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c108e90 con 0x7f9a4c19ade0 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c108d10 con 0x7f9a4c19b260 2024-04-26T20:00:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a540e2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c109010 con 0x7f9a4c103ab0 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.165+0000 7f9a51e7e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52184/0 (socket says 172.21.15.157:52184) 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 -- 172.21.15.157:0/1275738061 learned_addr learned my addr 172.21.15.157:0/1275738061 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a5267f700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c19b260 0x7f9a4c1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 -- 172.21.15.157:0/1275738061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c19b260 msgr2=0x7f9a4c1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c19b260 0x7f9a4c1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 -- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a4c19ade0 msgr2=0x7f9a4c19fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a4c19ade0 0x7f9a4c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 -- 172.21.15.157:0/1275738061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c1a2cb0 con 0x7f9a4c103ab0 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a51e7e700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c19a8a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00b830 tx=0x7f9a3c00bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a4affd700 1 -- 172.21.15.157:0/1275738061 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c017040 con 0x7f9a4c103ab0 2024-04-26T20:00:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c1a2f40 con 0x7f9a4c103ab0 2024-04-26T20:00:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a4affd700 1 -- 172.21.15.157:0/1275738061 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3c005780 con 0x7f9a4c103ab0 2024-04-26T20:00:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a4affd700 1 -- 172.21.15.157:0/1275738061 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c012a10 con 0x7f9a4c103ab0 2024-04-26T20:00:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.166+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1a3470 con 0x7f9a4c103ab0 2024-04-26T20:00:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.167+0000 7f9a4affd700 1 -- 172.21.15.157:0/1275738061 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f9a3c012bb0 con 0x7f9a4c103ab0 2024-04-26T20:00:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.167+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c064a80 con 0x7f9a4c103ab0 2024-04-26T20:00:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.167+0000 7f9a4affd700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9a34039ae0 0x7f9a3403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.167+0000 7f9a4affd700 1 -- 172.21.15.157:0/1275738061 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9a3c00f020 con 0x7f9a4c103ab0 2024-04-26T20:00:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.168+0000 7f9a5167d700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9a34039ae0 0x7f9a3403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.169+0000 7f9a5167d700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9a34039ae0 0x7f9a3403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a400098e0 tx=0x7f9a40006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:28.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.173+0000 7f9a4affd700 1 -- 172.21.15.157:0/1275738061 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9a3c029020 con 0x7f9a4c103ab0 2024-04-26T20:00:28.382 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.381+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}) -- 0x7f9a4c0638f0 con 0x7f9a34039ae0 2024-04-26T20:00:28.395 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.395+0000 7f9a4affd700 1 -- 172.21.15.157:0/1275738061 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9a4c0638f0 con 0x7f9a34039ae0 2024-04-26T20:00:28.396 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9a34039ae0 msgr2=0x7f9a3403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9a34039ae0 0x7f9a3403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a400098e0 tx=0x7f9a40006dd0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4c103ab0 msgr2=0x7f9a4c19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c19a8a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00b830 tx=0x7f9a3c00bd10 comp rx=0 tx=0).stop 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 shutdown_connections 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9a34039ae0 0x7f9a3403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c19b260 0x7f9a4c1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4c103ab0 0x7f9a4c19a8a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 --2- 172.21.15.157:0/1275738061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a4c19ade0 0x7f9a4c19fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 >> 172.21.15.157:0/1275738061 conn(0x7f9a4c0fd460 msgr2=0x7f9a4c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 shutdown_connections 2024-04-26T20:00:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:00:28.397+0000 7f9a540e2700 1 -- 172.21.15.157:0/1275738061 wait complete. 2024-04-26T20:00:28.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:28.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: Reconfiguring mon.b (monmap changed)... 2024-04-26T20:00:28.733 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:28.733 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:28.734 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:28.734 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: Reconfiguring daemon mon.b on smithi148 2024-04-26T20:00:28.734 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:28.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: Reconfiguring mon.b (monmap changed)... 2024-04-26T20:00:28.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:28.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:28.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:28.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: Reconfiguring daemon mon.b on smithi148 2024-04-26T20:00:28.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: Reconfiguring mon.b (monmap changed)... 2024-04-26T20:00:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: Reconfiguring daemon mon.b on smithi148 2024-04-26T20:00:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:29.086 DEBUG:teuthology.orchestra.run.smithi148:mgr.y> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T20:00:29.088 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T20:00:29.090 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T20:00:29.090 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:00:29.090 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:00:29.149 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:00:29.149 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-04-26T20:00:29.201 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:00:29.201 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:29.201 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T20:00:29.201 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:29.201 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:29.201 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-26 19:59:30.065850399 +0000 2024-04-26T20:00:29.201 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-26 19:54:07.405806521 +0000 2024-04-26T20:00:29.202 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-26 19:54:07.405806521 +0000 2024-04-26T20:00:29.202 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-26 19:54:07.405806521 +0000 2024-04-26T20:00:29.202 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:00:29.286 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-26T20:00:29.286 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-26T20:00:29.286 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000369909 s, 1.4 MB/s 2024-04-26T20:00:29.289 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:00:29.342 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-04-26T20:00:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:00:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:29.392 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T20:00:29.393 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:29.393 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:29.393 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-26 19:59:30.128848986 +0000 2024-04-26T20:00:29.393 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-26 19:54:07.993793218 +0000 2024-04-26T20:00:29.393 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-26 19:54:07.993793218 +0000 2024-04-26T20:00:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-26 19:54:07.993793218 +0000 2024-04-26T20:00:29.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:00:29.473 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-26T20:00:29.474 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-26T20:00:29.474 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000348605 s, 1.5 MB/s 2024-04-26T20:00:29.477 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:00:29.531 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-04-26T20:00:29.581 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:00:29.581 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:29.581 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T20:00:29.581 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:29.581 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:29.582 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-26 19:59:30.173847977 +0000 2024-04-26T20:00:29.582 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-26 19:54:08.592779667 +0000 2024-04-26T20:00:29.582 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-26 19:54:08.592779667 +0000 2024-04-26T20:00:29.582 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-26 19:54:08.592779667 +0000 2024-04-26T20:00:29.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:00:29.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:29 smithi131 ceph-mon[27919]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:00:29.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:29 smithi131 ceph-mon[27919]: Saving service mgr spec with placement smithi131=x;smithi148=y;smithi157=z;count:3 2024-04-26T20:00:29.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:29.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:29.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:29.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:29.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:29.659 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-26T20:00:29.660 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-26T20:00:29.660 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000339266 s, 1.5 MB/s 2024-04-26T20:00:29.664 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:00:29.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:29 smithi157 ceph-mon[32703]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:00:29.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:29 smithi157 ceph-mon[32703]: Saving service mgr spec with placement smithi131=x;smithi148=y;smithi157=z;count:3 2024-04-26T20:00:29.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:29.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:29.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:29.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:29.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:29.714 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-04-26T20:00:29.763 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:00:29.763 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:29.763 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-26T20:00:29.763 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:29.763 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:29.763 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-26 19:59:30.218846967 +0000 2024-04-26T20:00:29.763 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-26 19:54:09.210765684 +0000 2024-04-26T20:00:29.764 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-26 19:54:09.210765684 +0000 2024-04-26T20:00:29.764 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-26 19:54:09.210765684 +0000 2024-04-26T20:00:29.764 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:00:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:29 smithi148 ceph-mon[32281]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:00:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:29 smithi148 ceph-mon[32281]: Saving service mgr spec with placement smithi131=x;smithi148=y;smithi157=z;count:3 2024-04-26T20:00:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T20:00:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T20:00:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-26T20:00:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-26T20:00:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000307653 s, 1.7 MB/s 2024-04-26T20:00:29.842 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:00:29.896 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-26T20:00:29.896 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:00:29.914 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:00:29.915 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-26 20:00:15.618858687 +0000 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-26 19:53:17.909542703 +0000 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-26 19:53:17.909542703 +0000 2024-04-26T20:00:29.972 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-26 19:53:17.909542703 +0000 2024-04-26T20:00:29.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:00:30.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-26T20:00:30.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-26T20:00:30.037 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000233363 s, 2.2 MB/s 2024-04-26T20:00:30.038 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:00:30.096 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-04-26T20:00:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:00:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:30.153 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-26T20:00:30.153 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:30.153 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:30.153 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-26 20:00:15.618858687 +0000 2024-04-26T20:00:30.153 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-26 19:53:18.267533068 +0000 2024-04-26T20:00:30.154 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-26 19:53:18.267533068 +0000 2024-04-26T20:00:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-26 19:53:18.267533068 +0000 2024-04-26T20:00:30.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:00:30.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-26T20:00:30.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-26T20:00:30.218 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000233667 s, 2.2 MB/s 2024-04-26T20:00:30.219 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:00:30.276 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-26 20:00:15.618858687 +0000 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-26 19:53:18.616523675 +0000 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-26 19:53:18.616523675 +0000 2024-04-26T20:00:30.332 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-26 19:53:18.616523675 +0000 2024-04-26T20:00:30.332 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:00:30.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-26T20:00:30.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-26T20:00:30.395 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000227624 s, 2.2 MB/s 2024-04-26T20:00:30.396 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:00:30.453 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-04-26T20:00:30.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:30 smithi157 ceph-mon[32703]: Reconfiguring mon.c (monmap changed)... 2024-04-26T20:00:30.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:30 smithi157 ceph-mon[32703]: Reconfiguring daemon mon.c on smithi157 2024-04-26T20:00:30.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:30 smithi157 ceph-mon[32703]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:30.508 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:00:30.508 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:30.508 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T20:00:30.509 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:30.509 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:30.509 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-26 20:00:15.619858662 +0000 2024-04-26T20:00:30.509 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-26 19:53:18.945514821 +0000 2024-04-26T20:00:30.509 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-26 19:53:18.945514821 +0000 2024-04-26T20:00:30.509 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-26 19:53:18.945514821 +0000 2024-04-26T20:00:30.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:00:30.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-26T20:00:30.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-26T20:00:30.572 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000196832 s, 2.6 MB/s 2024-04-26T20:00:30.573 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:00:30.630 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-26T20:00:30.630 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:00:30.645 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:00:30.645 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-26 20:00:06.169547328 +0000 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-26 19:53:18.418145790 +0000 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-26 19:53:18.418145790 +0000 2024-04-26T20:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-26 19:53:18.418145790 +0000 2024-04-26T20:00:30.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:00:30.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:30 smithi131 ceph-mon[27919]: Reconfiguring mon.c (monmap changed)... 2024-04-26T20:00:30.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:30 smithi131 ceph-mon[27919]: Reconfiguring daemon mon.c on smithi157 2024-04-26T20:00:30.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:30 smithi131 ceph-mon[27919]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-26T20:00:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-26T20:00:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.9541e-05 s, 5.1 MB/s 2024-04-26T20:00:30.767 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:00:30.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:30 smithi148 ceph-mon[32281]: Reconfiguring mon.c (monmap changed)... 2024-04-26T20:00:30.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:30 smithi148 ceph-mon[32281]: Reconfiguring daemon mon.c on smithi157 2024-04-26T20:00:30.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:30 smithi148 ceph-mon[32281]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:30.824 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-04-26T20:00:30.878 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:00:30.878 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:30.878 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T20:00:30.878 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:30.878 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:30.878 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-26 20:00:06.169547328 +0000 2024-04-26T20:00:30.878 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-26 19:53:18.764138571 +0000 2024-04-26T20:00:30.879 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-26 19:53:18.764138571 +0000 2024-04-26T20:00:30.879 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-26 19:53:18.764138571 +0000 2024-04-26T20:00:30.879 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:00:30.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-26T20:00:30.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-26T20:00:30.941 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.5125e-05 s, 5.4 MB/s 2024-04-26T20:00:30.942 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:00:30.998 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-04-26T20:00:31.053 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:00:31.053 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:31.053 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T20:00:31.053 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:31.053 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:31.053 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-26 20:00:06.169547328 +0000 2024-04-26T20:00:31.054 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-26 19:53:19.083131916 +0000 2024-04-26T20:00:31.054 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-26 19:53:19.083131916 +0000 2024-04-26T20:00:31.054 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-26 19:53:19.083131916 +0000 2024-04-26T20:00:31.054 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:00:31.118 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-26T20:00:31.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-26T20:00:31.119 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.1921e-05 s, 5.6 MB/s 2024-04-26T20:00:31.120 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:00:31.177 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-26 20:00:06.170547309 +0000 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-26 19:53:19.422124844 +0000 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-26 19:53:19.422124844 +0000 2024-04-26T20:00:31.232 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-26 19:53:19.422124844 +0000 2024-04-26T20:00:31.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:00:31.293 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-26T20:00:31.293 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-26T20:00:31.293 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000111407 s, 4.6 MB/s 2024-04-26T20:00:31.295 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:00:31.350 INFO:tasks.cephadm:Deploying osd.0 on smithi131 with /dev/vg_nvme/lv_4... 2024-04-26T20:00:31.350 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T20:00:31.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:31.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: Deploying daemon mgr.z on smithi157 2024-04-26T20:00:31.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:31 smithi157 ceph-mon[32703]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:31.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:31.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:31.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:31.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:31.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T20:00:31.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:31.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:31.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: Deploying daemon mgr.z on smithi157 2024-04-26T20:00:31.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:31 smithi131 ceph-mon[27919]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:00:31.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: Deploying daemon mgr.z on smithi157 2024-04-26T20:00:31.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:31 smithi148 ceph-mon[32281]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:32.730 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:32 smithi157 systemd[1]: Starting Ceph mgr.z for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:00:33.065 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:32 smithi157 podman[34561]: 2024-04-26 20:00:32.672416015 +0000 UTC m=+0.017782710 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:00:33.065 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:32 smithi157 podman[34561]: 2024-04-26 20:00:32.771069848 +0000 UTC m=+0.116436519 container create 8e82ed36c14dba33ec661c1e1887293ddd2cb92c70423ce981662cb3a3d7dd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:00:33.066 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:32 smithi157 podman[34561]: 2024-04-26 20:00:32.981976572 +0000 UTC m=+0.327343244 container init 8e82ed36c14dba33ec661c1e1887293ddd2cb92c70423ce981662cb3a3d7dd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T20:00:33.066 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:32 smithi157 podman[34561]: 2024-04-26 20:00:32.984527949 +0000 UTC m=+0.329894621 container start 8e82ed36c14dba33ec661c1e1887293ddd2cb92c70423ce981662cb3a3d7dd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T20:00:33.066 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 ceph-mgr[34580]: -- 172.21.15.157:0/4142765323 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfaf55c5a0 con 0x55cfaf530800 2024-04-26T20:00:33.066 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 bash[34561]: 8e82ed36c14dba33ec661c1e1887293ddd2cb92c70423ce981662cb3a3d7dd58 2024-04-26T20:00:33.339 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 systemd[1]: Started Ceph mgr.z for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:00:33.340 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:33.152+0000 7fb820cfe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:00:33.340 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:33.338+0000 7fb820cfe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:00:33.730 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:33.439+0000 7fb820cfe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:00:34.146 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:33.743+0000 7fb820cfe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:00:34.147 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:33.825+0000 7fb820cfe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:00:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T20:00:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:34.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:34.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: Deploying daemon mgr.y on smithi148 2024-04-26T20:00:34.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:34 smithi157 ceph-mon[32703]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:34.502 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T20:00:34.502 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T20:00:34.502 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-04-26T20:00:34.502 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-04-26T20:00:34.502 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0340163 s, 308 MB/s 2024-04-26T20:00:34.502 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-04-26T20:00:34.502 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.502 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:34.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T20:00:34.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:34.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:34.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: Deploying daemon mgr.y on smithi148 2024-04-26T20:00:34.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:34 smithi131 ceph-mon[27919]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: Deploying daemon mgr.y on smithi148 2024-04-26T20:00:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:34 smithi148 ceph-mon[32281]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:35.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-04-26T20:00:35.311 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 systemd[1]: Starting Ceph mgr.y for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:00:35.350 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:35.016+0000 7fb820cfe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:00:35.351 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:35.181+0000 7fb820cfe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:00:35.351 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:35.349+0000 7fb820cfe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:00:35.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 podman[33659]: 2024-04-26 20:00:35.311116643 +0000 UTC m=+0.107741271 container create 4409d5162bb4dbf87b4bd955035529e48f39ceb47869b6d4f065a83388e3413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T20:00:35.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 podman[33659]: 2024-04-26 20:00:35.221624971 +0000 UTC m=+0.018249627 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:00:35.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 podman[33659]: 2024-04-26 20:00:35.513433885 +0000 UTC m=+0.310058514 container init 4409d5162bb4dbf87b4bd955035529e48f39ceb47869b6d4f065a83388e3413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T20:00:35.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 podman[33659]: 2024-04-26 20:00:35.516728461 +0000 UTC m=+0.313353097 container start 4409d5162bb4dbf87b4bd955035529e48f39ceb47869b6d4f065a83388e3413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T20:00:35.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 ceph-mgr[33678]: -- 172.21.15.148:0/3746344571 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56435da7c5a0 con 0x56435da50800 2024-04-26T20:00:35.734 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:00:35.857 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 bash[33659]: 4409d5162bb4dbf87b4bd955035529e48f39ceb47869b6d4f065a83388e3413c 2024-04-26T20:00:35.857 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 systemd[1]: Started Ceph mgr.y for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:00:35.857 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:35.670+0000 7fdf5884f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:00:35.857 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:35.856+0000 7fdf5884f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:00:35.924 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:35.629+0000 7fb820cfe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:00:35.924 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:35.720+0000 7fb820cfe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:00:35.924 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:35.922+0000 7fb820cfe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:00:36.230 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:36.122+0000 7fb820cfe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:00:36.293 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:35.963+0000 7fdf5884f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:00:36.566 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:36.307+0000 7fdf5884f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:00:36.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:36.399+0000 7fdf5884f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:00:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-mon[32281]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:36.779 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:36 smithi131 ceph-mon[27919]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:36.779 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.779 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.779 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.779 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.780 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:36.820 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:36.764+0000 7fb820cfe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:00:36.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-mon[32703]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:36.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:36.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:37.480 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:37.209+0000 7fb820cfe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:00:37.480 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:37.307+0000 7fb820cfe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 -- 172.21.15.131:0/869207669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 msgr2=0x7f41300762a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 --2- 172.21.15.131:0/869207669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f41300762a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f411c009990 tx=0x7f411c009ca0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 -- 172.21.15.131:0/869207669 shutdown_connections 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 --2- 172.21.15.131:0/869207669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413010b3b0 0x7f413010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 --2- 172.21.15.131:0/869207669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41300773f0 0x7f4130075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 --2- 172.21.15.131:0/869207669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f41300762a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 -- 172.21.15.131:0/869207669 >> 172.21.15.131:0/869207669 conn(0x7f41300fea70 msgr2=0x7f4130100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 -- 172.21.15.131:0/869207669 shutdown_connections 2024-04-26T20:00:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.851+0000 7f4135c00700 1 -- 172.21.15.131:0/869207669 wait complete. 2024-04-26T20:00:37.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.853+0000 7f4135c00700 1 Processor -- start 2024-04-26T20:00:37.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.853+0000 7f4135c00700 1 -- start start 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f4135c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f413019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f4135c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41300773f0 0x7f413019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f4135c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413010b3b0 0x7f41301a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f4135c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4130111170 con 0x7f41300773f0 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f4135c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4130110ff0 con 0x7f4130075e20 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f4135c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f41301112f0 con 0x7f413010b3b0 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f413019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412e000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41300773f0 0x7f413019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f413019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:48386/0 (socket says 172.21.15.131:48386) 2024-04-26T20:00:37.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412ea00700 1 -- 172.21.15.131:0/3221014031 learned_addr learned my addr 172.21.15.131:0/3221014031 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:00:37.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412e000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41300773f0 0x7f413019bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57188/0 (socket says 172.21.15.131:57188) 2024-04-26T20:00:37.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413010b3b0 0x7f41301a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412ea00700 1 -- 172.21.15.131:0/3221014031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413010b3b0 msgr2=0x7f41301a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.854+0000 7f412ea00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413010b3b0 0x7f41301a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f412ea00700 1 -- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41300773f0 msgr2=0x7f413019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f412ea00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41300773f0 0x7f413019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f412ea00700 1 -- 172.21.15.131:0/3221014031 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41301a3440 con 0x7f4130075e20 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f412f400700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413010b3b0 0x7f41301a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f412ea00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f413019b830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f412000e9a0 tx=0x7f412000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000ca10 con 0x7f4130075e20 2024-04-26T20:00:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412000cbb0 con 0x7f4130075e20 2024-04-26T20:00:37.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120018680 con 0x7f4130075e20 2024-04-26T20:00:37.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41301a3730 con 0x7f4130075e20 2024-04-26T20:00:37.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.855+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41301a3c20 con 0x7f4130075e20 2024-04-26T20:00:37.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.856+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f4120018820 con 0x7f4130075e20 2024-04-26T20:00:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.857+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413006a110 con 0x7f4130075e20 2024-04-26T20:00:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.857+0000 7f4127400700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4114039890 0x7f411403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.857+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f412004cb50 con 0x7f4130075e20 2024-04-26T20:00:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.857+0000 7f412e000700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4114039890 0x7f411403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.858+0000 7f412e000700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4114039890 0x7f411403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b2e0 tx=0x7f411c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:37.863+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f412000fc70 con 0x7f4130075e20 2024-04-26T20:00:37.980 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:37.506+0000 7fb820cfe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:00:38.064 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:37.722+0000 7fdf5884f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:00:38.065 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:37.891+0000 7fdf5884f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:00:38.065 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:38.063+0000 7fdf5884f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:00:38.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:38.103+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4130063860 con 0x7f4114039890 2024-04-26T20:00:38.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:38.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:38.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:00:38.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:00:38.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.351 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:38.024+0000 7fb820cfe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:00:38.351 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:38.113+0000 7fb820cfe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:00:38.351 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.351 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:00:38.352 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T20:00:38.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T20:00:38.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:00:38.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:00:38.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:38.652 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:38.352+0000 7fdf5884f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:00:38.652 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:38.445+0000 7fdf5884f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:00:38.652 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:38.651+0000 7fdf5884f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:00:38.685 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:38.349+0000 7fb820cfe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:00:38.685 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:38.479+0000 7fb820cfe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:00:38.685 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:38.572+0000 7fb820cfe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:00:38.685 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:38.683+0000 7fb820cfe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:00:39.066 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:38.843+0000 7fdf5884f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:00:39.390 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:38.984+0000 7fb820cfe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:00:39.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:39 smithi131 ceph-mon[27919]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T20:00:39.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:39 smithi131 ceph-mon[27919]: Reconfiguring daemon mgr.x on smithi131 2024-04-26T20:00:39.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:39 smithi131 ceph-mon[27919]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:00:39.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:39.730 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:39.574+0000 7fb820cfe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:00:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:39 smithi157 ceph-mon[32703]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T20:00:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:39 smithi157 ceph-mon[32703]: Reconfiguring daemon mgr.x on smithi131 2024-04-26T20:00:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:39 smithi157 ceph-mon[32703]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:00:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:39.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:39 smithi148 ceph-mon[32281]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T20:00:39.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:39 smithi148 ceph-mon[32281]: Reconfiguring daemon mgr.x on smithi131 2024-04-26T20:00:39.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:39 smithi148 ceph-mon[32281]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:00:39.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:39.817 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:39.451+0000 7fdf5884f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:00:40.177 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:39.876+0000 7fdf5884f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:00:40.177 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:39.974+0000 7fdf5884f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:00:40.177 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:40.176+0000 7fdf5884f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:00:40.230 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:00:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z[34576]: 2024-04-26T20:00:39.785+0000 7fb820cfe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: Standby manager daemon z started 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T20:00:40.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T20:00:40.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:40.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: Standby manager daemon z started 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T20:00:40.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:40 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T20:00:40.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:40.731+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7f41200178f0 con 0x7f4130075e20 2024-04-26T20:00:40.734 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:40.734 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.734 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.734 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:40.734 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:00:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: Standby manager daemon z started 2024-04-26T20:00:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T20:00:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T20:00:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T20:00:40.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:40 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.157:0/3428590865' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T20:00:41.051 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:40.712+0000 7fdf5884f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:00:41.051 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:40.804+0000 7fdf5884f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:00:41.317 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:41.050+0000 7fdf5884f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:00:41.317 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:41.182+0000 7fdf5884f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:00:41.317 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:41.277+0000 7fdf5884f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:00:41.698 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:41.393+0000 7fdf5884f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:00:41.698 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:41.697+0000 7fdf5884f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:00:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:41 smithi131 ceph-mon[27919]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-26T20:00:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T20:00:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:41 smithi131 ceph-mon[27919]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:42.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-mon[32281]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-26T20:00:42.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T20:00:42.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:41 smithi148 ceph-mon[32281]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:41 smithi157 ceph-mon[32703]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-26T20:00:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T20:00:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:41 smithi157 ceph-mon[32703]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:42.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:42.298+0000 7fdf5884f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:00:42.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y[33674]: 2024-04-26T20:00:42.515+0000 7fdf5884f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:00:42.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:00:42.756+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mon.2 v2:172.21.15.148:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f4120018c10 con 0x7f4130075e20 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: Standby manager daemon y started 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1534440744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]: dispatch 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]: dispatch 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]': finished 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T20:00:43.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:42 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: Standby manager daemon y started 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1534440744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]: dispatch 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]: dispatch 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]': finished 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T20:00:43.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:42 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:00:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T20:00:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: Standby manager daemon y started 2024-04-26T20:00:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T20:00:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T20:00:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='mgr.? 172.21.15.148:0/1342840078' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T20:00:43.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1534440744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]: dispatch 2024-04-26T20:00:43.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]: dispatch 2024-04-26T20:00:43.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec125b6b-d0b4-42f7-80a6-66edb6de6715"}]': finished 2024-04-26T20:00:43.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T20:00:43.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:42 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:00:44.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:43 smithi148 ceph-mon[32281]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-26T20:00:44.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:43 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T20:00:44.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:43 smithi148 ceph-mon[32281]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:44.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:43 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2461564832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:00:44.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:43 smithi157 ceph-mon[32703]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-26T20:00:44.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:43 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T20:00:44.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:43 smithi157 ceph-mon[32703]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:44.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:43 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2461564832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:00:44.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:43 smithi131 ceph-mon[27919]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-26T20:00:44.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:43 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T20:00:44.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:43 smithi131 ceph-mon[27919]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:44.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:43 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2461564832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:00:46.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:46 smithi148 ceph-mon[32281]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:46.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:46 smithi157 ceph-mon[32703]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:47.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:46 smithi131 ceph-mon[27919]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:47 smithi148 ceph-mon[32281]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:47.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:47 smithi157 ceph-mon[32703]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:48.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:47 smithi131 ceph-mon[27919]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:50.665 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:50 smithi131 ceph-mon[27919]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:50.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:50 smithi148 ceph-mon[32281]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:50.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:50 smithi157 ceph-mon[32703]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:51.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:51 smithi131 ceph-mon[27919]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:51.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:51 smithi157 ceph-mon[32703]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:52.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:51 smithi148 ceph-mon[32281]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:52.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T20:00:52.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:52.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:52 smithi131 ceph-mon[27919]: Deploying daemon osd.0 on smithi131 2024-04-26T20:00:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T20:00:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:52 smithi157 ceph-mon[32703]: Deploying daemon osd.0 on smithi131 2024-04-26T20:00:53.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T20:00:53.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:53.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:52 smithi148 ceph-mon[32281]: Deploying daemon osd.0 on smithi131 2024-04-26T20:00:53.940 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:53 smithi131 ceph-mon[27919]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:53.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:53 smithi157 ceph-mon[32703]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:53 smithi148 ceph-mon[32281]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:56.635 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:56 smithi131 ceph-mon[27919]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:56.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:56 smithi148 ceph-mon[32281]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:56.960 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:56 smithi157 ceph-mon[32703]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:57.840 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:00:57 smithi131 ceph-mon[27919]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:57.881 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:00:57 smithi157 ceph-mon[32703]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:00:58.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:00:57 smithi148 ceph-mon[32281]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:00 smithi131 ceph-mon[27919]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:00.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:00 smithi148 ceph-mon[32281]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:00.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:00.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:00.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:00 smithi157 ceph-mon[32703]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:02.652 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:02 smithi131 ceph-mon[27919]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:02.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:02 smithi148 ceph-mon[32281]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:02.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:02 smithi157 ceph-mon[32703]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:03.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:03 smithi157 ceph-mon[32703]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:04.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:03 smithi131 ceph-mon[27919]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:04.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:03 smithi148 ceph-mon[32281]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:05.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:05.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:05.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:04 smithi131 ceph-mon[27919]: from='osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T20:01:05.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:04 smithi131 ceph-mon[27919]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T20:01:05.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:05.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:05.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:04 smithi157 ceph-mon[32703]: from='osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T20:01:05.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:04 smithi157 ceph-mon[32703]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T20:01:05.240 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.240 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.240 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:05.241 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:05.241 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:05.241 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:04 smithi148 ceph-mon[32281]: from='osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T20:01:05.241 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:04 smithi148 ceph-mon[32281]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T20:01:05.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.345+0000 7f4127400700 1 -- 172.21.15.131:0/3221014031 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4130063860 con 0x7f4114039890 2024-04-26T20:01:05.347 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 0 on host 'smithi131' 2024-04-26T20:01:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4114039890 msgr2=0x7f411403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4114039890 0x7f411403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b2e0 tx=0x7f411c005fb0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 msgr2=0x7f413019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f413019b830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f412000e9a0 tx=0x7f412000ecb0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 shutdown_connections 2024-04-26T20:01:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413010b3b0 0x7f41301a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4114039890 0x7f411403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4130075e20 0x7f413019b830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 --2- 172.21.15.131:0/3221014031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41300773f0 0x7f413019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 >> 172.21.15.131:0/3221014031 conn(0x7f41300fea70 msgr2=0x7f4130107300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 shutdown_connections 2024-04-26T20:01:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:01:05.349+0000 7f4135c00700 1 -- 172.21.15.131:0/3221014031 wait complete. 2024-04-26T20:01:06.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T20:01:06.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: from='osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:01:06.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T20:01:06.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:06.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:01:06.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:06.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:06.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:06.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:05 smithi131 ceph-mon[27919]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:06.156 DEBUG:teuthology.orchestra.run.smithi131:osd.0> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T20:01:06.161 INFO:tasks.cephadm:Deploying osd.1 on smithi148 with /dev/vg_nvme/lv_4... 2024-04-26T20:01:06.162 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T20:01:06.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T20:01:06.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: from='osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:01:06.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T20:01:06.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:06.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:01:06.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:06.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:06.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:06.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:05 smithi157 ceph-mon[32703]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: from='osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:05 smithi148 ceph-mon[32281]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:06.395 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:01:07.239 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:06 smithi131 ceph-mon[27919]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:01:07.239 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:06 smithi131 ceph-mon[27919]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T20:01:07.239 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:07.239 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:07.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:06 smithi148 ceph-mon[32281]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:01:07.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:06 smithi148 ceph-mon[32281]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T20:01:07.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:07.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:07.391 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:06 smithi157 ceph-mon[32703]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:01:07.391 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:06 smithi157 ceph-mon[32703]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T20:01:07.391 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:07.391 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:08.148 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T20:01:08.148 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T20:01:08.148 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-26T20:01:08.148 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-26T20:01:08.148 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122201 s, 858 MB/s 2024-04-26T20:01:08.148 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-26T20:01:08.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:07 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:01:08.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:07 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:01:08.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:07 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:08.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:07 smithi157 ceph-mon[32703]: osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] boot 2024-04-26T20:01:08.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:07 smithi157 ceph-mon[32703]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T20:01:08.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:07 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:08.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:07 smithi157 ceph-mon[32703]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:08.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:07 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:01:08.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:07 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:01:08.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:07 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:08.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:07 smithi131 ceph-mon[27919]: osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] boot 2024-04-26T20:01:08.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:07 smithi131 ceph-mon[27919]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T20:01:08.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:08 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:08.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:08 smithi131 ceph-mon[27919]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:08.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:07 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:01:08.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:07 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:01:08.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:07 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:08.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:07 smithi148 ceph-mon[32281]: osd.0 [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] boot 2024-04-26T20:01:08.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:07 smithi148 ceph-mon[32281]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T20:01:08.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:07 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T20:01:08.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:07 smithi148 ceph-mon[32281]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:01:08.783 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-04-26T20:01:09.029 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:01:09.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:09 smithi148 ceph-mon[32281]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T20:01:09.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:09 smithi131 ceph-mon[27919]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T20:01:09.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:09 smithi157 ceph-mon[32703]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T20:01:10.201 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:10 smithi148 ceph-mon[32281]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 -- 172.21.15.148:0/2700387060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 msgr2=0x7f78b4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 --2- 172.21.15.148:0/2700387060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4108a90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009990 tx=0x7f78a4009ca0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 -- 172.21.15.148:0/2700387060 shutdown_connections 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 --2- 172.21.15.148:0/2700387060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109060 0x7f78b410b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 --2- 172.21.15.148:0/2700387060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4101120 0x7f78b4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 --2- 172.21.15.148:0/2700387060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4108a90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 -- 172.21.15.148:0/2700387060 >> 172.21.15.148:0/2700387060 conn(0x7f78b40fcba0 msgr2=0x7f78b40fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:10.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 -- 172.21.15.148:0/2700387060 shutdown_connections 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.418+0000 7f78bad9e700 1 -- 172.21.15.148:0/2700387060 wait complete. 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 Processor -- start 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 -- start start 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4101120 0x7f78b41043d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109060 0x7f78b4102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78b410ef10 con 0x7f78b4101120 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78b410ed90 con 0x7f78b4101a80 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78bad9e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78b410f090 con 0x7f78b4109060 2024-04-26T20:01:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4102a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57410/0 (socket says 172.21.15.148:57410) 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 -- 172.21.15.148:0/501139507 learned_addr learned my addr 172.21.15.148:0/501139507 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78ba59d700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109060 0x7f78b4102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 -- 172.21.15.148:0/501139507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109060 msgr2=0x7f78b4102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109060 0x7f78b4102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 -- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4101120 msgr2=0x7f78b41043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4101120 0x7f78b41043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78b959b700 1 -- 172.21.15.148:0/501139507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b4103810 con 0x7f78b4101a80 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.419+0000 7f78ba59d700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109060 0x7f78b4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.420+0000 7f78b959b700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4102a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f78a40098e0 tx=0x7f78a4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.420+0000 7f78aaffd700 1 -- 172.21.15.148:0/501139507 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4027070 con 0x7f78b4101a80 2024-04-26T20:01:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.420+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b41a3770 con 0x7f78b4101a80 2024-04-26T20:01:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.420+0000 7f78aaffd700 1 -- 172.21.15.148:0/501139507 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a4022470 con 0x7f78b4101a80 2024-04-26T20:01:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.420+0000 7f78aaffd700 1 -- 172.21.15.148:0/501139507 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a400f6a0 con 0x7f78b4101a80 2024-04-26T20:01:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.420+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b41a3c40 con 0x7f78b4101a80 2024-04-26T20:01:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.421+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b406a110 con 0x7f78b4101a80 2024-04-26T20:01:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.422+0000 7f78aaffd700 1 -- 172.21.15.148:0/501139507 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f78a402a070 con 0x7f78b4101a80 2024-04-26T20:01:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.422+0000 7f78aaffd700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78a00a3bb0 0x7f78a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.422+0000 7f78aaffd700 1 -- 172.21.15.148:0/501139507 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f78a40cf840 con 0x7f78b4101a80 2024-04-26T20:01:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.423+0000 7f78b9d9c700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78a00a3bb0 0x7f78a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.424+0000 7f78b9d9c700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78a00a3bb0 0x7f78a00a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78b00097b0 tx=0x7f78b0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.427+0000 7f78aaffd700 1 -- 172.21.15.148:0/501139507 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f78a4013070 con 0x7f78b4101a80 2024-04-26T20:01:10.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:10 smithi157 ceph-mon[32703]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:10.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:10 smithi131 ceph-mon[27919]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:10.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:10.637+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f78b4063860 con 0x7f78a00a3bb0 2024-04-26T20:01:11.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:01:11.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:01:11.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:11.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:01:11.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:01:11.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:11.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:01:11.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:01:11.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:12.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:01:12.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi131 2024-04-26T20:01:12.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.258 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:12.258 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi131 to 17089M 2024-04-26T20:01:12.258 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.258 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:12.258 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:12.258 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.258 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:12 smithi148 ceph-mon[32281]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:12.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:01:12.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi131 2024-04-26T20:01:12.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:12.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi131 to 17089M 2024-04-26T20:01:12.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:12.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:12.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:12 smithi157 ceph-mon[32703]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:12.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:01:12.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi131 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi131 to 17089M 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:12.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:12 smithi131 ceph-mon[27919]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:13.698 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:13 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/2037307963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]: dispatch 2024-04-26T20:01:13.698 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:13 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]: dispatch 2024-04-26T20:01:13.699 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:13 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]': finished 2024-04-26T20:01:13.699 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:13 smithi148 ceph-mon[32281]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T20:01:13.699 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:13.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:13 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/2037307963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]: dispatch 2024-04-26T20:01:13.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:13 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]: dispatch 2024-04-26T20:01:13.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:13 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]': finished 2024-04-26T20:01:13.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:13 smithi157 ceph-mon[32703]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T20:01:13.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:13 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/2037307963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]: dispatch 2024-04-26T20:01:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:13 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]: dispatch 2024-04-26T20:01:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:13 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35db8b5f-c13f-4d01-872d-aae172804538"}]': finished 2024-04-26T20:01:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:13 smithi131 ceph-mon[27919]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T20:01:13.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:14.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:14 smithi157 ceph-mon[32703]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:14.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:14 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/2351274504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:01:14.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:14 smithi131 ceph-mon[27919]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:14.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:14 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/2351274504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:01:14.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:14 smithi148 ceph-mon[32281]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:14.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:14 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/2351274504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:01:16.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:16 smithi148 ceph-mon[32281]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:16.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:16 smithi157 ceph-mon[32703]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:17.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:16 smithi131 ceph-mon[27919]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:17.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:17 smithi157 ceph-mon[32703]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:18.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:17 smithi131 ceph-mon[27919]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:18.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:17 smithi148 ceph-mon[32281]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:20.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:20 smithi148 ceph-mon[32281]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T20:01:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:20.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:20 smithi157 ceph-mon[32703]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:20.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T20:01:20.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:21.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:20 smithi131 ceph-mon[27919]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:21.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T20:01:21.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:21.680 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:21 smithi148 ceph-mon[32281]: Deploying daemon osd.1 on smithi148 2024-04-26T20:01:21.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:21 smithi157 ceph-mon[32703]: Deploying daemon osd.1 on smithi148 2024-04-26T20:01:22.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:21 smithi131 ceph-mon[27919]: Deploying daemon osd.1 on smithi148 2024-04-26T20:01:22.634 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:22 smithi148 ceph-mon[32281]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:22 smithi157 ceph-mon[32703]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:23.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:22 smithi131 ceph-mon[27919]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:23.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:23 smithi148 ceph-mon[32281]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:23.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:23 smithi157 ceph-mon[32703]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:24.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:23 smithi131 ceph-mon[27919]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:26.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:26 smithi148 ceph-mon[32281]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:26.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:26 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:26.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:26 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:26.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:26 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:26.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:26 smithi157 ceph-mon[32703]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:26.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:26 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:26.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:26 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:26.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:26 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:27.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:26 smithi131 ceph-mon[27919]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:27.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:26 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:27.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:26 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:27.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:26 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:27.755 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:27 smithi148 ceph-mon[32281]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:27.953 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:27 smithi157 ceph-mon[32703]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:28.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:27 smithi131 ceph-mon[27919]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:29.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.072+0000 7f78aaffd700 1 -- 172.21.15.148:0/501139507 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78b4063860 con 0x7f78a00a3bb0 2024-04-26T20:01:29.074 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 1 on host 'smithi148' 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78a00a3bb0 msgr2=0x7f78a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78a00a3bb0 0x7f78a00a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78b00097b0 tx=0x7f78b0006d20 comp rx=0 tx=0).stop 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 msgr2=0x7f78b4102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4102a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f78a40098e0 tx=0x7f78a4009f90 comp rx=0 tx=0).stop 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 shutdown_connections 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109060 0x7f78b4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4101120 0x7f78b41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78a00a3bb0 0x7f78a00a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 --2- 172.21.15.148:0/501139507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4101a80 0x7f78b4102a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 >> 172.21.15.148:0/501139507 conn(0x7f78b40fcba0 msgr2=0x7f78b4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 shutdown_connections 2024-04-26T20:01:29.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:01:29.075+0000 7f78bad9e700 1 -- 172.21.15.148:0/501139507 wait complete. 2024-04-26T20:01:29.713 DEBUG:teuthology.orchestra.run.smithi148:osd.1> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T20:01:29.716 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-04-26T20:01:29.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T20:01:29.960 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:01:30.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:30 smithi157 ceph-mon[32703]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:30.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:30 smithi157 ceph-mon[32703]: from='osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T20:01:30.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:30 smithi157 ceph-mon[32703]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T20:01:30.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:30 smithi148 ceph-mon[32281]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:30.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:30 smithi148 ceph-mon[32281]: from='osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T20:01:30.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:30 smithi148 ceph-mon[32281]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T20:01:30.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:30.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:30 smithi131 ceph-mon[27919]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:30.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:30 smithi131 ceph-mon[27919]: from='osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T20:01:30.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:30 smithi131 ceph-mon[27919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T20:01:31.405 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:31 smithi157 ceph-mon[32703]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T20:01:31.405 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:31 smithi157 ceph-mon[32703]: from='osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:01:31.405 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:31 smithi157 ceph-mon[32703]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T20:01:31.405 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:31.406 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:31 smithi157 ceph-mon[32703]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:01:31.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:31 smithi131 ceph-mon[27919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T20:01:31.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:31 smithi131 ceph-mon[27919]: from='osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:01:31.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:31 smithi131 ceph-mon[27919]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T20:01:31.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:31.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:31 smithi131 ceph-mon[27919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:01:31.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:31 smithi148 ceph-mon[32281]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T20:01:31.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:31 smithi148 ceph-mon[32281]: from='osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:01:31.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:31 smithi148 ceph-mon[32281]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T20:01:31.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:31.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:31 smithi148 ceph-mon[32281]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:01:31.567 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:01:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1[36919]: 2024-04-26T20:01:31.138+0000 7f623e318700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T20:01:31.568 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:01:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1[36919]: 2024-04-26T20:01:31.147+0000 7f623a10b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:01:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T20:01:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T20:01:31.959 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-26T20:01:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-26T20:01:31.959 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00850045 s, 1.2 GB/s 2024-04-26T20:01:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-26T20:01:32.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:32 smithi157 ceph-mon[32703]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:01:32.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:32 smithi157 ceph-mon[32703]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T20:01:32.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:32.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:32.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:32 smithi157 ceph-mon[32703]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:32.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:32 smithi148 ceph-mon[32281]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:01:32.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:32 smithi148 ceph-mon[32281]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T20:01:32.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:32.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:32.313 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:32 smithi148 ceph-mon[32281]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:32.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:32 smithi131 ceph-mon[27919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:01:32.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:32 smithi131 ceph-mon[27919]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T20:01:32.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:32.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:32.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:32 smithi131 ceph-mon[27919]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T20:01:32.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-04-26T20:01:32.804 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:01:33.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] boot 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:33.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:33.232 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:33.232 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] boot 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:33.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:33.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:33.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:33.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:33.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:01:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:01:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: osd.1 [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] boot 2024-04-26T20:01:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:33.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:33.569 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:33.569 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 -- 172.21.15.157:0/3830663562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 msgr2=0x7f1d1010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/3830663562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d1010b8d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800b0a0 tx=0x7f1d0800b3b0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 -- 172.21.15.157:0/3830663562 shutdown_connections 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/3830663562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d1010b8d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/3830663562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d10103480 0x7f1d101038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/3830663562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d10104680 0x7f1d10104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 -- 172.21.15.157:0/3830663562 >> 172.21.15.157:0/3830663562 conn(0x7f1d100fea20 msgr2=0x7f1d10100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 -- 172.21.15.157:0/3830663562 shutdown_connections 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 -- 172.21.15.157:0/3830663562 wait complete. 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 Processor -- start 2024-04-26T20:01:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.262+0000 7f1d15c3d700 1 -- start start 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d15c3d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d10103480 0x7f1d10199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d15c3d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d10104680 0x7f1d10199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d15c3d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d101a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d15c3d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1010efc0 con 0x7f1d10104680 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d15c3d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1010ee40 con 0x7f1d10103480 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d15c3d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1010f140 con 0x7f1d10105020 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d101a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d101a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34060/0 (socket says 172.21.15.157:34060) 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 -- 172.21.15.157:0/1924356846 learned_addr learned my addr 172.21.15.157:0/1924356846 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0f7fe700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d10103480 0x7f1d10199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d06ffd700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d10104680 0x7f1d10199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 -- 172.21.15.157:0/1924356846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d10103480 msgr2=0x7f1d10199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d10103480 0x7f1d10199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 -- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d10104680 msgr2=0x7f1d10199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d10104680 0x7f1d10199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 -- 172.21.15.157:0/1924356846 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d101a1290 con 0x7f1d10105020 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.263+0000 7f1d0ffff700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d101a0bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006070 tx=0x7f1d08004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.264+0000 7f1d0d7fa700 1 -- 172.21.15.157:0/1924356846 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08021070 con 0x7f1d10105020 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.264+0000 7f1d0d7fa700 1 -- 172.21.15.157:0/1924356846 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d08004020 con 0x7f1d10105020 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.264+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d101a1520 con 0x7f1d10105020 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.264+0000 7f1d0d7fa700 1 -- 172.21.15.157:0/1924356846 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0801d690 con 0x7f1d10105020 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.264+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d101a1a50 con 0x7f1d10105020 2024-04-26T20:01:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.265+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1006a110 con 0x7f1d10105020 2024-04-26T20:01:34.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.266+0000 7f1d0d7fa700 1 -- 172.21.15.157:0/1924356846 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f1d080044f0 con 0x7f1d10105020 2024-04-26T20:01:34.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.266+0000 7f1d0d7fa700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1cfc0a3bb0 0x7f1cfc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.266+0000 7f1d0d7fa700 1 -- 172.21.15.157:0/1924356846 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f1d0800f020 con 0x7f1d10105020 2024-04-26T20:01:34.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.267+0000 7f1d0f7fe700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1cfc0a3bb0 0x7f1cfc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.270 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.268+0000 7f1d0f7fe700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1cfc0a3bb0 0x7f1cfc0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006fd0 tx=0x7f1d00008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.272 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.270+0000 7f1d0d7fa700 1 -- 172.21.15.157:0/1924356846 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1d0802a020 con 0x7f1d10105020 2024-04-26T20:01:34.470 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:34 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi148 2024-04-26T20:01:34.471 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:34 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-26T20:01:34.471 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:34 smithi148 ceph-mon[32281]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:34.471 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:34 smithi148 ceph-mon[32281]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T20:01:34.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:34.474+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1d10063860 con 0x7f1cfc0a3bb0 2024-04-26T20:01:34.476 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:34 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi148 2024-04-26T20:01:34.476 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:34 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-26T20:01:34.476 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:34 smithi157 ceph-mon[32703]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:34.476 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:34 smithi157 ceph-mon[32703]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T20:01:34.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:34 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi148 2024-04-26T20:01:34.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:34 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-26T20:01:34.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:34 smithi131 ceph-mon[27919]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:34.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:34 smithi131 ceph-mon[27919]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T20:01:35.428 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:35 smithi157 ceph-mon[32703]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:01:35.428 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:01:35.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:01:35.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:35.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:35 smithi131 ceph-mon[27919]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:01:35.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:01:35.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:01:35.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:35.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:35 smithi148 ceph-mon[32281]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:01:35.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:01:35.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:01:35.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:36.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:36 smithi157 ceph-mon[32703]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:36.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:36 smithi131 ceph-mon[27919]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:36.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:36 smithi148 ceph-mon[32281]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:37.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:37 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/2856505607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]: dispatch 2024-04-26T20:01:37.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:37 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]: dispatch 2024-04-26T20:01:37.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:37 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]': finished 2024-04-26T20:01:37.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:37 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/2856505607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]: dispatch 2024-04-26T20:01:37.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:37 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]: dispatch 2024-04-26T20:01:37.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:37 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]': finished 2024-04-26T20:01:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:37 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/2856505607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]: dispatch 2024-04-26T20:01:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:37 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]: dispatch 2024-04-26T20:01:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:37 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44"}]': finished 2024-04-26T20:01:38.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:38 smithi157 ceph-mon[32703]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T20:01:38.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:38.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:38 smithi157 ceph-mon[32703]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:38.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:38 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/299813262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:01:38.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:38 smithi131 ceph-mon[27919]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T20:01:38.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:38.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:38 smithi131 ceph-mon[27919]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:38.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:38 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/299813262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:01:38.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:38 smithi148 ceph-mon[32281]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T20:01:38.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:38.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:38 smithi148 ceph-mon[32281]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:38.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:38 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/299813262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:01:40.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:40 smithi157 ceph-mon[32703]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:40.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:40 smithi148 ceph-mon[32281]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:40 smithi131 ceph-mon[27919]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:41.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:41 smithi157 ceph-mon[32703]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:41 smithi131 ceph-mon[27919]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:42.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:41 smithi148 ceph-mon[32281]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:44.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:44 smithi131 ceph-mon[27919]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:44.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T20:01:44.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:44.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:44 smithi148 ceph-mon[32281]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:44.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T20:01:44.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:44.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:44 smithi157 ceph-mon[32703]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:44.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T20:01:44.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:45.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:45 smithi148 ceph-mon[32281]: Deploying daemon osd.2 on smithi157 2024-04-26T20:01:45.849 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:45 smithi157 ceph-mon[32703]: Deploying daemon osd.2 on smithi157 2024-04-26T20:01:46.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:45 smithi131 ceph-mon[27919]: Deploying daemon osd.2 on smithi157 2024-04-26T20:01:46.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:46 smithi148 ceph-mon[32281]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:46.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:46 smithi157 ceph-mon[32703]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:47.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:46 smithi131 ceph-mon[27919]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:47.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:47 smithi157 ceph-mon[32703]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:47.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:47 smithi148 ceph-mon[32281]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:48.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:47 smithi131 ceph-mon[27919]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:50 smithi157 ceph-mon[32703]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:50 smithi148 ceph-mon[32281]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:51.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:50 smithi131 ceph-mon[27919]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:51.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:51.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:51.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:51.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:51 smithi148 ceph-mon[32281]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:51.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:51 smithi157 ceph-mon[32703]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:52.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:51 smithi131 ceph-mon[27919]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:53.562 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.561+0000 7f1d0d7fa700 1 -- 172.21.15.157:0/1924356846 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d10063860 con 0x7f1cfc0a3bb0 2024-04-26T20:01:53.563 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.563+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1cfc0a3bb0 msgr2=0x7f1cfc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.563+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1cfc0a3bb0 0x7f1cfc0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006fd0 tx=0x7f1d00008040 comp rx=0 tx=0).stop 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.563+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 msgr2=0x7f1d101a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.563+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d101a0bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006070 tx=0x7f1d08004ef0 comp rx=0 tx=0).stop 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 shutdown_connections 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10105020 0x7f1d101a0bf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1cfc0a3bb0 0x7f1cfc0a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d10103480 0x7f1d10199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 --2- 172.21.15.157:0/1924356846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d10104680 0x7f1d10199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 >> 172.21.15.157:0/1924356846 conn(0x7f1d100fea20 msgr2=0x7f1d10107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 shutdown_connections 2024-04-26T20:01:53.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:01:53.564+0000 7f1d15c3d700 1 -- 172.21.15.157:0/1924356846 wait complete. 2024-04-26T20:01:53.943 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:53.943 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:53.943 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T20:01:53.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:53 smithi157 ceph-mon[32703]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T20:01:54.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T20:01:54.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:53 smithi131 ceph-mon[27919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:54.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T20:01:54.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:53 smithi148 ceph-mon[32281]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T20:01:54.274 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T20:01:54.279 INFO:tasks.cephadm:Deploying osd.3 on smithi131 with /dev/vg_nvme/lv_3... 2024-04-26T20:01:54.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T20:01:54.726 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:01:55.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:55 smithi157 ceph-mon[32703]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T20:01:55.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:55 smithi157 ceph-mon[32703]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T20:01:55.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:55 smithi157 ceph-mon[32703]: from='osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:01:55.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:55 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:55.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:55 smithi157 ceph-mon[32703]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:01:55.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:55 smithi157 ceph-mon[32703]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:55.883 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:55 smithi131 ceph-mon[27919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T20:01:55.883 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:55 smithi131 ceph-mon[27919]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T20:01:55.884 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:55 smithi131 ceph-mon[27919]: from='osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:01:55.884 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:55.884 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:55 smithi131 ceph-mon[27919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:01:55.884 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:55 smithi131 ceph-mon[27919]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:56.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:55 smithi148 ceph-mon[32281]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T20:01:56.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:55 smithi148 ceph-mon[32281]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T20:01:56.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:55 smithi148 ceph-mon[32281]: from='osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:01:56.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:55 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:56.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:55 smithi148 ceph-mon[32281]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:01:56.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:55 smithi148 ceph-mon[32281]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:56.480 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2[37867]: 2024-04-26T20:01:56.142+0000 7f4b711de700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T20:01:56.481 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2[37867]: 2024-04-26T20:01:56.148+0000 7f4b68fc9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:01:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:56 smithi131 ceph-mon[27919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:01:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:56 smithi131 ceph-mon[27919]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T20:01:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:56 smithi131 ceph-mon[27919]: from='osd.2 ' entity='osd.2' 2024-04-26T20:01:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:56.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-mon[32703]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:01:56.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-mon[32703]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T20:01:56.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:56.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:56.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-mon[32703]: from='osd.2 ' entity='osd.2' 2024-04-26T20:01:56.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:57.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:56 smithi148 ceph-mon[32281]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:01:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:56 smithi148 ceph-mon[32281]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T20:01:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:56 smithi148 ceph-mon[32281]: from='osd.2 ' entity='osd.2' 2024-04-26T20:01:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:57.616 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T20:01:57.616 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T20:01:57.616 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-04-26T20:01:57.616 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-04-26T20:01:57.616 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0353341 s, 297 MB/s 2024-04-26T20:01:57.616 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-04-26T20:01:57.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:01:57.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:01:57.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] boot 2024-04-26T20:01:57.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T20:01:57.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi157 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi157 to 17088M 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:57 smithi131 ceph-mon[27919]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:57.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:01:57.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:01:57.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] boot 2024-04-26T20:01:57.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T20:01:57.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:57.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi157 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi157 to 17088M 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:57.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:57 smithi157 ceph-mon[32703]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: osd.2 [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] boot 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi157 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:01:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi157 to 17088M 2024-04-26T20:01:58.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:58.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:58.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:58.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:01:58.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:57 smithi148 ceph-mon[32281]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T20:01:58.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-04-26T20:01:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:01:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T20:01:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:58 smithi131 ceph-mon[27919]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T20:01:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:01:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:01:58.859 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:01:58.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:01:58.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T20:01:58.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:58 smithi157 ceph-mon[32703]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T20:01:58.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:01:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:01:59.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:01:59.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T20:01:59.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:58 smithi148 ceph-mon[32281]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T20:01:59.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:01:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:02:00.586 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T20:02:00.586 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 ceph-mon[27919]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T20:02:00.586 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 ceph-mon[27919]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:00.586 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46799]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T20:02:00.586 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46799]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.587 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46799]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.587 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46799]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.587 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46814]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T20:02:00.587 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.587 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46814]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.587 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46814]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.644 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T20:02:00.644 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 ceph-mon[32703]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T20:02:00.644 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 ceph-mon[32703]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:00.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T20:02:00.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 ceph-mon[32281]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T20:02:00.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 ceph-mon[32281]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41046]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41046]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41046]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41046]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41049]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41049]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41049]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.817 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41049]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T20:02:00.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46817]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46817]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T20:02:00.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46825]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:00 smithi131 sudo[46825]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41952]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T20:02:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41952]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41952]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41952]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41955]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T20:02:00.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41955]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41955]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41946]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41946]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41949]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41949]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:00.981 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:02:00 smithi157 sudo[41949]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 -- 172.21.15.131:0/4122659370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c107260 msgr2=0x7f9b9c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 --2- 172.21.15.131:0/4122659370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c107260 0x7f9b9c10db10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400b0d0 tx=0x7f9b9400b3e0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 -- 172.21.15.131:0/4122659370 shutdown_connections 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 --2- 172.21.15.131:0/4122659370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c107260 0x7f9b9c10db10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 --2- 172.21.15.131:0/4122659370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b9c1056c0 0x7f9b9c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 --2- 172.21.15.131:0/4122659370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b9c1068c0 0x7f9b9c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 -- 172.21.15.131:0/4122659370 >> 172.21.15.131:0/4122659370 conn(0x7f9b9c100c80 msgr2=0x7f9b9c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 -- 172.21.15.131:0/4122659370 shutdown_connections 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.114+0000 7f9ba2200700 1 -- 172.21.15.131:0/4122659370 wait complete. 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.115+0000 7f9ba2200700 1 Processor -- start 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.116+0000 7f9ba2200700 1 -- start start 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.116+0000 7f9ba2200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b9c1056c0 0x7f9b9c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.116+0000 7f9ba2200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c1068c0 0x7f9b9c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.116+0000 7f9ba2200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b9c107260 0x7f9b9c1a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.116+0000 7f9ba2200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c111190 con 0x7f9b9c1068c0 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.116+0000 7f9ba2200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c111010 con 0x7f9b9c107260 2024-04-26T20:02:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.116+0000 7f9ba2200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c111310 con 0x7f9b9c1056c0 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c1068c0 0x7f9b9c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b9c1056c0 0x7f9b9c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c1068c0 0x7f9b9c19bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37546/0 (socket says 172.21.15.131:37546) 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 -- 172.21.15.131:0/3177282690 learned_addr learned my addr 172.21.15.131:0/3177282690 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9be00700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b9c107260 0x7f9b9c1a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 -- 172.21.15.131:0/3177282690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b9c1056c0 msgr2=0x7f9b9c19b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b9c1056c0 0x7f9b9c19b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 -- 172.21.15.131:0/3177282690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b9c107260 msgr2=0x7f9b9c1a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b9c107260 0x7f9b9c1a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 -- 172.21.15.131:0/3177282690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c1a33a0 con 0x7f9b9c1068c0 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9be00700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b9c107260 0x7f9b9c1a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9b400700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b9c1056c0 0x7f9b9c19b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.117+0000 7f9b9aa00700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c1068c0 0x7f9b9c19bc80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800eb60 tx=0x7f9b8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.118+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800cbd0 con 0x7f9b9c1068c0 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.118+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b880044d0 con 0x7f9b9c1068c0 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.118+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88010430 con 0x7f9b9c1068c0 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.118+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c1a3690 con 0x7f9b9c1068c0 2024-04-26T20:02:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.118+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c1a3b80 con 0x7f9b9c1068c0 2024-04-26T20:02:01.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.119+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9c06a110 con 0x7f9b9c1068c0 2024-04-26T20:02:01.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.123+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f9b88003e70 con 0x7f9b9c1068c0 2024-04-26T20:02:01.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.124+0000 7f9b93e00700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b800a3c30 0x7f9b800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.124+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f9b880cf1a0 con 0x7f9b9c1068c0 2024-04-26T20:02:01.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.124+0000 7f9b9b400700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b800a3c30 0x7f9b800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.125+0000 7f9b9b400700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b800a3c30 0x7f9b800a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c106720 tx=0x7f9b8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:01.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.126+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9b8809d020 con 0x7f9b9c1068c0 2024-04-26T20:02:01.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41052]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T20:02:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41052]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41052]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41052]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41055]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T20:02:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41055]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T20:02:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41055]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T20:02:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:00 smithi148 sudo[41055]: pam_unix(sudo:session): session closed for user root 2024-04-26T20:02:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:01.365+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9b9c063860 con 0x7f9b800a3c30 2024-04-26T20:02:01.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T20:02:01.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.732 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.732 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:01 smithi157 ceph-mon[32703]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:01 smithi131 ceph-mon[27919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T20:02:01.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:01 smithi148 ceph-mon[32281]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T20:02:02.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:02.396+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9b88098f90 con 0x7f9b9c1068c0 2024-04-26T20:02:02.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:02 smithi131 ceph-mon[27919]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:02:02.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:02.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:02.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:02.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:02 smithi131 ceph-mon[27919]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:02.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:02 smithi157 ceph-mon[32703]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:02:02.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:02.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:02.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:02.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:02 smithi157 ceph-mon[32703]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:02.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:02 smithi148 ceph-mon[32281]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:02:02.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:02.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:02.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:02.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:02 smithi148 ceph-mon[32281]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:03.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:03 smithi131 ceph-mon[27919]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T20:02:03.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:03 smithi157 ceph-mon[32703]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T20:02:03.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:03 smithi148 ceph-mon[32281]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T20:02:04.718 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:04 smithi148 ceph-mon[32281]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:04.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:04 smithi157 ceph-mon[32703]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:04.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:04 smithi131 ceph-mon[27919]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:05.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:05 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2911337335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c248bb04-986a-4b38-af3b-2e2302a4313e"}]: dispatch 2024-04-26T20:02:05.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:05 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2911337335' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c248bb04-986a-4b38-af3b-2e2302a4313e"}]': finished 2024-04-26T20:02:05.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:05 smithi157 ceph-mon[32703]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T20:02:05.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:05 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:05.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:05 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2911337335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c248bb04-986a-4b38-af3b-2e2302a4313e"}]: dispatch 2024-04-26T20:02:05.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:05 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2911337335' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c248bb04-986a-4b38-af3b-2e2302a4313e"}]': finished 2024-04-26T20:02:05.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:05 smithi131 ceph-mon[27919]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T20:02:05.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:05 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:05.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:05 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2911337335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c248bb04-986a-4b38-af3b-2e2302a4313e"}]: dispatch 2024-04-26T20:02:05.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:05 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2911337335' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c248bb04-986a-4b38-af3b-2e2302a4313e"}]': finished 2024-04-26T20:02:05.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:05 smithi148 ceph-mon[32281]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T20:02:05.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:05 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:06.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:06 smithi157 ceph-mon[32703]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:06.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:06 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2272999915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:02:06.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:06 smithi131 ceph-mon[27919]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:06.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:06 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2272999915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:02:06.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:06 smithi148 ceph-mon[32281]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:06.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:06 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2272999915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:02:08.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:08 smithi148 ceph-mon[32281]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:08.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:08 smithi157 ceph-mon[32703]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:09.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:08 smithi131 ceph-mon[27919]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:09.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:09 smithi148 ceph-mon[32281]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:09.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:09 smithi157 ceph-mon[32703]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:10.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:09 smithi131 ceph-mon[27919]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:11.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:11 smithi131 ceph-mon[27919]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:11.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:11 smithi157 ceph-mon[32703]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:12.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:11 smithi148 ceph-mon[32281]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:14.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:14 smithi148 ceph-mon[32281]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:14.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T20:02:14.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:14.849 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:14 smithi131 ceph-mon[27919]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:14.849 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T20:02:14.849 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:14.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:14 smithi157 ceph-mon[32703]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:14.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T20:02:14.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:15.621 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:15 smithi131 ceph-mon[27919]: Deploying daemon osd.3 on smithi131 2024-04-26T20:02:15.621 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:15 smithi131 ceph-mon[27919]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:15.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:15 smithi148 ceph-mon[32281]: Deploying daemon osd.3 on smithi131 2024-04-26T20:02:15.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:15 smithi148 ceph-mon[32281]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:15.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:15 smithi157 ceph-mon[32703]: Deploying daemon osd.3 on smithi131 2024-04-26T20:02:15.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:15 smithi157 ceph-mon[32703]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:18.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:18 smithi131 ceph-mon[27919]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:18.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:18 smithi148 ceph-mon[32281]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:18.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:18 smithi157 ceph-mon[32703]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:19.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:19 smithi131 ceph-mon[27919]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:19.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:19 smithi148 ceph-mon[32281]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:19.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:19 smithi157 ceph-mon[32703]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:21.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:21 smithi131 ceph-mon[27919]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:21.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:21 smithi157 ceph-mon[32703]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:22.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:21 smithi148 ceph-mon[32281]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:23 smithi157 ceph-mon[32703]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:24.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:24.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:24.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:24.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:23 smithi131 ceph-mon[27919]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:23 smithi148 ceph-mon[32281]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:25.952 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:25 smithi131 ceph-mon[27919]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:25.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:25 smithi157 ceph-mon[32703]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:25 smithi148 ceph-mon[32281]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:27.722 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.722 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.722 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:27.722 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:27.722 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:27 smithi131 ceph-mon[27919]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T20:02:27.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:27.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:27.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:27 smithi157 ceph-mon[32703]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T20:02:27.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:27.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:27.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:27.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:27 smithi148 ceph-mon[32281]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T20:02:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.822+0000 7f9b93e00700 1 -- 172.21.15.131:0/3177282690 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b9c063860 con 0x7f9b800a3c30 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 3 on host 'smithi131' 2024-04-26T20:02:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b800a3c30 msgr2=0x7f9b800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b800a3c30 0x7f9b800a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c106720 tx=0x7f9b8c009260 comp rx=0 tx=0).stop 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c1068c0 msgr2=0x7f9b9c19bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c1068c0 0x7f9b9c19bc80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800eb60 tx=0x7f9b8800ee70 comp rx=0 tx=0).stop 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 shutdown_connections 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b800a3c30 0x7f9b800a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b9c107260 0x7f9b9c1a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b9c1056c0 0x7f9b9c19b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 --2- 172.21.15.131:0/3177282690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b9c1068c0 0x7f9b9c19bc80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.826+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 >> 172.21.15.131:0/3177282690 conn(0x7f9b9c100c80 msgr2=0x7f9b9c109b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.827+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 shutdown_connections 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:02:27.827+0000 7f9ba2200700 1 -- 172.21.15.131:0/3177282690 wait complete. 2024-04-26T20:02:28.684 DEBUG:teuthology.orchestra.run.smithi131:osd.3> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T20:02:28.689 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2024-04-26T20:02:28.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:28.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:28 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:28.920 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T20:02:28.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:28.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T20:02:28.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:02:28.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:28.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:28.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:28.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:28 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:28.937 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:02:29.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T20:02:29.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:29.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T20:02:29.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:02:29.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:29.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:29.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:28 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:29.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:02:29.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T20:02:29.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' 2024-04-26T20:02:29.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] boot 2024-04-26T20:02:29.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T20:02:29.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.757 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:02:29.437+0000 7f1644c00700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T20:02:29.757 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:02:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:02:29.458+0000 7f163aa00700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:02:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:02:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T20:02:29.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' 2024-04-26T20:02:29.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] boot 2024-04-26T20:02:29.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T20:02:29.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:29 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:02:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T20:02:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:29.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: from='osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815]' entity='osd.3' 2024-04-26T20:02:29.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: osd.3 [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] boot 2024-04-26T20:02:29.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T20:02:29.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:29 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T20:02:30.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:30 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:02:30.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:30 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:02:30.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:30 smithi131 ceph-mon[27919]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:30.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:30 smithi131 ceph-mon[27919]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T20:02:30.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:30 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:02:30.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:30 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:02:30.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:30 smithi148 ceph-mon[32281]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:30.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:30 smithi148 ceph-mon[32281]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T20:02:30.914 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T20:02:30.914 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T20:02:30.914 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-26T20:02:30.914 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-26T20:02:30.914 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138464 s, 757 MB/s 2024-04-26T20:02:30.914 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-26T20:02:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:30 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:02:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:30 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:02:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:30 smithi157 ceph-mon[32703]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T20:02:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:30 smithi157 ceph-mon[32703]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T20:02:31.475 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-04-26T20:02:31.725 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:02:32.543 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:32 smithi131 ceph-mon[27919]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:32.543 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:32 smithi131 ceph-mon[27919]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T20:02:32.543 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:32 smithi131 ceph-mon[27919]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T20:02:32.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:32 smithi148 ceph-mon[32281]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:32.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:32 smithi148 ceph-mon[32281]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T20:02:32.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:32 smithi148 ceph-mon[32281]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T20:02:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:32 smithi157 ceph-mon[32703]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:32 smithi157 ceph-mon[32703]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T20:02:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:32 smithi157 ceph-mon[32703]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T20:02:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 -- 172.21.15.148:0/714794873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc104a90 msgr2=0x7f87dc104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 --2- 172.21.15.148:0/714794873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc104a90 0x7f87dc104eb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009990 tx=0x7f87cc009ca0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 -- 172.21.15.148:0/714794873 shutdown_connections 2024-04-26T20:02:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 --2- 172.21.15.148:0/714794873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87dc101eb0 0x7f87dc10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 --2- 172.21.15.148:0/714794873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc104a90 0x7f87dc104eb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 --2- 172.21.15.148:0/714794873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87dc101510 0x7f87dc101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 -- 172.21.15.148:0/714794873 >> 172.21.15.148:0/714794873 conn(0x7f87dc0fd0d0 msgr2=0x7f87dc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 -- 172.21.15.148:0/714794873 shutdown_connections 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.470+0000 7f87e1135700 1 -- 172.21.15.148:0/714794873 wait complete. 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 Processor -- start 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 -- start start 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc101510 0x7f87dc1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87dc101eb0 0x7f87dc199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87dc104a90 0x7f87dc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc10f020 con 0x7f87dc101eb0 2024-04-26T20:02:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc10eea0 con 0x7f87dc101510 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87e1135700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc10f1a0 con 0x7f87dc104a90 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.471+0000 7f87dad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc101510 0x7f87dc1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc101510 0x7f87dc1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50206/0 (socket says 172.21.15.148:50206) 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 -- 172.21.15.148:0/3780579767 learned_addr learned my addr 172.21.15.148:0/3780579767 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87db59e700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87dc104a90 0x7f87dc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 -- 172.21.15.148:0/3780579767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87dc104a90 msgr2=0x7f87dc1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87dc104a90 0x7f87dc1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 -- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87dc101eb0 msgr2=0x7f87dc199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87dc101eb0 0x7f87dc199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 -- 172.21.15.148:0/3780579767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc1a1320 con 0x7f87dc101510 2024-04-26T20:02:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87db59e700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87dc104a90 0x7f87dc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87dad9d700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc101510 0x7f87dc1996c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009fb0 tx=0x7f87cc003940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87c3fff700 1 -- 172.21.15.148:0/3780579767 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc026070 con 0x7f87dc101510 2024-04-26T20:02:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc1a15b0 con 0x7f87dc101510 2024-04-26T20:02:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87c3fff700 1 -- 172.21.15.148:0/3780579767 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87cc0047e0 con 0x7f87dc101510 2024-04-26T20:02:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87c3fff700 1 -- 172.21.15.148:0/3780579767 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0054d0 con 0x7f87dc101510 2024-04-26T20:02:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.472+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1a1a90 con 0x7f87dc101510 2024-04-26T20:02:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.473+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc06a110 con 0x7f87dc101510 2024-04-26T20:02:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.474+0000 7f87c3fff700 1 -- 172.21.15.148:0/3780579767 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f87cc029070 con 0x7f87dc101510 2024-04-26T20:02:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.475+0000 7f87c3fff700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87c40a3c40 0x7f87c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.475+0000 7f87c3fff700 1 -- 172.21.15.148:0/3780579767 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f87cc0cffd0 con 0x7f87dc101510 2024-04-26T20:02:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.475+0000 7f87da59c700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87c40a3c40 0x7f87c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.476+0000 7f87da59c700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87c40a3c40 0x7f87c40a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87d00097b0 tx=0x7f87d0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.479+0000 7f87c3fff700 1 -- 172.21.15.148:0/3780579767 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f87cc05e060 con 0x7f87dc101510 2024-04-26T20:02:33.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:33.688+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f87dc063860 con 0x7f87c40a3c40 2024-04-26T20:02:34.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:34 smithi148 ceph-mon[32281]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T20:02:34.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:34 smithi148 ceph-mon[32281]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 368 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T20:02:34.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:34.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:34.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:34.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:34 smithi131 ceph-mon[27919]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T20:02:34.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:34 smithi131 ceph-mon[27919]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 368 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T20:02:34.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:34.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:34.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:34.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:34 smithi157 ceph-mon[32703]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T20:02:34.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:34 smithi157 ceph-mon[32703]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 368 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T20:02:34.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:34.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:34.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:35.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:02:35.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:35.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:35.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:35.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:35.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:35 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:35.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:35 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:02:35.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:35.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:35.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:35.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:35.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:35.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:35 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:36.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:36 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi131 2024-04-26T20:02:36.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:36 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi131 to 8544M 2024-04-26T20:02:36.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:36 smithi131 ceph-mon[27919]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T20:02:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:36 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi131 2024-04-26T20:02:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:36 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi131 to 8544M 2024-04-26T20:02:36.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:36 smithi148 ceph-mon[32281]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T20:02:36.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:36 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi131 2024-04-26T20:02:36.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:36 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi131 to 8544M 2024-04-26T20:02:36.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:36 smithi157 ceph-mon[32703]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T20:02:37.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:37 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/127269683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]: dispatch 2024-04-26T20:02:37.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:37 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]: dispatch 2024-04-26T20:02:37.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:37 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]': finished 2024-04-26T20:02:37.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:37 smithi131 ceph-mon[27919]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T20:02:37.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:37 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:37.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:37 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/1107759527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:02:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:37 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/127269683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]: dispatch 2024-04-26T20:02:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:37 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]: dispatch 2024-04-26T20:02:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:37 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]': finished 2024-04-26T20:02:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:37 smithi148 ceph-mon[32281]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T20:02:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:37 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:37.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:37 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/1107759527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:02:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:37 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/127269683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]: dispatch 2024-04-26T20:02:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:37 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]: dispatch 2024-04-26T20:02:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:37 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de2985f5-6971-41c7-aee0-1d437e281514"}]': finished 2024-04-26T20:02:37.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:37 smithi157 ceph-mon[32703]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T20:02:37.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:37 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:37.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:37 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/1107759527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:02:38.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:38 smithi148 ceph-mon[32281]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T20:02:38.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:38 smithi157 ceph-mon[32703]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T20:02:38.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:38 smithi131 ceph-mon[27919]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T20:02:39.959 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:39 smithi148 ceph-mon[32281]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T20:02:39.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:39 smithi157 ceph-mon[32703]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T20:02:40.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:39 smithi131 ceph-mon[27919]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T20:02:41.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:41 smithi157 ceph-mon[32703]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:41 smithi131 ceph-mon[27919]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:42.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:41 smithi148 ceph-mon[32281]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:43.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:43 smithi148 ceph-mon[32281]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:43.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:43 smithi157 ceph-mon[32703]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:44.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:43 smithi131 ceph-mon[27919]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:44.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T20:02:44.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:44.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:44 smithi148 ceph-mon[32281]: Deploying daemon osd.4 on smithi148 2024-04-26T20:02:44.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T20:02:44.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:44.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:44 smithi157 ceph-mon[32703]: Deploying daemon osd.4 on smithi148 2024-04-26T20:02:45.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T20:02:45.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:45.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:44 smithi131 ceph-mon[27919]: Deploying daemon osd.4 on smithi148 2024-04-26T20:02:45.967 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:45 smithi148 ceph-mon[32281]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:45.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:45 smithi157 ceph-mon[32703]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:46.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:45 smithi131 ceph-mon[27919]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:47.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:47 smithi157 ceph-mon[32703]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:48.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:47 smithi131 ceph-mon[27919]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:48.055 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:47 smithi148 ceph-mon[32281]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:50 smithi157 ceph-mon[32703]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:50 smithi131 ceph-mon[27919]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:50.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:50.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:50 smithi148 ceph-mon[32281]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:51.866 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:51 smithi148 ceph-mon[32281]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:51.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:51 smithi157 ceph-mon[32703]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:52.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:51 smithi131 ceph-mon[27919]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.965+0000 7f87c3fff700 1 -- 172.21.15.148:0/3780579767 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87dc063860 con 0x7f87c40a3c40 2024-04-26T20:02:52.967 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87c40a3c40 msgr2=0x7f87c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87c40a3c40 0x7f87c40a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87d00097b0 tx=0x7f87d0006d20 comp rx=0 tx=0).stop 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc101510 msgr2=0x7f87dc1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc101510 0x7f87dc1996c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009fb0 tx=0x7f87cc003940 comp rx=0 tx=0).stop 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 shutdown_connections 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87dc104a90 0x7f87dc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87c40a3c40 0x7f87c40a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc101510 0x7f87dc1996c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 --2- 172.21.15.148:0/3780579767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87dc101eb0 0x7f87dc199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:52.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 >> 172.21.15.148:0/3780579767 conn(0x7f87dc0fd0d0 msgr2=0x7f87dc107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:52.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 shutdown_connections 2024-04-26T20:02:52.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:02:52.968+0000 7f87e1135700 1 -- 172.21.15.148:0/3780579767 wait complete. 2024-04-26T20:02:53.575 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T20:02:53.577 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-04-26T20:02:53.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T20:02:53.825 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:02:54.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:53 smithi157 ceph-mon[32703]: from='osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T20:02:54.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:53 smithi157 ceph-mon[32703]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T20:02:54.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:53 smithi157 ceph-mon[32703]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:54.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:53 smithi131 ceph-mon[27919]: from='osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T20:02:54.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:53 smithi131 ceph-mon[27919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T20:02:54.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:53 smithi131 ceph-mon[27919]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:54.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:53 smithi148 ceph-mon[32281]: from='osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T20:02:54.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:53 smithi148 ceph-mon[32281]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T20:02:54.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:53 smithi148 ceph-mon[32281]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:54.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:54.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:55.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:54 smithi157 ceph-mon[32703]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T20:02:55.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:54 smithi157 ceph-mon[32703]: from='osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:02:55.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:54 smithi157 ceph-mon[32703]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T20:02:55.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:55.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:54 smithi157 ceph-mon[32703]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:02:55.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:54 smithi131 ceph-mon[27919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T20:02:55.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:54 smithi131 ceph-mon[27919]: from='osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:02:55.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:54 smithi131 ceph-mon[27919]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T20:02:55.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:55.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:54 smithi131 ceph-mon[27919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:02:55.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:54 smithi148 ceph-mon[32281]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T20:02:55.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:54 smithi148 ceph-mon[32281]: from='osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:02:55.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:54 smithi148 ceph-mon[32281]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T20:02:55.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:55.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:54 smithi148 ceph-mon[32281]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:02:55.318 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:02:54 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:02:54.971+0000 7fa599286700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T20:02:55.318 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:02:54 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:02:54.982+0000 7fa595079700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:02:55.892 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T20:02:55.892 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T20:02:55.892 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-26T20:02:55.892 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-26T20:02:55.892 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00863137 s, 1.2 GB/s 2024-04-26T20:02:55.892 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-26T20:02:56.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:55 smithi131 ceph-mon[27919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:02:56.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:55 smithi131 ceph-mon[27919]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T20:02:56.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:56.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:56.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:55 smithi131 ceph-mon[27919]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:56.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:55 smithi157 ceph-mon[32703]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:02:56.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:55 smithi157 ceph-mon[32703]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T20:02:56.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:55 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:56.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:55 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:56.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:55 smithi157 ceph-mon[32703]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:56.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:55 smithi148 ceph-mon[32281]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:02:56.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:55 smithi148 ceph-mon[32281]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T20:02:56.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:55 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:56.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:55 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:56.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:55 smithi148 ceph-mon[32281]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T20:02:56.596 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-04-26T20:02:56.845 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:02:57.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:56 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:02:57.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:56 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:02:57.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:57.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:56 smithi157 ceph-mon[32703]: osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] boot 2024-04-26T20:02:57.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:56 smithi157 ceph-mon[32703]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T20:02:57.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:57.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:56 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:02:57.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:56 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:02:57.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:57.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:56 smithi131 ceph-mon[27919]: osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] boot 2024-04-26T20:02:57.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:56 smithi131 ceph-mon[27919]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T20:02:57.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:57.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:56 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:02:57.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:56 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:02:57.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:57.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:56 smithi148 ceph-mon[32281]: osd.4 [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] boot 2024-04-26T20:02:57.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:56 smithi148 ceph-mon[32281]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T20:02:57.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T20:02:58.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T20:02:58.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi148 2024-04-26T20:02:58.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:58.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:57 smithi131 ceph-mon[27919]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:57 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi148 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:02:58.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:58.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:58.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:58.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi148 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:58.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:58.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:02:58.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:58.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:58.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:58.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:02:58.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- 172.21.15.157:0/2835002883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 msgr2=0x7f60b8103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 --2- 172.21.15.157:0/2835002883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8103890 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009990 tx=0x7f60b4009ca0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- 172.21.15.157:0/2835002883 shutdown_connections 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 --2- 172.21.15.157:0/2835002883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8105010 0x7f60b810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 --2- 172.21.15.157:0/2835002883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8103890 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 --2- 172.21.15.157:0/2835002883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b8104670 0x7f60b8104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- 172.21.15.157:0/2835002883 >> 172.21.15.157:0/2835002883 conn(0x7f60b80fea10 msgr2=0x7f60b8100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- 172.21.15.157:0/2835002883 shutdown_connections 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- 172.21.15.157:0/2835002883 wait complete. 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 Processor -- start 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- start start 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:58.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8104670 0x7f60b8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b8105010 0x7f60b81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810ef60 con 0x7f60b8105010 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810ede0 con 0x7f60b8104670 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.514+0000 7f60bf28c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810f0e0 con 0x7f60b8103470 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50112/0 (socket says 172.21.15.157:50112) 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 -- 172.21.15.157:0/546724815 learned_addr learned my addr 172.21.15.157:0/546724815 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bc827700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8104670 0x7f60b8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 -- 172.21.15.157:0/546724815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8104670 msgr2=0x7f60b8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8104670 0x7f60b8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 -- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b8105010 msgr2=0x7f60b81a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b8105010 0x7f60b81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 -- 172.21.15.157:0/546724815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b81a12a0 con 0x7f60b8103470 2024-04-26T20:02:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bd028700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8199690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f60b40047b0 tx=0x7f60b4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60ae7fc700 1 -- 172.21.15.157:0/546724815 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4026070 con 0x7f60b8103470 2024-04-26T20:02:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b81a1530 con 0x7f60b8103470 2024-04-26T20:02:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60ae7fc700 1 -- 172.21.15.157:0/546724815 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b4004140 con 0x7f60b8103470 2024-04-26T20:02:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60ae7fc700 1 -- 172.21.15.157:0/546724815 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4005290 con 0x7f60b8103470 2024-04-26T20:02:58.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.515+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b81a19e0 con 0x7f60b8103470 2024-04-26T20:02:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.516+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b806a110 con 0x7f60b8103470 2024-04-26T20:02:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.517+0000 7f60ae7fc700 1 -- 172.21.15.157:0/546724815 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f60b4029070 con 0x7f60b8103470 2024-04-26T20:02:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.518+0000 7f60ae7fc700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60a40a3c40 0x7f60a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.518+0000 7f60ae7fc700 1 -- 172.21.15.157:0/546724815 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f60b40cfe40 con 0x7f60b8103470 2024-04-26T20:02:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.519+0000 7f60bc827700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60a40a3c40 0x7f60a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:58.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.519+0000 7f60bc827700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60a40a3c40 0x7f60a40a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009a10 tx=0x7f60a8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:58.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.523+0000 7f60ae7fc700 1 -- 172.21.15.157:0/546724815 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f60b405e060 con 0x7f60b8103470 2024-04-26T20:02:58.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:02:58.730+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f60b8063860 con 0x7f60a40a3c40 2024-04-26T20:02:59.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:59 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:59.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:59 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:59.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:02:59 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:02:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:59.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:59 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:02:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:59 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:02:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:02:59 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:00.694 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:00 smithi157 ceph-mon[32703]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:03:00.694 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:00 smithi157 ceph-mon[32703]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:00 smithi131 ceph-mon[27919]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:03:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:00 smithi131 ceph-mon[27919]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:00.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:00 smithi148 ceph-mon[32281]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:03:00.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:00 smithi148 ceph-mon[32281]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:01.922 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:01 smithi157 ceph-mon[32703]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:01.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:01 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/173274068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]: dispatch 2024-04-26T20:03:01.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:01 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]: dispatch 2024-04-26T20:03:01.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:01 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]': finished 2024-04-26T20:03:01.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:01 smithi157 ceph-mon[32703]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T20:03:01.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:01 smithi131 ceph-mon[27919]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:01 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/173274068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]: dispatch 2024-04-26T20:03:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:01 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]: dispatch 2024-04-26T20:03:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:01 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]': finished 2024-04-26T20:03:02.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:01 smithi131 ceph-mon[27919]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T20:03:02.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:02.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:01 smithi148 ceph-mon[32281]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:02.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:01 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/173274068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]: dispatch 2024-04-26T20:03:02.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:01 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]: dispatch 2024-04-26T20:03:02.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:01 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa8194d9-9332-4a74-abe4-27a006a4f40a"}]': finished 2024-04-26T20:03:02.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:01 smithi148 ceph-mon[32281]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T20:03:02.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:02.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:02 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/560514808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:03:03.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:02 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/560514808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:03:03.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:02 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/560514808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:03:03.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:03 smithi157 ceph-mon[32703]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:04.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:03 smithi131 ceph-mon[27919]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:04.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:03 smithi148 ceph-mon[32281]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:05.861 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:05 smithi157 ceph-mon[32703]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:05.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:05 smithi148 ceph-mon[32281]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:06.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:05 smithi131 ceph-mon[27919]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:07.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:07 smithi157 ceph-mon[32703]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:08.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:07 smithi131 ceph-mon[27919]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:08.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:07 smithi148 ceph-mon[32281]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:09.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:09 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T20:03:09.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:09 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:09.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:09 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T20:03:09.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:09 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:09.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:09 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T20:03:09.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:09 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:10.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:10 smithi157 ceph-mon[32703]: Deploying daemon osd.5 on smithi157 2024-04-26T20:03:10.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:10 smithi157 ceph-mon[32703]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:10.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:10 smithi131 ceph-mon[27919]: Deploying daemon osd.5 on smithi157 2024-04-26T20:03:10.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:10 smithi131 ceph-mon[27919]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:10.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:10 smithi148 ceph-mon[32281]: Deploying daemon osd.5 on smithi157 2024-04-26T20:03:10.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:10 smithi148 ceph-mon[32281]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:11.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:11 smithi157 ceph-mon[32703]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:12.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:11 smithi131 ceph-mon[27919]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:12.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:11 smithi148 ceph-mon[32281]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:13.930 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:13 smithi157 ceph-mon[32703]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:14.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:13 smithi131 ceph-mon[27919]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:14.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:13 smithi148 ceph-mon[32281]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:15 smithi157 ceph-mon[32703]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:16.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:16.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:16.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:16.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:15 smithi131 ceph-mon[27919]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:16.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:16.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:16.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:16.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:15 smithi148 ceph-mon[32281]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:17.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:17 smithi157 ceph-mon[32703]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:18.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:17 smithi131 ceph-mon[27919]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:18.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:17 smithi148 ceph-mon[32281]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:18.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.520+0000 7f60ae7fc700 1 -- 172.21.15.157:0/546724815 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60b8063860 con 0x7f60a40a3c40 2024-04-26T20:03:18.523 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-04-26T20:03:18.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60a40a3c40 msgr2=0x7f60a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:03:18.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60a40a3c40 0x7f60a40a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009a10 tx=0x7f60a8009190 comp rx=0 tx=0).stop 2024-04-26T20:03:18.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 msgr2=0x7f60b8199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:03:18.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8199690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f60b40047b0 tx=0x7f60b4009580 comp rx=0 tx=0).stop 2024-04-26T20:03:18.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 shutdown_connections 2024-04-26T20:03:18.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b8105010 0x7f60b81a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:18.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60b8103470 0x7f60b8199690 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:18.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60a40a3c40 0x7f60a40a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:18.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 --2- 172.21.15.157:0/546724815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8104670 0x7f60b8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:18.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 >> 172.21.15.157:0/546724815 conn(0x7f60b80fea10 msgr2=0x7f60b8107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:03:18.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 shutdown_connections 2024-04-26T20:03:18.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:03:18.523+0000 7f60bf28c700 1 -- 172.21.15.157:0/546724815 wait complete. 2024-04-26T20:03:19.231 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T20:03:19.234 INFO:tasks.cephadm:Deploying osd.6 on smithi131 with /dev/vg_nvme/lv_2... 2024-04-26T20:03:19.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T20:03:19.676 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:03:19.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:19 smithi131 ceph-mon[27919]: from='osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T20:03:19.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:19 smithi131 ceph-mon[27919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T20:03:19.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:19 smithi157 ceph-mon[32703]: from='osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T20:03:19.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:19 smithi157 ceph-mon[32703]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T20:03:19.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:19.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:19 smithi148 ceph-mon[32281]: from='osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T20:03:19.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:19 smithi148 ceph-mon[32281]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T20:03:20.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-mon[32703]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:20.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-mon[32703]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T20:03:20.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-mon[32703]: from='osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:03:20.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-mon[32703]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T20:03:20.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:20.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-mon[32703]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:03:20.719 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:03:20.602+0000 7fda64c76700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T20:03:20.719 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:03:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:03:20.608+0000 7fda60a69700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:03:20.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:20 smithi131 ceph-mon[27919]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:20.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:20 smithi131 ceph-mon[27919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T20:03:20.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:20 smithi131 ceph-mon[27919]: from='osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:03:20.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:20 smithi131 ceph-mon[27919]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T20:03:20.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:20.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:20 smithi131 ceph-mon[27919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:03:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:20 smithi148 ceph-mon[32281]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:20 smithi148 ceph-mon[32281]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T20:03:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:20 smithi148 ceph-mon[32281]: from='osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:03:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:20 smithi148 ceph-mon[32281]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T20:03:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:20.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:20 smithi148 ceph-mon[32281]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:03:21.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:21 smithi157 ceph-mon[32703]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:03:21.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:21 smithi157 ceph-mon[32703]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T20:03:21.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:21.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:21.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:21 smithi157 ceph-mon[32703]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:21.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:21 smithi131 ceph-mon[27919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:03:21.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:21 smithi131 ceph-mon[27919]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T20:03:21.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:21.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:21.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:21 smithi131 ceph-mon[27919]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:22.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:21 smithi148 ceph-mon[32281]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:03:22.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:21 smithi148 ceph-mon[32281]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T20:03:22.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:22.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:22.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:21 smithi148 ceph-mon[32281]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T20:03:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:03:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:03:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T20:03:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T20:03:22.670 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-04-26T20:03:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-04-26T20:03:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0563128 s, 186 MB/s 2024-04-26T20:03:22.670 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 2024-04-26T20:03:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-04-26T20:03:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:03:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] boot 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi157 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi157 to 8544M 2024-04-26T20:03:22.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:22.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:22.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:22.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:22.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:22.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.994 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] boot 2024-04-26T20:03:22.994 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T20:03:22.994 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:22.994 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T20:03:22.995 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi157 2024-04-26T20:03:22.995 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.995 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:03:22.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:03:22.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi157 to 8544M 2024-04-26T20:03:22.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:22.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:22.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:22.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:22.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:22.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:22.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: osd.5 [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] boot 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi157 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi157 to 8544M 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:23.637 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-04-26T20:03:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:03:24.441 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:24 smithi131 ceph-mon[27919]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T20:03:24.441 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:24 smithi131 ceph-mon[27919]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:24.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:24 smithi148 ceph-mon[32281]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T20:03:24.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:24 smithi148 ceph-mon[32281]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:24.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:24 smithi157 ceph-mon[32703]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T20:03:24.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:24 smithi157 ceph-mon[32703]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:25.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:25 smithi131 ceph-mon[27919]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T20:03:25.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:25 smithi148 ceph-mon[32281]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T20:03:25.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:25 smithi157 ceph-mon[32703]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T20:03:26.436 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:26 smithi131 ceph-mon[27919]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T20:03:26.436 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:26 smithi131 ceph-mon[27919]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.449+0000 7f708d000700 1 -- 172.21.15.131:0/2688484254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881056d0 msgr2=0x7f7088105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.449+0000 7f708d000700 1 --2- 172.21.15.131:0/2688484254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881056d0 0x7f7088105af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7078009920 tx=0x7f7078009c30 comp rx=0 tx=0).stop 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.449+0000 7f708d000700 1 -- 172.21.15.131:0/2688484254 shutdown_connections 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.449+0000 7f708d000700 1 --2- 172.21.15.131:0/2688484254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7088107270 0x7f708810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.449+0000 7f708d000700 1 --2- 172.21.15.131:0/2688484254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881056d0 0x7f7088105af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.449+0000 7f708d000700 1 --2- 172.21.15.131:0/2688484254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70881068d0 0x7f7088106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.449+0000 7f708d000700 1 -- 172.21.15.131:0/2688484254 >> 172.21.15.131:0/2688484254 conn(0x7f7088100c70 msgr2=0x7f70881030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.450+0000 7f708d000700 1 -- 172.21.15.131:0/2688484254 shutdown_connections 2024-04-26T20:03:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.450+0000 7f708d000700 1 -- 172.21.15.131:0/2688484254 wait complete. 2024-04-26T20:03:26.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 Processor -- start 2024-04-26T20:03:26.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 -- start start 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70881056d0 0x7f708819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881068d0 0x7f708819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7088107270 0x7f70881a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f70881111c0 con 0x7f7088107270 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7088111040 con 0x7f70881068d0 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.451+0000 7f708d000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7088111340 con 0x7f70881056d0 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881068d0 0x7f708819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881068d0 0x7f708819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:52974/0 (socket says 172.21.15.131:52974) 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 -- 172.21.15.131:0/1344032101 learned_addr learned my addr 172.21.15.131:0/1344032101 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085e00700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70881056d0 0x7f708819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:03:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7086800700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7088107270 0x7f70881a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:03:26.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 -- 172.21.15.131:0/1344032101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70881056d0 msgr2=0x7f708819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:03:26.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70881056d0 0x7f708819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:26.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 -- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7088107270 msgr2=0x7f70881a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7088107270 0x7f70881a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085400700 1 -- 172.21.15.131:0/1344032101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70881a34f0 con 0x7f70881068d0 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.452+0000 7f7085e00700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70881056d0 0x7f708819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.453+0000 7f7085400700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881068d0 0x7f708819bdd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f707400ec10 tx=0x7f707400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.453+0000 7f707ea00700 1 -- 172.21.15.131:0/1344032101 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707400cc90 con 0x7f70881068d0 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.453+0000 7f707ea00700 1 -- 172.21.15.131:0/1344032101 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707400ce30 con 0x7f70881068d0 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.453+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70881a37e0 con 0x7f70881068d0 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.453+0000 7f707ea00700 1 -- 172.21.15.131:0/1344032101 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074010430 con 0x7f70881068d0 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.453+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70881a3d70 con 0x7f70881068d0 2024-04-26T20:03:26.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.454+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708806a110 con 0x7f70881068d0 2024-04-26T20:03:26.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.457+0000 7f707ea00700 1 -- 172.21.15.131:0/1344032101 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7074003fe0 con 0x7f70881068d0 2024-04-26T20:03:26.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.458+0000 7f707ea00700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f706c0a3cc0 0x7f706c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:03:26.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.458+0000 7f707ea00700 1 -- 172.21.15.131:0/1344032101 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f70740cf820 con 0x7f70881068d0 2024-04-26T20:03:26.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.458+0000 7f7085e00700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f706c0a3cc0 0x7f706c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:03:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.459+0000 7f7085e00700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f706c0a3cc0 0x7f706c0a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f70780098f0 tx=0x7f7078019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:03:26.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.463+0000 7f707ea00700 1 -- 172.21.15.131:0/1344032101 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f707409d020 con 0x7f70881068d0 2024-04-26T20:03:26.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:26 smithi148 ceph-mon[32281]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T20:03:26.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:26 smithi148 ceph-mon[32281]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:26.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:26.698+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7088063860 con 0x7f706c0a3cc0 2024-04-26T20:03:26.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:26 smithi157 ceph-mon[32703]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T20:03:26.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:26 smithi157 ceph-mon[32703]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:27.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:03:27.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:03:27.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:27.631 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:03:27.631 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:03:27.631 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:27.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:03:27.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:03:27.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:28.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:28 smithi131 ceph-mon[27919]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:03:28.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:28 smithi131 ceph-mon[27919]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:28.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:28 smithi148 ceph-mon[32281]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:03:28.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:28 smithi148 ceph-mon[32281]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:28.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:28 smithi157 ceph-mon[32703]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:03:28.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:28 smithi157 ceph-mon[32703]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:29 smithi157 ceph-mon[32703]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T20:03:30.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:29 smithi131 ceph-mon[27919]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T20:03:30.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:29 smithi148 ceph-mon[32281]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T20:03:31.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:30 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1372599403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]: dispatch 2024-04-26T20:03:31.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:30 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]: dispatch 2024-04-26T20:03:31.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:30 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]': finished 2024-04-26T20:03:31.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:30 smithi157 ceph-mon[32703]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T20:03:31.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:31.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:30 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1372599403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]: dispatch 2024-04-26T20:03:31.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:30 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]: dispatch 2024-04-26T20:03:31.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:30 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]': finished 2024-04-26T20:03:31.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:30 smithi131 ceph-mon[27919]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T20:03:31.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:31.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:30 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1372599403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]: dispatch 2024-04-26T20:03:31.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:30 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]: dispatch 2024-04-26T20:03:31.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:30 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4db9d2b9-02f6-4d5f-a40f-afbafa778151"}]': finished 2024-04-26T20:03:31.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:30 smithi148 ceph-mon[32281]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T20:03:31.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:32.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:31 smithi157 ceph-mon[32703]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T20:03:32.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:31 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/331915507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:03:32.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:31 smithi131 ceph-mon[27919]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T20:03:32.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:31 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/331915507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:03:32.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:31 smithi148 ceph-mon[32281]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T20:03:32.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:31 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/331915507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:03:33.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:33 smithi131 ceph-mon[27919]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T20:03:33.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:33 smithi157 ceph-mon[32703]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T20:03:34.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:33 smithi148 ceph-mon[32281]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T20:03:35.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:35 smithi148 ceph-mon[32281]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:03:35.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:35 smithi157 ceph-mon[32703]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:03:36.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:35 smithi131 ceph-mon[27919]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:03:37.722 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:37 smithi131 ceph-mon[27919]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:03:37.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:37 smithi157 ceph-mon[32703]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:03:38.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:37 smithi148 ceph-mon[32281]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:03:39.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:39 smithi157 ceph-mon[32703]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:39.984 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:39 smithi131 ceph-mon[27919]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:40.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:39 smithi148 ceph-mon[32281]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T20:03:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:41.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T20:03:41.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T20:03:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:41 smithi131 ceph-mon[27919]: Deploying daemon osd.6 on smithi131 2024-04-26T20:03:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:41 smithi131 ceph-mon[27919]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:42.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:41 smithi148 ceph-mon[32281]: Deploying daemon osd.6 on smithi131 2024-04-26T20:03:42.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:41 smithi148 ceph-mon[32281]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:41 smithi157 ceph-mon[32703]: Deploying daemon osd.6 on smithi131 2024-04-26T20:03:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:41 smithi157 ceph-mon[32703]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:43.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:43 smithi131 ceph-mon[27919]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:43.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:43 smithi157 ceph-mon[32703]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:44.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:43 smithi148 ceph-mon[32281]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:45.905 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:45 smithi131 ceph-mon[27919]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:45.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:45 smithi157 ceph-mon[32703]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:46.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:45 smithi148 ceph-mon[32281]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:47.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:47 smithi157 ceph-mon[32703]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:48.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:47 smithi131 ceph-mon[27919]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:48.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:47 smithi148 ceph-mon[32281]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:49.934 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:49 smithi131 ceph-mon[27919]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:49.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:49 smithi157 ceph-mon[32703]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:50.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:49 smithi148 ceph-mon[32281]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:50.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:50.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:50.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:50.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:50.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:50.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:51.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:51.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:51.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:51.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:51 smithi131 ceph-mon[27919]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:51.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:51 smithi157 ceph-mon[32703]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:52.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:51 smithi148 ceph-mon[32281]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:54.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:54 smithi148 ceph-mon[32281]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:54.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:54.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:54.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:54 smithi148 ceph-mon[32281]: from='osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T20:03:54.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:54 smithi148 ceph-mon[32281]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T20:03:54.818 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 6 on host 'smithi131' 2024-04-26T20:03:54.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.816+0000 7f707ea00700 1 -- 172.21.15.131:0/1344032101 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7088063860 con 0x7f706c0a3cc0 2024-04-26T20:03:54.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:54 smithi131 ceph-mon[27919]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:54.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:54.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:54.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:54 smithi131 ceph-mon[27919]: from='osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T20:03:54.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:54 smithi131 ceph-mon[27919]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T20:03:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.819+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f706c0a3cc0 msgr2=0x7f706c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:03:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f706c0a3cc0 0x7f706c0a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f70780098f0 tx=0x7f7078019040 comp rx=0 tx=0).stop 2024-04-26T20:03:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881068d0 msgr2=0x7f708819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:03:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881068d0 0x7f708819bdd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f707400ec10 tx=0x7f707400ef20 comp rx=0 tx=0).stop 2024-04-26T20:03:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 shutdown_connections 2024-04-26T20:03:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7088107270 0x7f70881a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70881056d0 0x7f708819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f706c0a3cc0 0x7f706c0a6180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 --2- 172.21.15.131:0/1344032101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70881068d0 0x7f708819bdd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:03:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 >> 172.21.15.131:0/1344032101 conn(0x7f7088100c70 msgr2=0x7f7088109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:03:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 shutdown_connections 2024-04-26T20:03:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:03:54.820+0000 7f708d000700 1 -- 172.21.15.131:0/1344032101 wait complete. 2024-04-26T20:03:54.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:54 smithi157 ceph-mon[32703]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:54.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:54.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:54.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:54 smithi157 ceph-mon[32703]: from='osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T20:03:54.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:54 smithi157 ceph-mon[32703]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: from='osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:55.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:55 smithi131 ceph-mon[27919]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:55.930 DEBUG:teuthology.orchestra.run.smithi131:osd.6> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T20:03:55.938 INFO:tasks.cephadm:Deploying osd.7 on smithi148 with /dev/vg_nvme/lv_2... 2024-04-26T20:03:55.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: from='osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:55.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:55 smithi148 ceph-mon[32281]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: from='osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:03:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:55 smithi157 ceph-mon[32703]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:56.185 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:03:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:56 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:03:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:56 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:03:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:56 smithi131 ceph-mon[27919]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:03:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:56 smithi131 ceph-mon[27919]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T20:03:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:56.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:56 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:03:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:56 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:03:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:56 smithi157 ceph-mon[32703]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:03:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:56 smithi157 ceph-mon[32703]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T20:03:56.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:56.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:56 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:03:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:56 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:03:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:56 smithi148 ceph-mon[32281]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:03:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:56 smithi148 ceph-mon[32281]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T20:03:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:57.881 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:57 smithi157 ceph-mon[32703]: osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] boot 2024-04-26T20:03:57.882 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:57 smithi157 ceph-mon[32703]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T20:03:57.882 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:57.882 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:57 smithi157 ceph-mon[32703]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:57.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:57 smithi131 ceph-mon[27919]: osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] boot 2024-04-26T20:03:57.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:57 smithi131 ceph-mon[27919]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T20:03:57.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:57.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:57 smithi131 ceph-mon[27919]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:57 smithi148 ceph-mon[32281]: osd.6 [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] boot 2024-04-26T20:03:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:57 smithi148 ceph-mon[32281]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T20:03:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T20:03:58.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:57 smithi148 ceph-mon[32281]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T20:03:58.205 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T20:03:58.205 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T20:03:58.205 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-26T20:03:58.205 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-26T20:03:58.205 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112938 s, 928 MB/s 2024-04-26T20:03:58.205 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-26T20:03:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:58 smithi131 ceph-mon[27919]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T20:03:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:58 smithi148 ceph-mon[32281]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T20:03:58.871 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-04-26T20:03:58.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:58 smithi157 ceph-mon[32703]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T20:03:59.150 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:03:59.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:03:59 smithi148 ceph-mon[32281]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:03:59.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:03:59 smithi157 ceph-mon[32703]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:00.005 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:03:59 smithi131 ceph-mon[27919]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:00.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 -- 172.21.15.148:0/2047435291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 msgr2=0x7f1bbc1063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 --2- 172.21.15.148:0/2047435291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc1063c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009990 tx=0x7f1bac009ca0 comp rx=0 tx=0).stop 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 -- 172.21.15.148:0/2047435291 shutdown_connections 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 --2- 172.21.15.148:0/2047435291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc109230 0x7f1bbc10b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 --2- 172.21.15.148:0/2047435291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc1063c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 --2- 172.21.15.148:0/2047435291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc106900 0x7f1bbc108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 -- 172.21.15.148:0/2047435291 >> 172.21.15.148:0/2047435291 conn(0x7f1bbc0fd920 msgr2=0x7f1bbc0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 -- 172.21.15.148:0/2047435291 shutdown_connections 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 -- 172.21.15.148:0/2047435291 wait complete. 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.782+0000 7f1bc1656700 1 Processor -- start 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 -- start start 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:00.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc106900 0x7f1bbc199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc109230 0x7f1bbc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc10efc0 con 0x7f1bbc109230 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc10ee40 con 0x7f1bbc103fd0 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc10f140 con 0x7f1bbc106900 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46492/0 (socket says 172.21.15.148:46492) 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 -- 172.21.15.148:0/324403399 learned_addr learned my addr 172.21.15.148:0/324403399 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bba7fc700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc106900 0x7f1bbc199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 -- 172.21.15.148:0/324403399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc106900 msgr2=0x7f1bbc199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc106900 0x7f1bbc199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 -- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc109230 msgr2=0x7f1bbc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc109230 0x7f1bbc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 -- 172.21.15.148:0/324403399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc1a1290 con 0x7f1bbc103fd0 2024-04-26T20:04:00.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bba7fc700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc106900 0x7f1bbc199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:04:00.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bbaffd700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc199630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0036f0 tx=0x7f1bac005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:04:00.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1ba3fff700 1 -- 172.21.15.148:0/324403399 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac027070 con 0x7f1bbc103fd0 2024-04-26T20:04:00.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1ba3fff700 1 -- 172.21.15.148:0/324403399 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bac004100 con 0x7f1bbc103fd0 2024-04-26T20:04:00.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.783+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc1a1520 con 0x7f1bbc103fd0 2024-04-26T20:04:00.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.784+0000 7f1ba3fff700 1 -- 172.21.15.148:0/324403399 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac022e30 con 0x7f1bbc103fd0 2024-04-26T20:04:00.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.784+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc1a19f0 con 0x7f1bbc103fd0 2024-04-26T20:04:00.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.784+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc06a110 con 0x7f1bbc103fd0 2024-04-26T20:04:00.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.785+0000 7f1ba3fff700 1 -- 172.21.15.148:0/324403399 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1bac02a070 con 0x7f1bbc103fd0 2024-04-26T20:04:00.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.786+0000 7f1ba3fff700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ba40a3c40 0x7f1ba40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:00.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.786+0000 7f1ba3fff700 1 -- 172.21.15.148:0/324403399 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f1bac0d04e0 con 0x7f1bbc103fd0 2024-04-26T20:04:00.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.787+0000 7f1bba7fc700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ba40a3c40 0x7f1ba40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:00.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.788+0000 7f1bba7fc700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ba40a3c40 0x7f1ba40a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000bcf0 tx=0x7f1bb000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:04:00.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.791+0000 7f1ba3fff700 1 -- 172.21.15.148:0/324403399 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1bac013070 con 0x7f1bbc103fd0 2024-04-26T20:04:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:00.995+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1bbc063860 con 0x7f1ba40a3c40 2024-04-26T20:04:01.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:01.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:01.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:01.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:01.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:01.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:01.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi131 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi131 to 5696M 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:02.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:02 smithi148 ceph-mon[32281]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:02.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:02.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi131 2024-04-26T20:04:02.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi131 to 5696M 2024-04-26T20:04:02.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:02.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:02.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:02.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:02.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:02.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:02.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:02 smithi157 ceph-mon[32703]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:02.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi131 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi131 to 5696M 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:02.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:02 smithi131 ceph-mon[27919]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:03.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:03 smithi157 ceph-mon[32703]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:04.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:03 smithi131 ceph-mon[27919]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:04.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:03 smithi148 ceph-mon[32281]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:04.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:04 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/797644910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]: dispatch 2024-04-26T20:04:04.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:04 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]: dispatch 2024-04-26T20:04:04.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:04 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]': finished 2024-04-26T20:04:04.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:04 smithi157 ceph-mon[32703]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T20:04:04.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:04.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:04 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/1146862970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:04:05.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:04 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/797644910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]: dispatch 2024-04-26T20:04:05.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:04 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]: dispatch 2024-04-26T20:04:05.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:04 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]': finished 2024-04-26T20:04:05.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:04 smithi131 ceph-mon[27919]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T20:04:05.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:05.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:04 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/1146862970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:04:05.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:04 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/797644910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]: dispatch 2024-04-26T20:04:05.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:04 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]: dispatch 2024-04-26T20:04:05.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:04 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5300878b-efa3-49be-afa6-09efbaae8c5b"}]': finished 2024-04-26T20:04:05.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:04 smithi148 ceph-mon[32281]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T20:04:05.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:05.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:04 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/1146862970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:04:05.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:05 smithi157 ceph-mon[32703]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:06.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:05 smithi131 ceph-mon[27919]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:06.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:05 smithi148 ceph-mon[32281]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:07.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:07 smithi148 ceph-mon[32281]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:07.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:07 smithi157 ceph-mon[32703]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:08.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:07 smithi131 ceph-mon[27919]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:09.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:09 smithi148 ceph-mon[32281]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:09.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:09 smithi157 ceph-mon[32703]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:10.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:09 smithi131 ceph-mon[27919]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:11.569 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T20:04:11.570 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:11.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T20:04:11.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:11.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T20:04:11.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:12.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:12 smithi148 ceph-mon[32281]: Deploying daemon osd.7 on smithi148 2024-04-26T20:04:12.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:12 smithi148 ceph-mon[32281]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:12.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:12 smithi157 ceph-mon[32703]: Deploying daemon osd.7 on smithi148 2024-04-26T20:04:12.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:12 smithi157 ceph-mon[32703]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:12.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:12 smithi131 ceph-mon[27919]: Deploying daemon osd.7 on smithi148 2024-04-26T20:04:12.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:12 smithi131 ceph-mon[27919]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:13.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:13 smithi157 ceph-mon[32703]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:14.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:13 smithi131 ceph-mon[27919]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:14.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:13 smithi148 ceph-mon[32281]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:15.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:15 smithi148 ceph-mon[32281]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:15.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:15 smithi157 ceph-mon[32703]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:16.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:15 smithi131 ceph-mon[27919]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:18.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:18.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:18.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:18.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:18 smithi148 ceph-mon[32281]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:18.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:18.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:18.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:18.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:18 smithi157 ceph-mon[32703]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:18.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:18.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:18.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:18.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:18 smithi131 ceph-mon[27919]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:19.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:19 smithi148 ceph-mon[32281]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:19.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:19 smithi157 ceph-mon[32703]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:20.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:19 smithi131 ceph-mon[27919]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:21.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.080+0000 7f1ba3fff700 1 -- 172.21.15.148:0/324403399 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1bbc063860 con 0x7f1ba40a3c40 2024-04-26T20:04:21.082 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 7 on host 'smithi148' 2024-04-26T20:04:21.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ba40a3c40 msgr2=0x7f1ba40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:21.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ba40a3c40 0x7f1ba40a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000bcf0 tx=0x7f1bb000b2c0 comp rx=0 tx=0).stop 2024-04-26T20:04:21.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 msgr2=0x7f1bbc199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:21.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc199630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0036f0 tx=0x7f1bac005e80 comp rx=0 tx=0).stop 2024-04-26T20:04:21.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 shutdown_connections 2024-04-26T20:04:21.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc109230 0x7f1bbc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:21.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ba40a3c40 0x7f1ba40a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:21.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc103fd0 0x7f1bbc199630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:21.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 --2- 172.21.15.148:0/324403399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc106900 0x7f1bbc199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:21.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.082+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 >> 172.21.15.148:0/324403399 conn(0x7f1bbc0fd920 msgr2=0x7f1bbc1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:04:21.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.083+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 shutdown_connections 2024-04-26T20:04:21.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:04:21.083+0000 7f1bc1656700 1 -- 172.21.15.148:0/324403399 wait complete. 2024-04-26T20:04:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:21.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T20:04:21.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T20:04:21.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:21.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T20:04:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T20:04:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:21.796 DEBUG:teuthology.orchestra.run.smithi148:osd.7> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T20:04:21.798 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-04-26T20:04:21.798 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T20:04:22.048 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:04:22.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-mon[32281]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T20:04:22.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-mon[32281]: from='osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:04:22.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-mon[32281]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T20:04:22.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:22.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-mon[32281]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:04:22.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-mon[32281]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:22.318 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:04:22.234+0000 7fefe9bd5700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T20:04:22.318 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:04:22 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:04:22.245+0000 7fefe59c8700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:04:22.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:22 smithi157 ceph-mon[32703]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T20:04:22.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:22 smithi157 ceph-mon[32703]: from='osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:04:22.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:22 smithi157 ceph-mon[32703]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T20:04:22.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:22.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:22 smithi157 ceph-mon[32703]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:04:22.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:22 smithi157 ceph-mon[32703]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:22 smithi131 ceph-mon[27919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T20:04:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:22 smithi131 ceph-mon[27919]: from='osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:04:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:22 smithi131 ceph-mon[27919]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T20:04:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:22.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:22 smithi131 ceph-mon[27919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:04:22.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:22 smithi131 ceph-mon[27919]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-26T20:04:23.448 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:23 smithi157 ceph-mon[32703]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:04:23.448 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:23 smithi157 ceph-mon[32703]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T20:04:23.448 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:23.448 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:23 smithi157 ceph-mon[32703]: osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] boot 2024-04-26T20:04:23.448 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:23 smithi157 ceph-mon[32703]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T20:04:23.448 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:23.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:23 smithi148 ceph-mon[32281]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:04:23.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:23 smithi148 ceph-mon[32281]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T20:04:23.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:23.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:23 smithi148 ceph-mon[32281]: osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] boot 2024-04-26T20:04:23.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:23 smithi148 ceph-mon[32281]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T20:04:23.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:23.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:23 smithi131 ceph-mon[27919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:04:23.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:23 smithi131 ceph-mon[27919]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T20:04:23.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:23.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:23 smithi131 ceph-mon[27919]: osd.7 [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] boot 2024-04-26T20:04:23.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:23 smithi131 ceph-mon[27919]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T20:04:23.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T20:04:24.195 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T20:04:24.196 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T20:04:24.196 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-26T20:04:24.196 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-26T20:04:24.196 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00870371 s, 1.2 GB/s 2024-04-26T20:04:24.196 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-26T20:04:24.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:24 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:04:24.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:24 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:04:24.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:24 smithi157 ceph-mon[32703]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T20:04:24.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:24 smithi157 ceph-mon[32703]: pgmap v197: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:24.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:24 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:04:24.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:24 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:04:24.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:24 smithi148 ceph-mon[32281]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T20:04:24.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:24 smithi148 ceph-mon[32281]: pgmap v197: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:24.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:24 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:04:24.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:24 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:04:24.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:24 smithi131 ceph-mon[27919]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T20:04:24.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:24 smithi131 ceph-mon[27919]: pgmap v197: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:24.859 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-04-26T20:04:25.101 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:04:25.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T20:04:25.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T20:04:25.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:25.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:25.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:25.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:25.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:25.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:26 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi148 2024-04-26T20:04:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:26 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-26T20:04:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:26 smithi157 ceph-mon[32703]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T20:04:26.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:26 smithi157 ceph-mon[32703]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:26.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:26 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi148 2024-04-26T20:04:26.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:26 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-26T20:04:26.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:26 smithi148 ceph-mon[32281]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T20:04:26.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:26 smithi148 ceph-mon[32281]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:26.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:26 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi148 2024-04-26T20:04:26.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:26 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-26T20:04:26.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:26 smithi131 ceph-mon[27919]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T20:04:26.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:26 smithi131 ceph-mon[27919]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:26.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 -- 172.21.15.157:0/4059560290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 msgr2=0x7ff7d41027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:26.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/4059560290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d41027a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009990 tx=0x7ff7d0009ca0 comp rx=0 tx=0).stop 2024-04-26T20:04:26.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 -- 172.21.15.157:0/4059560290 shutdown_connections 2024-04-26T20:04:26.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/4059560290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4103f20 0x7ff7d410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:26.809 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/4059560290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d41027a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/4059560290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7d4103580 0x7ff7d41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 -- 172.21.15.157:0/4059560290 >> 172.21.15.157:0/4059560290 conn(0x7ff7d40fd920 msgr2=0x7ff7d40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 -- 172.21.15.157:0/4059560290 shutdown_connections 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 -- 172.21.15.157:0/4059560290 wait complete. 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.793+0000 7ff7dc22e700 1 Processor -- start 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7dc22e700 1 -- start start 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7dc22e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7dc22e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4103580 0x7ff7d4198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7dc22e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7d4103f20 0x7ff7d419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7dc22e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d410def0 con 0x7ff7d4103580 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7dc22e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d410dd70 con 0x7ff7d4103f20 2024-04-26T20:04:26.810 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7dc22e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d410e070 con 0x7ff7d4102380 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d4198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54444/0 (socket says 172.21.15.157:54444) 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 -- 172.21.15.157:0/1778865203 learned_addr learned my addr 172.21.15.157:0/1778865203 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7da7cb700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7d4103f20 0x7ff7d419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 -- 172.21.15.157:0/1778865203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7d4103f20 msgr2=0x7ff7d419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7d4103f20 0x7ff7d419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 -- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4103580 msgr2=0x7ff7d4198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4103580 0x7ff7d4198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 -- 172.21.15.157:0/1778865203 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d41a0220 con 0x7ff7d4102380 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7d9fca700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d4198610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00047b0 tx=0x7ff7d0009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7caffd700 1 -- 172.21.15.157:0/1778865203 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0027070 con 0x7ff7d4102380 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.794+0000 7ff7caffd700 1 -- 172.21.15.157:0/1778865203 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d0022470 con 0x7ff7d4102380 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.795+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d41a04b0 con 0x7ff7d4102380 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.795+0000 7ff7caffd700 1 -- 172.21.15.157:0/1778865203 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000f6c0 con 0x7ff7d4102380 2024-04-26T20:04:26.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.795+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41a0980 con 0x7ff7d4102380 2024-04-26T20:04:26.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.796+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d406a110 con 0x7ff7d4102380 2024-04-26T20:04:26.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.797+0000 7ff7caffd700 1 -- 172.21.15.157:0/1778865203 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff7d002a070 con 0x7ff7d4102380 2024-04-26T20:04:26.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.798+0000 7ff7caffd700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff7c00a3c40 0x7ff7c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:26.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.798+0000 7ff7caffd700 1 -- 172.21.15.157:0/1778865203 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7ff7d00d0c10 con 0x7ff7d4102380 2024-04-26T20:04:26.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.798+0000 7ff7d97c9700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff7c00a3c40 0x7ff7c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:26.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.799+0000 7ff7d97c9700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff7c00a3c40 0x7ff7c00a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40097b0 tx=0x7ff7c4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:04:26.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:26.801+0000 7ff7caffd700 1 -- 172.21.15.157:0/1778865203 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff7d0013070 con 0x7ff7d4102380 2024-04-26T20:04:27.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:27.009+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff7d4063860 con 0x7ff7c00a3c40 2024-04-26T20:04:27.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:27 smithi148 ceph-mon[32281]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T20:04:27.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:27.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:27.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:27 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:27.651 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:27 smithi157 ceph-mon[32703]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T20:04:27.651 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:27.651 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:27.651 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:27 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:27.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:27 smithi131 ceph-mon[27919]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T20:04:27.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:27.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:27.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:27 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:28.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:28 smithi148 ceph-mon[32281]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:28.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:28 smithi148 ceph-mon[32281]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:28.658 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:28 smithi157 ceph-mon[32703]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:28.658 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:28 smithi157 ceph-mon[32703]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:28.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:28 smithi131 ceph-mon[27919]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:28.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:28 smithi131 ceph-mon[27919]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:29 smithi157 ceph-mon[32703]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:29 smithi157 ceph-mon[32703]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T20:04:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:29 smithi157 ceph-mon[32703]: Cluster is now healthy 2024-04-26T20:04:30.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:29 smithi131 ceph-mon[27919]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:30.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:29 smithi131 ceph-mon[27919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T20:04:30.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:29 smithi131 ceph-mon[27919]: Cluster is now healthy 2024-04-26T20:04:30.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:29 smithi148 ceph-mon[32281]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:30.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:29 smithi148 ceph-mon[32281]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T20:04:30.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:29 smithi148 ceph-mon[32281]: Cluster is now healthy 2024-04-26T20:04:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:30 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/2326154350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]: dispatch 2024-04-26T20:04:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:30 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]: dispatch 2024-04-26T20:04:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:30 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]': finished 2024-04-26T20:04:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:30 smithi157 ceph-mon[32703]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T20:04:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:30 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/463371901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:04:31.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:30 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/2326154350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]: dispatch 2024-04-26T20:04:31.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:30 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]: dispatch 2024-04-26T20:04:31.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:30 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]': finished 2024-04-26T20:04:31.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:30 smithi131 ceph-mon[27919]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T20:04:31.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:30 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/463371901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:04:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:30 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/2326154350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]: dispatch 2024-04-26T20:04:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:30 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]: dispatch 2024-04-26T20:04:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:30 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb399a8-15a1-48dc-a538-63745daf7fa0"}]': finished 2024-04-26T20:04:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:30 smithi148 ceph-mon[32281]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T20:04:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:30 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/463371901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:04:31.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:31 smithi157 ceph-mon[32703]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:32.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:31 smithi131 ceph-mon[27919]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:32.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:31 smithi148 ceph-mon[32281]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:33.901 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:33 smithi157 ceph-mon[32703]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:34.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:33 smithi131 ceph-mon[27919]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:34.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:33 smithi148 ceph-mon[32281]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:35.908 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:35 smithi157 ceph-mon[32703]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:36.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:35 smithi131 ceph-mon[27919]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:36.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:35 smithi148 ceph-mon[32281]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:37.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:37 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T20:04:37.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:37 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:37 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T20:04:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:37 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:37.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:37 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T20:04:37.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:37 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:38.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:38 smithi157 ceph-mon[32703]: Deploying daemon osd.8 on smithi157 2024-04-26T20:04:38.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:38 smithi157 ceph-mon[32703]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:38.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:38 smithi131 ceph-mon[27919]: Deploying daemon osd.8 on smithi157 2024-04-26T20:04:38.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:38 smithi131 ceph-mon[27919]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:38.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:38 smithi148 ceph-mon[32281]: Deploying daemon osd.8 on smithi157 2024-04-26T20:04:38.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:38 smithi148 ceph-mon[32281]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:39.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:39 smithi157 ceph-mon[32703]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:40.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:39 smithi131 ceph-mon[27919]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:40.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:39 smithi148 ceph-mon[32281]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:41.943 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:41 smithi157 ceph-mon[32703]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:41 smithi131 ceph-mon[27919]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:42.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:41 smithi148 ceph-mon[32281]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:44.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:44.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:44.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:44.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:44 smithi157 ceph-mon[32703]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:44.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:44.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:44.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:44.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:44 smithi131 ceph-mon[27919]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:44.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:44.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:44.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:44.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:44 smithi148 ceph-mon[32281]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:45.886 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:45 smithi157 ceph-mon[32703]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:46.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:45 smithi131 ceph-mon[27919]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:46.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:45 smithi148 ceph-mon[32281]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:47.197 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-04-26T20:04:47.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.196+0000 7ff7caffd700 1 -- 172.21.15.157:0/1778865203 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff7d4063860 con 0x7ff7c00a3c40 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.198+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff7c00a3c40 msgr2=0x7ff7c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.198+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff7c00a3c40 0x7ff7c00a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40097b0 tx=0x7ff7c4006d20 comp rx=0 tx=0).stop 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 msgr2=0x7ff7d4198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d4198610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00047b0 tx=0x7ff7d0009f50 comp rx=0 tx=0).stop 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 shutdown_connections 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff7c00a3c40 0x7ff7c00a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7d4103f20 0x7ff7d419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4102380 0x7ff7d4198610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 --2- 172.21.15.157:0/1778865203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4103580 0x7ff7d4198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:47.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 >> 172.21.15.157:0/1778865203 conn(0x7ff7d40fd920 msgr2=0x7ff7d41069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:04:47.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 shutdown_connections 2024-04-26T20:04:47.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:04:47.199+0000 7ff7dc22e700 1 -- 172.21.15.157:0/1778865203 wait complete. 2024-04-26T20:04:47.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:47.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:47.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T20:04:47.569 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T20:04:47.569 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.569 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.569 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:47.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:47.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:47.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T20:04:47.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T20:04:47.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:47.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:48.020 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T20:04:48.022 INFO:tasks.cephadm:Deploying osd.9 on smithi131 with /dev/vg_nvme/lv_1... 2024-04-26T20:04:48.023 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T20:04:48.472 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:04:48.563 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-mon[32703]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T20:04:48.654 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-mon[32703]: from='osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:04:48.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-mon[32703]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T20:04:48.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:48.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-mon[32703]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:04:48.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-mon[32703]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:48.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:48 smithi131 ceph-mon[27919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T20:04:48.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:48 smithi131 ceph-mon[27919]: from='osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:04:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:48 smithi131 ceph-mon[27919]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T20:04:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:48 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:48 smithi131 ceph-mon[27919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:04:48.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:48 smithi131 ceph-mon[27919]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:48.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:48 smithi148 ceph-mon[32281]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T20:04:48.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:48 smithi148 ceph-mon[32281]: from='osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:04:48.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:48 smithi148 ceph-mon[32281]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T20:04:48.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:48 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:48.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:48 smithi148 ceph-mon[32281]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:04:48.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:48 smithi148 ceph-mon[32281]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:49.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:04:48.951+0000 7fef7605c700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T20:04:49.231 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:04:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:04:48.962+0000 7fef6d646700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:04:49.645 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:49 smithi157 ceph-mon[32703]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:04:49.645 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:49 smithi157 ceph-mon[32703]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T20:04:49.645 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:49.645 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:49.645 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:49 smithi157 ceph-mon[32703]: from='osd.8 ' entity='osd.8' 2024-04-26T20:04:49.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:49 smithi131 ceph-mon[27919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:04:49.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:49 smithi131 ceph-mon[27919]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T20:04:49.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:49.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:49.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:49 smithi131 ceph-mon[27919]: from='osd.8 ' entity='osd.8' 2024-04-26T20:04:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:49 smithi148 ceph-mon[32281]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:04:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:49 smithi148 ceph-mon[32281]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T20:04:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:49 smithi148 ceph-mon[32281]: from='osd.8 ' entity='osd.8' 2024-04-26T20:04:50.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:04:50.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:04:50.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:50.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] boot 2024-04-26T20:04:50.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T20:04:50.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:50.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:50.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:50 smithi131 ceph-mon[27919]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T20:04:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:04:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:04:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:50.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] boot 2024-04-26T20:04:50.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T20:04:50.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:50.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:50.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:50 smithi157 ceph-mon[32703]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: osd.8 [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] boot 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-26T20:04:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:50 smithi148 ceph-mon[32281]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T20:04:51.608 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T20:04:51.608 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T20:04:51.608 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-04-26T20:04:51.608 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-04-26T20:04:51.608 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0415394 s, 252 MB/s 2024-04-26T20:04:51.608 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-04-26T20:04:52.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T20:04:52.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:04:52.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi157 2024-04-26T20:04:52.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:52 smithi131 ceph-mon[27919]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T20:04:52.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi157 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:52.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:52.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:52 smithi157 ceph-mon[32703]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi157 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:52.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:04:52.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:52 smithi148 ceph-mon[32281]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T20:04:53.146 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:04:53.569 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:53 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi157 to 5696M 2024-04-26T20:04:53.569 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:53 smithi131 ceph-mon[27919]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T20:04:53.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:53 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi157 to 5696M 2024-04-26T20:04:53.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:53 smithi157 ceph-mon[32703]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T20:04:53.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:53 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi157 to 5696M 2024-04-26T20:04:53.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:53 smithi148 ceph-mon[32281]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T20:04:54.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:54 smithi157 ceph-mon[32703]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T20:04:54.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:54 smithi131 ceph-mon[27919]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T20:04:54.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:54 smithi148 ceph-mon[32281]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T20:04:55.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 -- 172.21.15.131:0/451976709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08106870 msgr2=0x7fea08108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 --2- 172.21.15.131:0/451976709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08106870 0x7fea08108c60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009990 tx=0x7fe9f4009ca0 comp rx=0 tx=0).stop 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 -- 172.21.15.131:0/451976709 shutdown_connections 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 --2- 172.21.15.131:0/451976709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea081091a0 0x7fea0810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 --2- 172.21.15.131:0/451976709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 0x7fea08106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 --2- 172.21.15.131:0/451976709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08106870 0x7fea08108c60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 -- 172.21.15.131:0/451976709 >> 172.21.15.131:0/451976709 conn(0x7fea080fd970 msgr2=0x7fea080ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 -- 172.21.15.131:0/451976709 shutdown_connections 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.438+0000 7fea0e200700 1 -- 172.21.15.131:0/451976709 wait complete. 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.439+0000 7fea0e200700 1 Processor -- start 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.439+0000 7fea0e200700 1 -- start start 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea0e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 0x7fea0819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea0e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08106870 0x7fea0819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea0e200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea081091a0 0x7fea081a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea0e200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea0810efc0 con 0x7fea08106870 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea0e200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea0810ee40 con 0x7fea081091a0 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea0e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea0810f140 con 0x7fea08103f40 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea07400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 0x7fea0819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea07400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 0x7fea0819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48942/0 (socket says 172.21.15.131:48942) 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea07400700 1 -- 172.21.15.131:0/585280683 learned_addr learned my addr 172.21.15.131:0/585280683 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea06a00700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08106870 0x7fea0819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea07e00700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea081091a0 0x7fea081a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea07400700 1 -- 172.21.15.131:0/585280683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea081091a0 msgr2=0x7fea081a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.440+0000 7fea07400700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea081091a0 0x7fea081a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fea07400700 1 -- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08106870 msgr2=0x7fea0819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:04:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fea07400700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08106870 0x7fea0819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fea07400700 1 -- 172.21.15.131:0/585280683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea081a3480 con 0x7fea08103f40 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fea07400700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 0x7fea0819b820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d9b0 tx=0x7fe9f800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fe9ffe00700 1 -- 172.21.15.131:0/585280683 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8016070 con 0x7fea08103f40 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081a3770 con 0x7fea08103f40 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fe9ffe00700 1 -- 172.21.15.131:0/585280683 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f8004830 con 0x7fea08103f40 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fe9ffe00700 1 -- 172.21.15.131:0/585280683 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8005420 con 0x7fea08103f40 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.441+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081a3d00 con 0x7fea08103f40 2024-04-26T20:04:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.443+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0806a110 con 0x7fea08103f40 2024-04-26T20:04:55.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.445+0000 7fe9ffe00700 1 -- 172.21.15.131:0/585280683 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe9f80055c0 con 0x7fea08103f40 2024-04-26T20:04:55.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.446+0000 7fe9ffe00700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9ec0a3c40 0x7fe9ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:04:55.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.446+0000 7fe9ffe00700 1 -- 172.21.15.131:0/585280683 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fe9f80cf860 con 0x7fea08103f40 2024-04-26T20:04:55.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.446+0000 7fea06a00700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9ec0a3c40 0x7fe9ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:04:55.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.447+0000 7fea06a00700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9ec0a3c40 0x7fe9ec0a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400b2e0 tx=0x7fe9f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:04:55.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.451+0000 7fe9ffe00700 1 -- 172.21.15.131:0/585280683 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe9f801f020 con 0x7fea08103f40 2024-04-26T20:04:55.694 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:55 smithi131 ceph-mon[27919]: pgmap v224: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-26T20:04:55.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:04:55.693+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fea08063860 con 0x7fe9ec0a3c40 2024-04-26T20:04:55.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:55 smithi157 ceph-mon[32703]: pgmap v224: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-26T20:04:56.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:55 smithi148 ceph-mon[32281]: pgmap v224: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-26T20:04:56.939 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:56 smithi131 ceph-mon[27919]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:56.939 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:56.939 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:56.939 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:56 smithi157 ceph-mon[32703]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:56.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:56 smithi148 ceph-mon[32281]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:04:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:04:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:04:57.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:57.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:57 smithi131 ceph-mon[27919]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-26T20:04:57.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:57 smithi157 ceph-mon[32703]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-26T20:04:58.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:57 smithi148 ceph-mon[32281]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-26T20:04:59.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:04:59 smithi157 ceph-mon[32703]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T20:05:00.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:04:59 smithi131 ceph-mon[27919]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T20:05:00.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:04:59 smithi148 ceph-mon[32281]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T20:05:00.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:00 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1371611661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]: dispatch 2024-04-26T20:05:00.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:00 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]: dispatch 2024-04-26T20:05:00.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:00 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]': finished 2024-04-26T20:05:00.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:00 smithi131 ceph-mon[27919]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T20:05:00.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:00 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1371611661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]: dispatch 2024-04-26T20:05:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:00 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]: dispatch 2024-04-26T20:05:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:00 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]': finished 2024-04-26T20:05:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:00 smithi157 ceph-mon[32703]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T20:05:00.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:01.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:00 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1371611661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]: dispatch 2024-04-26T20:05:01.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:00 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]: dispatch 2024-04-26T20:05:01.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:00 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b119ef2-f46f-4511-9440-283185ed5f09"}]': finished 2024-04-26T20:05:01.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:00 smithi148 ceph-mon[32281]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T20:05:01.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:01.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:01 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1616736396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:05:01.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:01 smithi157 ceph-mon[32703]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:01 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1616736396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:05:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:01 smithi131 ceph-mon[27919]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:02.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:01 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1616736396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:05:02.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:01 smithi148 ceph-mon[32281]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:03.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:03 smithi157 ceph-mon[32703]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:04.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:03 smithi131 ceph-mon[27919]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:04.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:03 smithi148 ceph-mon[32281]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:05.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:05 smithi131 ceph-mon[27919]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:05.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:05 smithi157 ceph-mon[32703]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:06.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:05 smithi148 ceph-mon[32281]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:07.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:07 smithi131 ceph-mon[27919]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:07.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:07 smithi157 ceph-mon[32703]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:08.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:07 smithi148 ceph-mon[32281]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:09.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:09 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T20:05:09.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:09 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:09.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:09 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T20:05:09.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:09 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:09.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:09 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T20:05:09.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:09 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:10.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:10 smithi131 ceph-mon[27919]: Deploying daemon osd.9 on smithi131 2024-04-26T20:05:10.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:10 smithi131 ceph-mon[27919]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:10.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:10 smithi148 ceph-mon[32281]: Deploying daemon osd.9 on smithi131 2024-04-26T20:05:10.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:10 smithi148 ceph-mon[32281]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:10.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:10 smithi157 ceph-mon[32703]: Deploying daemon osd.9 on smithi131 2024-04-26T20:05:10.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:10 smithi157 ceph-mon[32703]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:11.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:11 smithi131 ceph-mon[27919]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:11.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:11 smithi157 ceph-mon[32703]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:12.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:11 smithi148 ceph-mon[32281]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:13.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:13 smithi131 ceph-mon[27919]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:13.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:13 smithi157 ceph-mon[32703]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:14.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:13 smithi148 ceph-mon[32281]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:15.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:15 smithi157 ceph-mon[32703]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:16.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:15 smithi131 ceph-mon[27919]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:16.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:15 smithi148 ceph-mon[32281]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:17.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:17 smithi131 ceph-mon[27919]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:17.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:17 smithi157 ceph-mon[32703]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:18.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:17 smithi148 ceph-mon[32281]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:19.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:19.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:19.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:19.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:19.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:19.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:19.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:19.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:19.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:20.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:20 smithi131 ceph-mon[27919]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:20.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:20 smithi157 ceph-mon[32703]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:20.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:20 smithi148 ceph-mon[32281]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:21.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:21 smithi157 ceph-mon[32703]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:22.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:21 smithi131 ceph-mon[27919]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:22.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:21 smithi148 ceph-mon[32281]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:22.670 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:22 smithi131 ceph-mon[27919]: from='osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T20:05:22.670 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:22 smithi131 ceph-mon[27919]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T20:05:22.670 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:22.670 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:22 smithi157 ceph-mon[32703]: from='osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T20:05:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:22 smithi157 ceph-mon[32703]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T20:05:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:22 smithi148 ceph-mon[32281]: from='osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T20:05:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:22 smithi148 ceph-mon[32281]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T20:05:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.521+0000 7fe9ffe00700 1 -- 172.21.15.131:0/585280683 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fea08063860 con 0x7fe9ec0a3c40 2024-04-26T20:05:23.524 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 9 on host 'smithi131' 2024-04-26T20:05:23.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9ec0a3c40 msgr2=0x7fe9ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:23.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9ec0a3c40 0x7fe9ec0a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400b2e0 tx=0x7fe9f4005fb0 comp rx=0 tx=0).stop 2024-04-26T20:05:23.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 msgr2=0x7fea0819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:23.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 0x7fea0819b820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d9b0 tx=0x7fe9f800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:05:23.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 shutdown_connections 2024-04-26T20:05:23.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9ec0a3c40 0x7fe9ec0a6100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:23.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea081091a0 0x7fea081a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:23.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08103f40 0x7fea0819b820 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:23.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 --2- 172.21.15.131:0/585280683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08106870 0x7fea0819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:23.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.524+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 >> 172.21.15.131:0/585280683 conn(0x7fea080fd970 msgr2=0x7fea08103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:05:23.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.525+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 shutdown_connections 2024-04-26T20:05:23.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:05:23.525+0000 7fea0e200700 1 -- 172.21.15.131:0/585280683 wait complete. 2024-04-26T20:05:23.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T20:05:23.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:05:23.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:23 smithi131 ceph-mon[27919]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:23.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T20:05:23.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:05:23.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:23.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:23 smithi157 ceph-mon[32703]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:24.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:24.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:23 smithi148 ceph-mon[32281]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:24.392 DEBUG:teuthology.orchestra.run.smithi131:osd.9> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T20:05:24.397 INFO:tasks.cephadm:Deploying osd.10 on smithi148 with /dev/vg_nvme/lv_1... 2024-04-26T20:05:24.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T20:05:24.648 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:05:24.662 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:05:24.502+0000 7fd474600700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T20:05:24.662 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:05:24.530+0000 7fd46a400700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:05:24.770 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:24 smithi148 ceph-mon[32281]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:05:24.770 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:24 smithi148 ceph-mon[32281]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T20:05:24.770 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:24.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:24.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:24 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:24.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:24 smithi148 ceph-mon[32281]: from='osd.9 ' entity='osd.9' 2024-04-26T20:05:24.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:24 smithi157 ceph-mon[32703]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:05:24.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:24 smithi157 ceph-mon[32703]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T20:05:24.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:24.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:24.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:24 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:24.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:24 smithi157 ceph-mon[32703]: from='osd.9 ' entity='osd.9' 2024-04-26T20:05:25.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-mon[27919]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-04-26T20:05:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-mon[27919]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T20:05:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:25.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:24 smithi131 ceph-mon[27919]: from='osd.9 ' entity='osd.9' 2024-04-26T20:05:25.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:25 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:05:25.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:25 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:05:25.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:25.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:25 smithi157 ceph-mon[32703]: osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] boot 2024-04-26T20:05:25.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:25 smithi157 ceph-mon[32703]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T20:05:25.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:25.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:25 smithi157 ceph-mon[32703]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:26.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:25 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:05:26.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:25 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:05:26.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:26.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:25 smithi131 ceph-mon[27919]: osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] boot 2024-04-26T20:05:26.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:25 smithi131 ceph-mon[27919]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T20:05:26.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:26.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:25 smithi131 ceph-mon[27919]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:25 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:05:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:25 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:05:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:25 smithi148 ceph-mon[32281]: osd.9 [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] boot 2024-04-26T20:05:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:25 smithi148 ceph-mon[32281]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T20:05:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T20:05:26.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:25 smithi148 ceph-mon[32281]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-26T20:05:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T20:05:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T20:05:26.757 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-26T20:05:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-26T20:05:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121424 s, 864 MB/s 2024-04-26T20:05:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-26T20:05:27.433 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2024-04-26T20:05:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:05:27.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:27 smithi148 ceph-mon[32281]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T20:05:27.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:27 smithi148 ceph-mon[32281]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:27.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:27 smithi157 ceph-mon[32703]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T20:05:27.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:27 smithi157 ceph-mon[32703]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:28.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:27 smithi131 ceph-mon[27919]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T20:05:28.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:27 smithi131 ceph-mon[27919]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:29.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 -- 172.21.15.148:0/659456712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 msgr2=0x7f71c4104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 --2- 172.21.15.148:0/659456712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4104ae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f71b40099c0 tx=0x7f71b4009cd0 comp rx=0 tx=0).stop 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 -- 172.21.15.148:0/659456712 shutdown_connections 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 --2- 172.21.15.148:0/659456712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c4105020 0x7f71c410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 --2- 172.21.15.148:0/659456712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c4103480 0x7f71c41038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 --2- 172.21.15.148:0/659456712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4104ae0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 -- 172.21.15.148:0/659456712 >> 172.21.15.148:0/659456712 conn(0x7f71c40fea20 msgr2=0x7f71c4100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 -- 172.21.15.148:0/659456712 shutdown_connections 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.487+0000 7f71c91b7700 1 -- 172.21.15.148:0/659456712 wait complete. 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 Processor -- start 2024-04-26T20:05:29.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 -- start start 2024-04-26T20:05:29.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c4103480 0x7f71c4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c4105020 0x7f71c41a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71c410ef90 con 0x7f71c4105020 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71c410ee10 con 0x7f71c4104680 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c91b7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71c410f110 con 0x7f71c4103480 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39358/0 (socket says 172.21.15.148:39358) 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c259c700 1 -- 172.21.15.148:0/1168208208 learned_addr learned my addr 172.21.15.148:0/1168208208 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.488+0000 7f71c2d9d700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c4103480 0x7f71c4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:05:29.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c259c700 1 -- 172.21.15.148:0/1168208208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c4103480 msgr2=0x7f71c4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c259c700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c4103480 0x7f71c4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c259c700 1 -- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c4105020 msgr2=0x7f71c41a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c259c700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c4105020 0x7f71c41a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c259c700 1 -- 172.21.15.148:0/1168208208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c41a12a0 con 0x7f71c4104680 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c259c700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4199b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71b4003790 tx=0x7f71b4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71bbfff700 1 -- 172.21.15.148:0/1168208208 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4027070 con 0x7f71c4104680 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c41a1530 con 0x7f71c4104680 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71bbfff700 1 -- 172.21.15.148:0/1168208208 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b4022470 con 0x7f71c4104680 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71bbfff700 1 -- 172.21.15.148:0/1168208208 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b400f6c0 con 0x7f71c4104680 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.489+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c41a1a60 con 0x7f71c4104680 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.490+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c406a110 con 0x7f71c4104680 2024-04-26T20:05:29.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.491+0000 7f71bbfff700 1 -- 172.21.15.148:0/1168208208 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f71b402a070 con 0x7f71c4104680 2024-04-26T20:05:29.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.492+0000 7f71bbfff700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f71b00a3cc0 0x7f71b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:29.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.492+0000 7f71bbfff700 1 -- 172.21.15.148:0/1168208208 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f71b40d0ae0 con 0x7f71c4104680 2024-04-26T20:05:29.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.492+0000 7f71c2d9d700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f71b00a3cc0 0x7f71b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:05:29.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.493+0000 7f71c2d9d700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f71b00a3cc0 0x7f71b00a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005950 tx=0x7f71ac00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:05:29.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.496+0000 7f71bbfff700 1 -- 172.21.15.148:0/1168208208 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f71b4013070 con 0x7f71c4104680 2024-04-26T20:05:29.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:29.705+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f71c4063860 con 0x7f71b00a3cc0 2024-04-26T20:05:29.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:29 smithi131 ceph-mon[27919]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:29 smithi157 ceph-mon[32703]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:30.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:29 smithi148 ceph-mon[32281]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:05:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:05:30.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi131 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi131 to 4272M 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:30.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi131 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi131 to 4272M 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:31.008 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:31.008 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi131 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:31.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi131 to 4272M 2024-04-26T20:05:31.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:31.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:31.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:31.968 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:31 smithi148 ceph-mon[32281]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:31.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:31 smithi157 ceph-mon[32703]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:32.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:31 smithi131 ceph-mon[27919]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:32.944 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:32 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/3039004052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]: dispatch 2024-04-26T20:05:32.944 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:32 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]: dispatch 2024-04-26T20:05:32.944 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:32 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]': finished 2024-04-26T20:05:32.944 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:32 smithi148 ceph-mon[32281]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T20:05:32.945 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:32 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/3039004052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]: dispatch 2024-04-26T20:05:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:32 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]: dispatch 2024-04-26T20:05:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:32 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]': finished 2024-04-26T20:05:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:32 smithi157 ceph-mon[32703]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T20:05:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:33.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:32 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/3039004052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]: dispatch 2024-04-26T20:05:33.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:32 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]: dispatch 2024-04-26T20:05:33.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:32 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0272c03a-fb92-456b-aee8-3380875cef29"}]': finished 2024-04-26T20:05:33.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:32 smithi131 ceph-mon[27919]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T20:05:33.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:33.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:33 smithi157 ceph-mon[32703]: from='client.? 172.21.15.148:0/1002178335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:05:33.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:33 smithi157 ceph-mon[32703]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:34.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:33 smithi131 ceph-mon[27919]: from='client.? 172.21.15.148:0/1002178335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:05:34.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:33 smithi131 ceph-mon[27919]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:34.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:33 smithi148 ceph-mon[32281]: from='client.? 172.21.15.148:0/1002178335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:05:34.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:33 smithi148 ceph-mon[32281]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:35.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:35 smithi157 ceph-mon[32703]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:36.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:35 smithi131 ceph-mon[27919]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:36.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:35 smithi148 ceph-mon[32281]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:37.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:37 smithi131 ceph-mon[27919]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:37.896 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:37 smithi148 ceph-mon[32281]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:37.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:37 smithi157 ceph-mon[32703]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:39.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:39 smithi148 ceph-mon[32281]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:39.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:39 smithi157 ceph-mon[32703]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:40.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:39 smithi131 ceph-mon[27919]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:40.907 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T20:05:40.907 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:40.907 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:40 smithi148 ceph-mon[32281]: Deploying daemon osd.10 on smithi148 2024-04-26T20:05:40.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T20:05:40.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:40.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:40 smithi157 ceph-mon[32703]: Deploying daemon osd.10 on smithi148 2024-04-26T20:05:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T20:05:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:40 smithi131 ceph-mon[27919]: Deploying daemon osd.10 on smithi148 2024-04-26T20:05:41.911 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:41 smithi148 ceph-mon[32281]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:41.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:41 smithi157 ceph-mon[32703]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:41 smithi131 ceph-mon[27919]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:43.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:43 smithi157 ceph-mon[32703]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:44.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:43 smithi131 ceph-mon[27919]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:44.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:43 smithi148 ceph-mon[32281]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:45.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:45 smithi148 ceph-mon[32281]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:45.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:45 smithi157 ceph-mon[32703]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:46.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:45 smithi131 ceph-mon[27919]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:47.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:47.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:47.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:47.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:47.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:47.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:47.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:47.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:47.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:48.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:47 smithi131 ceph-mon[27919]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:48.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:47 smithi148 ceph-mon[32281]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:48.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:47 smithi157 ceph-mon[32703]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.947+0000 7f71bbfff700 1 -- 172.21.15.148:0/1168208208 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f71c4063860 con 0x7f71b00a3cc0 2024-04-26T20:05:49.948 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 10 on host 'smithi148' 2024-04-26T20:05:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f71b00a3cc0 msgr2=0x7f71b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f71b00a3cc0 0x7f71b00a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005950 tx=0x7f71ac00a300 comp rx=0 tx=0).stop 2024-04-26T20:05:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 msgr2=0x7f71c4199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4199b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71b4003790 tx=0x7f71b4005ef0 comp rx=0 tx=0).stop 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 shutdown_connections 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c4105020 0x7f71c41a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c4103480 0x7f71c4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f71b00a3cc0 0x7f71b00a6180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 --2- 172.21.15.148:0/1168208208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c4104680 0x7f71c4199b80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.949+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 >> 172.21.15.148:0/1168208208 conn(0x7f71c40fea20 msgr2=0x7f71c4107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.950+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 shutdown_connections 2024-04-26T20:05:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-26T20:05:49.950+0000 7f71c91b7700 1 -- 172.21.15.148:0/1168208208 wait complete. 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T20:05:50.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:50 smithi148 ceph-mon[32281]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T20:05:50.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:50.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:50.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:50.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:50.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T20:05:50.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:50 smithi157 ceph-mon[32703]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T20:05:50.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T20:05:50.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:50 smithi131 ceph-mon[27919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T20:05:50.654 DEBUG:teuthology.orchestra.run.smithi148:osd.10> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T20:05:50.656 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-04-26T20:05:50.657 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T20:05:50.904 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:05:52.227 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:05:51.945+0000 7fbe4eac1700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T20:05:52.227 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:05:51.958+0000 7fbe4a8b4700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:05:52.228 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-mon[32281]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T20:05:52.228 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-mon[32281]: from='osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:05:52.228 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-mon[32281]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T20:05:52.228 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:52.228 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-mon[32281]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:05:52.228 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:51 smithi148 ceph-mon[32281]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:52.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:51 smithi157 ceph-mon[32703]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T20:05:52.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:51 smithi157 ceph-mon[32703]: from='osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:05:52.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:51 smithi157 ceph-mon[32703]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T20:05:52.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:51 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:52.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:51 smithi157 ceph-mon[32703]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:05:52.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:51 smithi157 ceph-mon[32703]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:52.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:51 smithi131 ceph-mon[27919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T20:05:52.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:51 smithi131 ceph-mon[27919]: from='osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:05:52.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:51 smithi131 ceph-mon[27919]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T20:05:52.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:51 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:52.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:51 smithi131 ceph-mon[27919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-26T20:05:52.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:51 smithi131 ceph-mon[27919]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-26T20:05:53.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:05:53.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:05:53.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:05:53.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T20:05:53.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:53.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] boot 2024-04-26T20:05:53.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T20:05:53.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T20:05:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T20:05:53.193 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-26T20:05:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-26T20:05:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00860979 s, 1.2 GB/s 2024-04-26T20:05:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-26T20:05:53.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:05:53.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:05:53.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:05:53.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T20:05:53.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:53.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] boot 2024-04-26T20:05:53.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T20:05:53.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: osd.10 [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] boot 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T20:05:53.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T20:05:53.926 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-04-26T20:05:54.168 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:05:54.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T20:05:54.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:54.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:54.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:54.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:54.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:54.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:05:55.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:55 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi148 2024-04-26T20:05:55.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:55 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-26T20:05:55.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:55 smithi148 ceph-mon[32281]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T20:05:55.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:55 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi148 2024-04-26T20:05:55.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:55 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-26T20:05:55.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:55 smithi157 ceph-mon[32703]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T20:05:55.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:55 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi148 2024-04-26T20:05:55.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:55 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-26T20:05:55.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:55 smithi131 ceph-mon[27919]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T20:05:55.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 -- 172.21.15.157:0/1770888339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170105010 msgr2=0x7f617010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:55.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 --2- 172.21.15.157:0/1770888339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170105010 0x7f617010b8c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f616800b0a0 tx=0x7f616800b3b0 comp rx=0 tx=0).stop 2024-04-26T20:05:55.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 -- 172.21.15.157:0/1770888339 shutdown_connections 2024-04-26T20:05:55.949 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 --2- 172.21.15.157:0/1770888339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170105010 0x7f617010b8c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:55.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 --2- 172.21.15.157:0/1770888339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6170103470 0x7f6170103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:55.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 --2- 172.21.15.157:0/1770888339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6170104670 0x7f6170104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:55.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 -- 172.21.15.157:0/1770888339 >> 172.21.15.157:0/1770888339 conn(0x7f61700fea10 msgr2=0x7f6170100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:05:55.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 -- 172.21.15.157:0/1770888339 shutdown_connections 2024-04-26T20:05:55.950 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.948+0000 7f6177045700 1 -- 172.21.15.157:0/1770888339 wait complete. 2024-04-26T20:05:55.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 Processor -- start 2024-04-26T20:05:55.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 -- start start 2024-04-26T20:05:55.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170103470 0x7f6170199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:55.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6170104670 0x7f6170199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:55.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6170105010 0x7f61701a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f617010f000 con 0x7f6170104670 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f617010ee80 con 0x7f6170105010 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6177045700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f617010f180 con 0x7f6170103470 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170103470 0x7f6170199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170103470 0x7f6170199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38238/0 (socket says 172.21.15.157:38238) 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 -- 172.21.15.157:0/345904491 learned_addr learned my addr 172.21.15.157:0/345904491 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 -- 172.21.15.157:0/345904491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6170105010 msgr2=0x7f61701a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6170105010 0x7f61701a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 -- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6170104670 msgr2=0x7f6170199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6170104670 0x7f6170199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.949+0000 7f6174de1700 1 -- 172.21.15.157:0/345904491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61701a1360 con 0x7f6170103470 2024-04-26T20:05:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.950+0000 7f6174de1700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170103470 0x7f6170199700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f616c00b530 tx=0x7f616c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:05:55.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.950+0000 7f6165ffb700 1 -- 172.21.15.157:0/345904491 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c015020 con 0x7f6170103470 2024-04-26T20:05:55.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.950+0000 7f6165ffb700 1 -- 172.21.15.157:0/345904491 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f616c011ce0 con 0x7f6170103470 2024-04-26T20:05:55.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.950+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61701a1650 con 0x7f6170103470 2024-04-26T20:05:55.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.950+0000 7f6165ffb700 1 -- 172.21.15.157:0/345904491 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c010d60 con 0x7f6170103470 2024-04-26T20:05:55.953 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.950+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701a1c10 con 0x7f6170103470 2024-04-26T20:05:55.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.951+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617006a110 con 0x7f6170103470 2024-04-26T20:05:55.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.952+0000 7f6165ffb700 1 -- 172.21.15.157:0/345904491 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f616c018020 con 0x7f6170103470 2024-04-26T20:05:55.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.952+0000 7f6165ffb700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f61580a3c40 0x7f61580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:55.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.953+0000 7f6165ffb700 1 -- 172.21.15.157:0/345904491 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f616c0cf580 con 0x7f6170103470 2024-04-26T20:05:55.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.953+0000 7f6167fff700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f61580a3c40 0x7f61580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:05:55.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.954+0000 7f6167fff700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f61580a3c40 0x7f61580a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61600097b0 tx=0x7f6160006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:05:55.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:55.957+0000 7f6165ffb700 1 -- 172.21.15.157:0/345904491 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f616c01f020 con 0x7f6170103470 2024-04-26T20:05:56.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:05:56.162+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6170063860 con 0x7f61580a3c40 2024-04-26T20:05:56.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:56 smithi157 ceph-mon[32703]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T20:05:56.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:56 smithi157 ceph-mon[32703]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:56.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:05:56.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:05:56.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:56 smithi131 ceph-mon[27919]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T20:05:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:56 smithi131 ceph-mon[27919]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:05:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:05:56.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:56.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:56 smithi148 ceph-mon[32281]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T20:05:56.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:56 smithi148 ceph-mon[32281]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:56.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T20:05:56.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T20:05:56.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:57.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:57 smithi157 ceph-mon[32703]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:05:57.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:57 smithi157 ceph-mon[32703]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T20:05:57.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:57 smithi131 ceph-mon[27919]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:05:57.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:57 smithi131 ceph-mon[27919]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T20:05:57.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:57 smithi148 ceph-mon[32281]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:05:57.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:57 smithi148 ceph-mon[32281]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T20:05:58.363 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:58 smithi157 ceph-mon[32703]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:58.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:58 smithi131 ceph-mon[27919]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:58.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:58 smithi148 ceph-mon[32281]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:05:59.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:59 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/1732115797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]: dispatch 2024-04-26T20:05:59.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:59 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]: dispatch 2024-04-26T20:05:59.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:59 smithi157 ceph-mon[32703]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]': finished 2024-04-26T20:05:59.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:59 smithi157 ceph-mon[32703]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T20:05:59.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:05:59 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:05:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:59 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/1732115797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]: dispatch 2024-04-26T20:05:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:59 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]: dispatch 2024-04-26T20:05:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:59 smithi131 ceph-mon[27919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]': finished 2024-04-26T20:05:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:59 smithi131 ceph-mon[27919]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T20:05:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:05:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:05:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:59 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/1732115797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]: dispatch 2024-04-26T20:05:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:59 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]: dispatch 2024-04-26T20:05:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:59 smithi148 ceph-mon[32281]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc8dd61-ca71-4459-ba2e-2a6e07f933d2"}]': finished 2024-04-26T20:05:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:59 smithi148 ceph-mon[32281]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T20:05:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:05:59 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:00.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:00 smithi157 ceph-mon[32703]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T20:06:00.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:00 smithi157 ceph-mon[32703]: from='client.? 172.21.15.157:0/4199054347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:06:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:00 smithi131 ceph-mon[27919]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T20:06:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:00 smithi131 ceph-mon[27919]: from='client.? 172.21.15.157:0/4199054347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:06:00.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:00 smithi148 ceph-mon[32281]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T20:06:00.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:00 smithi148 ceph-mon[32281]: from='client.? 172.21.15.157:0/4199054347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T20:06:01.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:01 smithi157 ceph-mon[32703]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T20:06:02.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:01 smithi131 ceph-mon[27919]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T20:06:02.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:01 smithi148 ceph-mon[32281]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T20:06:03.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:03 smithi157 ceph-mon[32703]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T20:06:04.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:03 smithi131 ceph-mon[27919]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T20:06:04.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:03 smithi148 ceph-mon[32281]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T20:06:05.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:05 smithi157 ceph-mon[32703]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T20:06:06.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:05 smithi131 ceph-mon[27919]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T20:06:06.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:05 smithi148 ceph-mon[32281]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T20:06:06.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T20:06:06.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:07.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T20:06:07.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:07.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T20:06:07.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:08.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:08 smithi157 ceph-mon[32703]: Deploying daemon osd.11 on smithi157 2024-04-26T20:06:08.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:08 smithi157 ceph-mon[32703]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:06:08.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:08 smithi148 ceph-mon[32281]: Deploying daemon osd.11 on smithi157 2024-04-26T20:06:08.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:08 smithi148 ceph-mon[32281]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:06:08.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:08 smithi131 ceph-mon[27919]: Deploying daemon osd.11 on smithi157 2024-04-26T20:06:08.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:08 smithi131 ceph-mon[27919]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T20:06:09.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:09 smithi157 ceph-mon[32703]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:10.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:09 smithi131 ceph-mon[27919]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:10.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:09 smithi148 ceph-mon[32281]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:11.957 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:11 smithi157 ceph-mon[32703]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:12.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:11 smithi131 ceph-mon[27919]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:12.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:11 smithi148 ceph-mon[32281]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:13.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:13.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:13.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:13.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:13 smithi157 ceph-mon[32703]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:14.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:14.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:14.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:14.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:13 smithi131 ceph-mon[27919]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:14.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:14.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:14.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:14.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:13 smithi148 ceph-mon[32281]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:15.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:15 smithi157 ceph-mon[32703]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:16.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:15 smithi131 ceph-mon[27919]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:16.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:15 smithi148 ceph-mon[32281]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:16.605 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-04-26T20:06:16.605 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.603+0000 7f6165ffb700 1 -- 172.21.15.157:0/345904491 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6170063860 con 0x7f61580a3c40 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f61580a3c40 msgr2=0x7f61580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f61580a3c40 0x7f61580a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61600097b0 tx=0x7f6160006d20 comp rx=0 tx=0).stop 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170103470 msgr2=0x7f6170199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170103470 0x7f6170199700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f616c00b530 tx=0x7f616c00b8f0 comp rx=0 tx=0).stop 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 shutdown_connections 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f61580a3c40 0x7f61580a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6170105010 0x7f61701a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6170103470 0x7f6170199700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 --2- 172.21.15.157:0/345904491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6170104670 0x7f6170199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 >> 172.21.15.157:0/345904491 conn(0x7f61700fea10 msgr2=0x7f6170107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 shutdown_connections 2024-04-26T20:06:16.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:16.606+0000 7f6177045700 1 -- 172.21.15.157:0/345904491 wait complete. 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T20:06:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T20:06:17.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:17.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:17.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:17.314 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T20:06:17.317 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T20:06:17.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T20:06:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:06:18.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:17 smithi148 ceph-mon[32281]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T20:06:18.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:17 smithi148 ceph-mon[32281]: from='osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:06:18.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:17 smithi148 ceph-mon[32281]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T20:06:18.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:17 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:18.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:17 smithi148 ceph-mon[32281]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:06:18.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:17 smithi148 ceph-mon[32281]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:18.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:17 smithi131 ceph-mon[27919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T20:06:18.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:17 smithi131 ceph-mon[27919]: from='osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:06:18.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:17 smithi131 ceph-mon[27919]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T20:06:18.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:17 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:18.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:17 smithi131 ceph-mon[27919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:06:18.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:17 smithi131 ceph-mon[27919]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:18.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-mon[32703]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T20:06:18.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-mon[32703]: from='osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:06:18.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-mon[32703]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T20:06:18.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:18.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-mon[32703]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-26T20:06:18.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-mon[32703]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:18.231 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:06:17.809+0000 7fd4e98d8700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T20:06:18.231 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:06:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:06:17.826+0000 7fd4e56cb700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:06:18.898 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:18 smithi157 ceph-mon[32703]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:06:18.898 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:18 smithi157 ceph-mon[32703]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T20:06:18.898 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:18.899 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:19.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:18 smithi148 ceph-mon[32281]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:06:19.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:18 smithi148 ceph-mon[32281]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T20:06:19.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:19.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:19.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:18 smithi131 ceph-mon[27919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-26T20:06:19.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:18 smithi131 ceph-mon[27919]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T20:06:19.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:19.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:19.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:19 smithi157 ceph-mon[32703]: purged_snaps scrub starts 2024-04-26T20:06:19.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:19 smithi157 ceph-mon[32703]: purged_snaps scrub ok 2024-04-26T20:06:19.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:19.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:19 smithi157 ceph-mon[32703]: osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] boot 2024-04-26T20:06:19.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:19 smithi157 ceph-mon[32703]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T20:06:19.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:19.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:19 smithi157 ceph-mon[32703]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:20.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:19 smithi148 ceph-mon[32281]: purged_snaps scrub starts 2024-04-26T20:06:20.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:19 smithi148 ceph-mon[32281]: purged_snaps scrub ok 2024-04-26T20:06:20.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:20.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:19 smithi148 ceph-mon[32281]: osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] boot 2024-04-26T20:06:20.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:19 smithi148 ceph-mon[32281]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T20:06:20.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:20.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:19 smithi148 ceph-mon[32281]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 -- 172.21.15.131:0/815374114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41091d0 msgr2=0x7f1aa410b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/815374114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41091d0 0x7f1aa410b5e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00b0a0 tx=0x7f1a9c00b3b0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:19 smithi131 ceph-mon[27919]: purged_snaps scrub starts 2024-04-26T20:06:20.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:19 smithi131 ceph-mon[27919]: purged_snaps scrub ok 2024-04-26T20:06:20.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:20.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:19 smithi131 ceph-mon[27919]: osd.11 [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] boot 2024-04-26T20:06:20.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:19 smithi131 ceph-mon[27919]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T20:06:20.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T20:06:20.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:19 smithi131 ceph-mon[27919]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-26T20:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 -- 172.21.15.131:0/815374114 shutdown_connections 2024-04-26T20:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/815374114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41091d0 0x7f1aa410b5e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/815374114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1aa4103f70 0x7f1aa4106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/815374114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa41068a0 0x7f1aa4108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 -- 172.21.15.131:0/815374114 >> 172.21.15.131:0/815374114 conn(0x7f1aa40fd920 msgr2=0x7f1aa40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 -- 172.21.15.131:0/815374114 shutdown_connections 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.071+0000 7f1aa8c00700 1 -- 172.21.15.131:0/815374114 wait complete. 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 Processor -- start 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 -- start start 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa4103f70 0x7f1aa4103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41068a0 0x7f1aa4101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1aa41091d0 0x7f1aa4102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410ef30 con 0x7f1aa41068a0 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410edb0 con 0x7f1aa41091d0 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa8c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410f0b0 con 0x7f1aa4103f70 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa4103f70 0x7f1aa4103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1a9be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41068a0 0x7f1aa4101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa4103f70 0x7f1aa4103880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40110/0 (socket says 172.21.15.131:40110) 2024-04-26T20:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.073+0000 7f1aa1e00700 1 -- 172.21.15.131:0/3451541193 learned_addr learned my addr 172.21.15.131:0/3451541193 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1aa2800700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1aa41091d0 0x7f1aa4102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1aa1e00700 1 -- 172.21.15.131:0/3451541193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1aa41091d0 msgr2=0x7f1aa4102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1aa1e00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1aa41091d0 0x7f1aa4102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1aa1e00700 1 -- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41068a0 msgr2=0x7f1aa4101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1aa1e00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41068a0 0x7f1aa4101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1aa1e00700 1 -- 172.21.15.131:0/3451541193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa4102d30 con 0x7f1aa4103f70 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1aa1e00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa4103f70 0x7f1aa4103880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e9a0 tx=0x7f1a9400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.074+0000 7f1a9aa00700 1 -- 172.21.15.131:0/3451541193 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400ca10 con 0x7f1aa4103f70 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.075+0000 7f1a9aa00700 1 -- 172.21.15.131:0/3451541193 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9400cbb0 con 0x7f1aa4103f70 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.075+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41a3770 con 0x7f1aa4103f70 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.075+0000 7f1a9aa00700 1 -- 172.21.15.131:0/3451541193 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94018680 con 0x7f1aa4103f70 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.075+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41a3c80 con 0x7f1aa4103f70 2024-04-26T20:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.076+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa406a110 con 0x7f1aa4103f70 2024-04-26T20:06:20.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.078+0000 7f1a9aa00700 1 -- 172.21.15.131:0/3451541193 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1a94010430 con 0x7f1aa4103f70 2024-04-26T20:06:20.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.079+0000 7f1a9aa00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a880a3c40 0x7f1a880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:20.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.079+0000 7f1a9aa00700 1 -- 172.21.15.131:0/3451541193 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1a940d0190 con 0x7f1aa4103f70 2024-04-26T20:06:20.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.079+0000 7f1a9be00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a880a3c40 0x7f1a880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:20.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.080+0000 7f1a9be00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a880a3c40 0x7f1a880a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009c60 tx=0x7f1a8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:20.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.084+0000 7f1a9aa00700 1 -- 172.21.15.131:0/3451541193 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a9409cf70 con 0x7f1aa4103f70 2024-04-26T20:06:20.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.310+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1aa402e8d0 con 0x7f1aa4103f70 2024-04-26T20:06:20.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.311+0000 7f1a9aa00700 1 -- 172.21.15.131:0/3451541193 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f1aa402e8d0 con 0x7f1aa4103f70 2024-04-26T20:06:20.313 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.313+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a880a3c40 msgr2=0x7f1a880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.313+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a880a3c40 0x7f1a880a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009c60 tx=0x7f1a8c009260 comp rx=0 tx=0).stop 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.313+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa4103f70 msgr2=0x7f1aa4103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.313+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa4103f70 0x7f1aa4103880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e9a0 tx=0x7f1a9400ecb0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 shutdown_connections 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a880a3c40 0x7f1a880a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1aa41091d0 0x7f1aa4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa4103f70 0x7f1aa4103880 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 --2- 172.21.15.131:0/3451541193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1aa41068a0 0x7f1aa4101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 >> 172.21.15.131:0/3451541193 conn(0x7f1aa40fd920 msgr2=0x7f1aa41048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 shutdown_connections 2024-04-26T20:06:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:20.314+0000 7f1aa8c00700 1 -- 172.21.15.131:0/3451541193 wait complete. 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3451541193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: Detected new or changed devices on smithi157 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi157 to 4272M 2024-04-26T20:06:21.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.008 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:21.008 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:21.008 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:20 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.155 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714161978,"num_in_osds":12,"osd_in_since":1714161959,"num_remapped_pgs":0} 2024-04-26T20:06:21.156 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T20:06:21.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T20:06:21.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3451541193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T20:06:21.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: Detected new or changed devices on smithi157 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi157 to 4272M 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:21.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:20 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3451541193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: Detected new or changed devices on smithi157 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi157 to 4272M 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:21.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:21.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:20 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:21.597 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:06:22.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:21 smithi131 ceph-mon[27919]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:22.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:21 smithi157 ceph-mon[32703]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:22.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:21 smithi148 ceph-mon[32281]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:23.693 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:23 smithi131 ceph-mon[27919]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:23.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:23 smithi157 ceph-mon[32703]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:23.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.980+0000 7fb98c400700 1 -- 172.21.15.131:0/2264856836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075f70 msgr2=0x7fb98410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.980+0000 7fb98c400700 1 --2- 172.21.15.131:0/2264856836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075f70 0x7fb98410f8e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb9700098f0 tx=0x7fb970009c00 comp rx=0 tx=0).stop 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.980+0000 7fb98c400700 1 -- 172.21.15.131:0/2264856836 shutdown_connections 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.980+0000 7fb98c400700 1 --2- 172.21.15.131:0/2264856836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb98410fe20 0x7fb984112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.980+0000 7fb98c400700 1 --2- 172.21.15.131:0/2264856836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb984075650 0x7fb984075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.980+0000 7fb98c400700 1 --2- 172.21.15.131:0/2264856836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075f70 0x7fb98410f8e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.980+0000 7fb98c400700 1 -- 172.21.15.131:0/2264856836 >> 172.21.15.131:0/2264856836 conn(0x7fb98407a580 msgr2=0x7fb98407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.981+0000 7fb98c400700 1 -- 172.21.15.131:0/2264856836 shutdown_connections 2024-04-26T20:06:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.981+0000 7fb98c400700 1 -- 172.21.15.131:0/2264856836 wait complete. 2024-04-26T20:06:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.982+0000 7fb98c400700 1 Processor -- start 2024-04-26T20:06:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.982+0000 7fb98c400700 1 -- start start 2024-04-26T20:06:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb98c400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075650 0x7fb98419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb98c400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb984075f70 0x7fb98419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb98c400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb98410fe20 0x7fb9841a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb98c400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb984115850 con 0x7fb984075f70 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb98c400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841156d0 con 0x7fb984075650 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb98c400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841159d0 con 0x7fb98410fe20 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075650 0x7fb98419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075650 0x7fb98419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:37726/0 (socket says 172.21.15.131:37726) 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb984075f70 0x7fb98419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989a00700 1 -- 172.21.15.131:0/2445114960 learned_addr learned my addr 172.21.15.131:0/2445114960 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb98a400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb98410fe20 0x7fb9841a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989000700 1 -- 172.21.15.131:0/2445114960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb98410fe20 msgr2=0x7fb9841a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:23.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989000700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb98410fe20 0x7fb9841a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:23.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989000700 1 -- 172.21.15.131:0/2445114960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075650 msgr2=0x7fb98419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:23.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989000700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075650 0x7fb98419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:23.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989000700 1 -- 172.21.15.131:0/2445114960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9841a0d70 con 0x7fb984075f70 2024-04-26T20:06:23.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.983+0000 7fb989a00700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075650 0x7fb98419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:06:23.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.984+0000 7fb989000700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb984075f70 0x7fb98419c2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c010 tx=0x7fb97000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:23.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.984+0000 7fb982000700 1 -- 172.21.15.131:0/2445114960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9700050c0 con 0x7fb984075f70 2024-04-26T20:06:23.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.984+0000 7fb982000700 1 -- 172.21.15.131:0/2445114960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb97000b560 con 0x7fb984075f70 2024-04-26T20:06:23.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.984+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9841a1000 con 0x7fb984075f70 2024-04-26T20:06:23.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.984+0000 7fb982000700 1 -- 172.21.15.131:0/2445114960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970003a00 con 0x7fb984075f70 2024-04-26T20:06:23.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.984+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9841a88e0 con 0x7fb984075f70 2024-04-26T20:06:23.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.986+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98406a110 con 0x7fb984075f70 2024-04-26T20:06:23.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.988+0000 7fb982000700 1 -- 172.21.15.131:0/2445114960 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb97000b710 con 0x7fb984075f70 2024-04-26T20:06:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.989+0000 7fb982000700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb96c0a3c40 0x7fb96c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.989+0000 7fb982000700 1 -- 172.21.15.131:0/2445114960 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb9700d1080 con 0x7fb984075f70 2024-04-26T20:06:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.989+0000 7fb989a00700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb96c0a3c40 0x7fb96c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.991+0000 7fb989a00700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb96c0a3c40 0x7fb96c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb978005950 tx=0x7fb978009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:23.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:23.992+0000 7fb982000700 1 -- 172.21.15.131:0/2445114960 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb970013030 con 0x7fb984075f70 2024-04-26T20:06:24.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:23 smithi148 ceph-mon[32281]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:24.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.219+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb98402e490 con 0x7fb984075f70 2024-04-26T20:06:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.222+0000 7fb982000700 1 -- 172.21.15.131:0/2445114960 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18515 (secure 0 0 0) 0x7fb97000e3e0 con 0x7fb984075f70 2024-04-26T20:06:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:24.224 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:06:19.818466+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:06:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb96c0a3c40 msgr2=0x7fb96c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb96c0a3c40 0x7fb96c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb978005950 tx=0x7fb978009310 comp rx=0 tx=0).stop 2024-04-26T20:06:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb984075f70 msgr2=0x7fb98419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb984075f70 0x7fb98419c2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c010 tx=0x7fb97000fa00 comp rx=0 tx=0).stop 2024-04-26T20:06:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 shutdown_connections 2024-04-26T20:06:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb98410fe20 0x7fb9841a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb96c0a3c40 0x7fb96c0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984075650 0x7fb98419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 --2- 172.21.15.131:0/2445114960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb984075f70 0x7fb98419c2a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 >> 172.21.15.131:0/2445114960 conn(0x7fb98407a580 msgr2=0x7fb9841006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 shutdown_connections 2024-04-26T20:06:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:24.225+0000 7fb98c400700 1 -- 172.21.15.131:0/2445114960 wait complete. 2024-04-26T20:06:24.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:24 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2445114960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:06:25.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:24 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2445114960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:06:25.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:24 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2445114960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:06:25.158 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:06:25.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T20:06:25.628 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:06:25.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:25 smithi131 ceph-mon[27919]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:25.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:25 smithi157 ceph-mon[32703]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:26.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:25 smithi148 ceph-mon[32281]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:27.869 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:27 smithi131 ceph-mon[27919]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:27.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 -- 172.21.15.131:0/2203872425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c109a50 msgr2=0x7fd28c109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 --2- 172.21.15.131:0/2203872425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c109a50 0x7fd28c109ed0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd278009990 tx=0x7fd278009ca0 comp rx=0 tx=0).stop 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 -- 172.21.15.131:0/2203872425 shutdown_connections 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 --2- 172.21.15.131:0/2203872425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c10a410 0x7fd28c116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 --2- 172.21.15.131:0/2203872425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10fa50 0x7fd28c10fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 --2- 172.21.15.131:0/2203872425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c109a50 0x7fd28c109ed0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 -- 172.21.15.131:0/2203872425 >> 172.21.15.131:0/2203872425 conn(0x7fd28c076bf0 msgr2=0x7fd28c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 -- 172.21.15.131:0/2203872425 shutdown_connections 2024-04-26T20:06:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.897+0000 7fd291400700 1 -- 172.21.15.131:0/2203872425 wait complete. 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 Processor -- start 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 -- start start 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c109a50 0x7fd28c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10a410 0x7fd28c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c10fa50 0x7fd28c1a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c119c90 con 0x7fd28c10fa50 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c119b10 con 0x7fd28c109a50 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd291400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c119e10 con 0x7fd28c10a410 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.899+0000 7fd289e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10a410 0x7fd28c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd28a800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c109a50 0x7fd28c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10a410 0x7fd28c1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:49622/0 (socket says 172.21.15.131:49622) 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 -- 172.21.15.131:0/933713802 learned_addr learned my addr 172.21.15.131:0/933713802 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd28b200700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c10fa50 0x7fd28c1a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 -- 172.21.15.131:0/933713802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c109a50 msgr2=0x7fd28c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c109a50 0x7fd28c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 -- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c10fa50 msgr2=0x7fd28c1a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c10fa50 0x7fd28c1a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 -- 172.21.15.131:0/933713802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c1a5200 con 0x7fd28c10a410 2024-04-26T20:06:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.900+0000 7fd289e00700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10a410 0x7fd28c1a0730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd2780057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.901+0000 7fd283400700 1 -- 172.21.15.131:0/933713802 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278027070 con 0x7fd28c10a410 2024-04-26T20:06:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.901+0000 7fd283400700 1 -- 172.21.15.131:0/933713802 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd278022470 con 0x7fd28c10a410 2024-04-26T20:06:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.901+0000 7fd283400700 1 -- 172.21.15.131:0/933713802 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278017650 con 0x7fd28c10a410 2024-04-26T20:06:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.901+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1a5490 con 0x7fd28c10a410 2024-04-26T20:06:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.901+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1acd70 con 0x7fd28c10a410 2024-04-26T20:06:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.902+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c06a110 con 0x7fd28c10a410 2024-04-26T20:06:27.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.904+0000 7fd283400700 1 -- 172.21.15.131:0/933713802 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd27802a070 con 0x7fd28c10a410 2024-04-26T20:06:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.906+0000 7fd283400700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd2700a3cc0 0x7fd2700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.906+0000 7fd283400700 1 -- 172.21.15.131:0/933713802 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd2780d0c90 con 0x7fd28c10a410 2024-04-26T20:06:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.906+0000 7fd28a800700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd2700a3cc0 0x7fd2700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.907+0000 7fd28a800700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd2700a3cc0 0x7fd2700a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009680 tx=0x7fd27c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:27.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:27.907+0000 7fd283400700 1 -- 172.21.15.131:0/933713802 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd278013070 con 0x7fd28c10a410 2024-04-26T20:06:27.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:27 smithi157 ceph-mon[32703]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:28.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:27 smithi148 ceph-mon[32281]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.135+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd28c1a1550 con 0x7fd28c10a410 2024-04-26T20:06:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.135+0000 7fd283400700 1 -- 172.21.15.131:0/933713802 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fd27809db00 con 0x7fd28c10a410 2024-04-26T20:06:28.136 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:06:28.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.138+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd2700a3cc0 msgr2=0x7fd2700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.138+0000 7fd291400700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd2700a3cc0 0x7fd2700a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009680 tx=0x7fd27c009260 comp rx=0 tx=0).stop 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.138+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10a410 msgr2=0x7fd28c1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.138+0000 7fd291400700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10a410 0x7fd28c1a0730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd2780057c0 comp rx=0 tx=0).stop 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 shutdown_connections 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd28c10fa50 0x7fd28c1a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd2700a3cc0 0x7fd2700a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c109a50 0x7fd28c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 --2- 172.21.15.131:0/933713802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd28c10a410 0x7fd28c1a0730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 >> 172.21.15.131:0/933713802 conn(0x7fd28c076bf0 msgr2=0x7fd28c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 shutdown_connections 2024-04-26T20:06:28.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:28.139+0000 7fd291400700 1 -- 172.21.15.131:0/933713802 wait complete. 2024-04-26T20:06:28.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:28 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/933713802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:06:28.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:28 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/933713802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:06:29.030 INFO:tasks.cephadm:Adding mds.a on smithi131 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.d on smithi131 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.g on smithi131 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.j on smithi131 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.b on smithi148 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.e on smithi148 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.h on smithi148 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.k on smithi148 2024-04-26T20:06:29.031 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-04-26T20:06:29.032 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-04-26T20:06:29.032 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-04-26T20:06:29.032 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-04-26T20:06:29.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-04-26T20:06:29.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:28 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/933713802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:06:29.269 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:06:29.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:29 smithi157 ceph-mon[32703]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:30.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:29 smithi131 ceph-mon[27919]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:30.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:29 smithi148 ceph-mon[32281]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:31.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 -- 172.21.15.157:0/1438394264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394103570 msgr2=0x7f83941039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:31.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 --2- 172.21.15.157:0/1438394264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394103570 0x7f83941039d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8384009990 tx=0x7f8384009ca0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 -- 172.21.15.157:0/1438394264 shutdown_connections 2024-04-26T20:06:31.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 --2- 172.21.15.157:0/1438394264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8394103f10 0x7f839410a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 --2- 172.21.15.157:0/1438394264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8394102370 0x7f8394102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 --2- 172.21.15.157:0/1438394264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394103570 0x7f83941039d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 -- 172.21.15.157:0/1438394264 >> 172.21.15.157:0/1438394264 conn(0x7f83940fd910 msgr2=0x7f83940ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 -- 172.21.15.157:0/1438394264 shutdown_connections 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.052+0000 7f8398824700 1 -- 172.21.15.157:0/1438394264 wait complete. 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 Processor -- start 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 -- start start 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394102370 0x7f8394106290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8394103570 0x7f8394104950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8394103f10 0x7f8394104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f839410de60 con 0x7f8394103570 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f839410dce0 con 0x7f8394103f10 2024-04-26T20:06:31.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f8398824700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f839410dfe0 con 0x7f8394102370 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f839259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394102370 0x7f8394106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f839259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394102370 0x7f8394106290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59798/0 (socket says 172.21.15.157:59798) 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.053+0000 7f839259c700 1 -- 172.21.15.157:0/1534877802 learned_addr learned my addr 172.21.15.157:0/1534877802 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f839259c700 1 -- 172.21.15.157:0/1534877802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8394103f10 msgr2=0x7f8394104e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f8392d9d700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8394103f10 0x7f8394104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f839259c700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8394103f10 0x7f8394104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f839259c700 1 -- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8394103570 msgr2=0x7f8394104950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f839259c700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8394103570 0x7f8394104950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f839259c700 1 -- 172.21.15.157:0/1534877802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83941056d0 con 0x7f8394102370 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f839259c700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394102370 0x7f8394106290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f838800c970 tx=0x7f838800cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f83837fe700 1 -- 172.21.15.157:0/1534877802 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388004220 con 0x7f8394102370 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83941a3700 con 0x7f8394102370 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f83837fe700 1 -- 172.21.15.157:0/1534877802 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8388007d50 con 0x7f8394102370 2024-04-26T20:06:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f83837fe700 1 -- 172.21.15.157:0/1534877802 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388003cc0 con 0x7f8394102370 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.054+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83941a3c40 con 0x7f8394102370 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.055+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839406a110 con 0x7f8394102370 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.056+0000 7f83837fe700 1 -- 172.21.15.157:0/1534877802 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f83880043c0 con 0x7f8394102370 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.056+0000 7f83837fe700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f837c0a3c40 0x7f837c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.057+0000 7f83837fe700 1 -- 172.21.15.157:0/1534877802 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f83880cfbb0 con 0x7f8394102370 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.057+0000 7f8391d9b700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f837c0a3c40 0x7f837c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.058+0000 7f8391d9b700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f837c0a3c40 0x7f837c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f838400b5c0 tx=0x7f8384003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.061+0000 7f83837fe700 1 -- 172.21.15.157:0/1534877802 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8388014030 con 0x7f8394102370 2024-04-26T20:06:31.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.268+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) -- 0x7f83940ffbf0 con 0x7f837c0a3c40 2024-04-26T20:06:31.286 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.285+0000 7f83837fe700 1 -- 172.21.15.157:0/1534877802 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f83940ffbf0 con 0x7f837c0a3c40 2024-04-26T20:06:31.286 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-04-26T20:06:31.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f837c0a3c40 msgr2=0x7f837c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:31.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f837c0a3c40 0x7f837c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f838400b5c0 tx=0x7f8384003860 comp rx=0 tx=0).stop 2024-04-26T20:06:31.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394102370 msgr2=0x7f8394106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:31.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394102370 0x7f8394106290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f838800c970 tx=0x7f838800cd30 comp rx=0 tx=0).stop 2024-04-26T20:06:31.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 shutdown_connections 2024-04-26T20:06:31.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f837c0a3c40 0x7f837c0a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8394103f10 0x7f8394104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8394102370 0x7f8394106290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 --2- 172.21.15.157:0/1534877802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8394103570 0x7f8394104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:31.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 >> 172.21.15.157:0/1534877802 conn(0x7f83940fd910 msgr2=0x7f83940ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:31.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 shutdown_connections 2024-04-26T20:06:31.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-26T20:06:31.287+0000 7f8398824700 1 -- 172.21.15.157:0/1534877802 wait complete. 2024-04-26T20:06:32.090 DEBUG:teuthology.orchestra.run.smithi131:mds.a> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T20:06:32.097 DEBUG:teuthology.orchestra.run.smithi131:mds.d> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T20:06:32.143 DEBUG:teuthology.orchestra.run.smithi131:mds.g> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T20:06:32.189 DEBUG:teuthology.orchestra.run.smithi131:mds.j> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T20:06:32.234 DEBUG:teuthology.orchestra.run.smithi148:mds.b> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T20:06:32.237 DEBUG:teuthology.orchestra.run.smithi148:mds.e> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T20:06:32.281 DEBUG:teuthology.orchestra.run.smithi148:mds.h> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T20:06:32.324 DEBUG:teuthology.orchestra.run.smithi148:mds.k> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T20:06:32.366 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T20:06:32.369 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T20:06:32.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:32 smithi157 ceph-mon[32703]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:06:32.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:32 smithi157 ceph-mon[32703]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-26T20:06:32.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:32.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:32.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:32 smithi157 ceph-mon[32703]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:32.389 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T20:06:32.432 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T20:06:32.475 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T20:06:32.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:06:32.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:32 smithi131 ceph-mon[27919]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:06:32.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:32 smithi131 ceph-mon[27919]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-26T20:06:32.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:32.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:32.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:32 smithi131 ceph-mon[27919]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:32.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:32 smithi148 ceph-mon[32281]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:06:32.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:32 smithi148 ceph-mon[32281]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-26T20:06:32.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:32.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:32.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:32 smithi148 ceph-mon[32281]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 -- 172.21.15.131:0/1136937553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8108d90 msgr2=0x7f39c81091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 --2- 172.21.15.131:0/1136937553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8108d90 0x7f39c81091b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009860 tx=0x7f39b802a340 comp rx=0 tx=0).stop 2024-04-26T20:06:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 -- 172.21.15.131:0/1136937553 shutdown_connections 2024-04-26T20:06:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 --2- 172.21.15.131:0/1136937553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c810a940 0x7f39c81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 --2- 172.21.15.131:0/1136937553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8109f80 0x7f39c810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 --2- 172.21.15.131:0/1136937553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8108d90 0x7f39c81091b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 -- 172.21.15.131:0/1136937553 >> 172.21.15.131:0/1136937553 conn(0x7f39c8077d30 msgr2=0x7f39c807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:32.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 -- 172.21.15.131:0/1136937553 shutdown_connections 2024-04-26T20:06:32.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.692+0000 7f39cd000640 1 -- 172.21.15.131:0/1136937553 wait complete. 2024-04-26T20:06:32.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.693+0000 7f39cd000640 1 Processor -- start 2024-04-26T20:06:32.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.693+0000 7f39cd000640 1 -- start start 2024-04-26T20:06:32.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.693+0000 7f39cd000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8108d90 0x7f39c819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:32.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.693+0000 7f39cd000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8109f80 0x7f39c819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39cd000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c810a940 0x7f39c81a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39cd000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8114890 con 0x7f39c810a940 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39cd000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8114710 con 0x7f39c8109f80 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39cd000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8114a10 con 0x7f39c8108d90 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8108d90 0x7f39c819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c810a940 0x7f39c81a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8108d90 0x7f39c819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:49646/0 (socket says 172.21.15.131:49646) 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7400640 1 -- 172.21.15.131:0/1013017768 learned_addr learned my addr 172.21.15.131:0/1013017768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39bfe00640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8109f80 0x7f39c819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7e00640 1 -- 172.21.15.131:0/1013017768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8108d90 msgr2=0x7f39c819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7e00640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8108d90 0x7f39c819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7e00640 1 -- 172.21.15.131:0/1013017768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8109f80 msgr2=0x7f39c819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:32.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7e00640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8109f80 0x7f39c819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7e00640 1 -- 172.21.15.131:0/1013017768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c81a6bc0 con 0x7f39c810a940 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39c7400640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8108d90 0x7f39c819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.694+0000 7f39bfe00640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8109f80 0x7f39c819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.695+0000 7f39c7e00640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c810a940 0x7f39c81a6550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efc0 tx=0x7f39c000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.695+0000 7f39c4c00640 1 -- 172.21.15.131:0/1013017768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0003970 con 0x7f39c810a940 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.695+0000 7f39c4c00640 1 -- 172.21.15.131:0/1013017768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c0004020 con 0x7f39c810a940 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.695+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81a6eb0 con 0x7f39c810a940 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.695+0000 7f39c4c00640 1 -- 172.21.15.131:0/1013017768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c000b940 con 0x7f39c810a940 2024-04-26T20:06:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.695+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a7320 con 0x7f39c810a940 2024-04-26T20:06:32.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.696+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c8110270 con 0x7f39c810a940 2024-04-26T20:06:32.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.698+0000 7f39c4c00640 1 -- 172.21.15.131:0/1013017768 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f39c001d020 con 0x7f39c810a940 2024-04-26T20:06:32.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.699+0000 7f39c4c00640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39ac0a3bb0 0x7f39ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:32.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.700+0000 7f39c4c00640 1 -- 172.21.15.131:0/1013017768 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f39c00cfe90 con 0x7f39c810a940 2024-04-26T20:06:32.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.700+0000 7f39c7400640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39ac0a3bb0 0x7f39ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:32.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.700+0000 7f39c7400640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39ac0a3bb0 0x7f39ac0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39b80095f0 tx=0x7f39b8030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.704+0000 7f39c4c00640 1 -- 172.21.15.131:0/1013017768 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f39c009cc70 con 0x7f39c810a940 2024-04-26T20:06:32.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.906+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39c80635d0 con 0x7f39c810a940 2024-04-26T20:06:32.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.908+0000 7f39c4c00640 1 -- 172.21.15.131:0/1013017768 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18515 (secure 0 0 0) 0x7f39c009c570 con 0x7f39c810a940 2024-04-26T20:06:32.911 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:32.911 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:06:19.818466+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:06:32.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39ac0a3bb0 msgr2=0x7f39ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:32.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39ac0a3bb0 0x7f39ac0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39b80095f0 tx=0x7f39b8030450 comp rx=0 tx=0).stop 2024-04-26T20:06:32.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c810a940 msgr2=0x7f39c81a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:32.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c810a940 0x7f39c81a6550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efc0 tx=0x7f39c000c800 comp rx=0 tx=0).stop 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 shutdown_connections 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c810a940 0x7f39c81a6550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8109f80 0x7f39c819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39ac0a3bb0 0x7f39ac0a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 --2- 172.21.15.131:0/1013017768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c8108d90 0x7f39c819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 >> 172.21.15.131:0/1013017768 conn(0x7f39c8077d30 msgr2=0x7f39c810d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 shutdown_connections 2024-04-26T20:06:32.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:32.911+0000 7f39cd000640 1 -- 172.21.15.131:0/1013017768 wait complete. 2024-04-26T20:06:32.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:06:32.933 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:06:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 -- 172.21.15.131:0/1075169874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120075390 msgr2=0x7f5120075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 --2- 172.21.15.131:0/1075169874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120075390 0x7f5120075810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5108009860 tx=0x7f510802a340 comp rx=0 tx=0).stop 2024-04-26T20:06:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 -- 172.21.15.131:0/1075169874 shutdown_connections 2024-04-26T20:06:33.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 --2- 172.21.15.131:0/1075169874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f512010ee90 0x7f5120111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 --2- 172.21.15.131:0/1075169874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120075390 0x7f5120075810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 --2- 172.21.15.131:0/1075169874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5120076970 0x7f5120074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 -- 172.21.15.131:0/1075169874 >> 172.21.15.131:0/1075169874 conn(0x7f51200fde30 msgr2=0x7f5120100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:33.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 -- 172.21.15.131:0/1075169874 shutdown_connections 2024-04-26T20:06:33.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.182+0000 7f5124a00640 1 -- 172.21.15.131:0/1075169874 wait complete. 2024-04-26T20:06:33.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.183+0000 7f5124a00640 1 Processor -- start 2024-04-26T20:06:33.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.183+0000 7f5124a00640 1 -- start start 2024-04-26T20:06:33.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5124a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5120075390 0x7f512019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5124a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120076970 0x7f512019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5124a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512010ee90 0x7f51201a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5124a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5120114970 con 0x7f5120075390 2024-04-26T20:06:33.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5124a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51201147f0 con 0x7f512010ee90 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5124a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5120114af0 con 0x7f5120076970 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f511e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5120075390 0x7f512019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5115e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120076970 0x7f512019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5115e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120076970 0x7f512019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:49666/0 (socket says 172.21.15.131:49666) 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f5115e00640 1 -- 172.21.15.131:0/693995093 learned_addr learned my addr 172.21.15.131:0/693995093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.184+0000 7f511f200640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512010ee90 0x7f51201a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5115e00640 1 -- 172.21.15.131:0/693995093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512010ee90 msgr2=0x7f51201a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5115e00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512010ee90 0x7f51201a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5115e00640 1 -- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5120075390 msgr2=0x7f512019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5115e00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5120075390 0x7f512019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5115e00640 1 -- 172.21.15.131:0/693995093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51201a6ca0 con 0x7f5120076970 2024-04-26T20:06:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5115e00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120076970 0x7f512019f5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5108005940 tx=0x7f5108032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5117e00640 1 -- 172.21.15.131:0/693995093 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108040020 con 0x7f5120076970 2024-04-26T20:06:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51201a6f30 con 0x7f5120076970 2024-04-26T20:06:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5117e00640 1 -- 172.21.15.131:0/693995093 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5108030e80 con 0x7f5120076970 2024-04-26T20:06:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5117e00640 1 -- 172.21.15.131:0/693995093 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108037040 con 0x7f5120076970 2024-04-26T20:06:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.185+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51201a7390 con 0x7f5120076970 2024-04-26T20:06:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.186+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5120104f50 con 0x7f5120076970 2024-04-26T20:06:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.188+0000 7f5117e00640 1 -- 172.21.15.131:0/693995093 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5108043020 con 0x7f5120076970 2024-04-26T20:06:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.190+0000 7f5117e00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f51040a3bb0 0x7f51040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.190+0000 7f5117e00640 1 -- 172.21.15.131:0/693995093 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f51080efdf0 con 0x7f5120076970 2024-04-26T20:06:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.190+0000 7f511e800640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f51040a3bb0 0x7f51040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.190+0000 7f511e800640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f51040a3bb0 0x7f51040a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5110009bd0 tx=0x7f5110005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:33.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.194+0000 7f5117e00640 1 -- 172.21.15.131:0/693995093 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f510804a020 con 0x7f5120076970 2024-04-26T20:06:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: Deploying daemon mds.l on smithi157 2024-04-26T20:06:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:33 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1013017768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:06:33.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.394+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5120063530 con 0x7f5120076970 2024-04-26T20:06:33.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.395+0000 7f5117e00640 1 -- 172.21.15.131:0/693995093 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f51080328c0 con 0x7f5120076970 2024-04-26T20:06:33.396 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f51040a3bb0 msgr2=0x7f51040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f51040a3bb0 0x7f51040a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5110009bd0 tx=0x7f5110005f90 comp rx=0 tx=0).stop 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120076970 msgr2=0x7f512019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120076970 0x7f512019f5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5108005940 tx=0x7f5108032dc0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 shutdown_connections 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512010ee90 0x7f51201a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f51040a3bb0 0x7f51040a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5120076970 0x7f512019f5c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 --2- 172.21.15.131:0/693995093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5120075390 0x7f512019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 >> 172.21.15.131:0/693995093 conn(0x7f51200fde30 msgr2=0x7f5120102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 shutdown_connections 2024-04-26T20:06:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.398+0000 7f5124a00640 1 -- 172.21.15.131:0/693995093 wait complete. 2024-04-26T20:06:33.417 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: Deploying daemon mds.l on smithi157 2024-04-26T20:06:33.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:33 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1013017768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:06:33.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 -- 172.21.15.131:0/3406895711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 msgr2=0x7fa4001111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/3406895711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 0x7fa4001111d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009ca0 tx=0x7fa3f802a8c0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 -- 172.21.15.131:0/3406895711 shutdown_connections 2024-04-26T20:06:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/3406895711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 0x7fa4001111d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/3406895711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 0x7fa40010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/3406895711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa400108d90 0x7fa4001091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 -- 172.21.15.131:0/3406895711 >> 172.21.15.131:0/3406895711 conn(0x7fa400077d30 msgr2=0x7fa40007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 -- 172.21.15.131:0/3406895711 shutdown_connections 2024-04-26T20:06:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.620+0000 7fa3ffe00640 1 -- 172.21.15.131:0/3406895711 wait complete. 2024-04-26T20:06:33.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.621+0000 7fa3ffe00640 1 Processor -- start 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.621+0000 7fa3ffe00640 1 -- start start 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3ffe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa400108d90 0x7fa40019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3ffe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 0x7fa40019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3ffe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 0x7fa4001a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3ffe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4001148a0 con 0x7fa400108d90 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3ffe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa400114720 con 0x7fa40010a940 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3ffe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa400114a20 con 0x7fa400109f80 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 0x7fa40019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 0x7fa40019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:49694/0 (socket says 172.21.15.131:49694) 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 -- 172.21.15.131:0/2374675723 learned_addr learned my addr 172.21.15.131:0/2374675723 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3fea00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa400108d90 0x7fa40019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3ff400640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 0x7fa4001a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 -- 172.21.15.131:0/2374675723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 msgr2=0x7fa4001a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 0x7fa4001a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 -- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa400108d90 msgr2=0x7fa40019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa400108d90 0x7fa40019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.622+0000 7fa3f6000640 1 -- 172.21.15.131:0/2374675723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4001a6b40 con 0x7fa400109f80 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.623+0000 7fa3f6000640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 0x7fa40019f460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e9f0 tx=0x7fa3e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.623+0000 7fa3f7e00640 1 -- 172.21.15.131:0/2374675723 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800cd80 con 0x7fa400109f80 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.623+0000 7fa3f7e00640 1 -- 172.21.15.131:0/2374675723 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e80108e0 con 0x7fa400109f80 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.623+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4001a6e30 con 0x7fa400109f80 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.623+0000 7fa3f7e00640 1 -- 172.21.15.131:0/2374675723 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8018980 con 0x7fa400109f80 2024-04-26T20:06:33.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.623+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4001a7340 con 0x7fa400109f80 2024-04-26T20:06:33.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.624+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa400110180 con 0x7fa400109f80 2024-04-26T20:06:33.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.626+0000 7fa3f7e00640 1 -- 172.21.15.131:0/2374675723 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa3e8010430 con 0x7fa400109f80 2024-04-26T20:06:33.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.627+0000 7fa3f7e00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3e40a3c30 0x7fa3e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:33.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.628+0000 7fa3f7e00640 1 -- 172.21.15.131:0/2374675723 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa3e80d0260 con 0x7fa400109f80 2024-04-26T20:06:33.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.628+0000 7fa3fea00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3e40a3c30 0x7fa3e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:33.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.628+0000 7fa3fea00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3e40a3c30 0x7fa3e40a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa400109de0 tx=0x7fa3f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:33.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.632+0000 7fa3f7e00640 1 -- 172.21.15.131:0/2374675723 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa3e809c9f0 con 0x7fa400109f80 2024-04-26T20:06:33.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:33.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:33.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:33.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:33.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: Deploying daemon mds.l on smithi157 2024-04-26T20:06:33.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:33 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1013017768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:06:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.877+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4000635d0 con 0x7fa400109f80 2024-04-26T20:06:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.878+0000 7fa3f7e00640 1 -- 172.21.15.131:0/2374675723 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa3e80a0cf0 con 0x7fa400109f80 2024-04-26T20:06:33.879 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:33.879 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T20:06:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 1 2024-04-26T20:06:33.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3e40a3c30 msgr2=0x7fa3e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3e40a3c30 0x7fa3e40a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa400109de0 tx=0x7fa3f0005f90 comp rx=0 tx=0).stop 2024-04-26T20:06:33.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 msgr2=0x7fa40019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:33.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 0x7fa40019f460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e9f0 tx=0x7fa3e800eeb0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 shutdown_connections 2024-04-26T20:06:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40010a940 0x7fa4001a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3e40a3c30 0x7fa3e40a60f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa400109f80 0x7fa40019f460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 --2- 172.21.15.131:0/2374675723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa400108d90 0x7fa40019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.880+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 >> 172.21.15.131:0/2374675723 conn(0x7fa400077d30 msgr2=0x7fa40010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.881+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 shutdown_connections 2024-04-26T20:06:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:33.881+0000 7fa3ffe00640 1 -- 172.21.15.131:0/2374675723 wait complete. 2024-04-26T20:06:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:34 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/693995093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:06:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:34 smithi157 ceph-mon[32703]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:34.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:34 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2374675723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:34 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/693995093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:06:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:34 smithi148 ceph-mon[32281]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:34.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:34 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2374675723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:34.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:34 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/693995093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:06:34.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:34 smithi131 ceph-mon[27919]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:34.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:34 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2374675723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:34.947 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:34 smithi157 systemd[1]: Starting Ceph mds.l for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:06:35.221 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:34 smithi157 podman[62899]: 2024-04-26 20:06:34.850156768 +0000 UTC m=+0.017920525 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:06:35.221 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:34 smithi157 podman[62899]: 2024-04-26 20:06:34.976603653 +0000 UTC m=+0.144367400 container create d6aba73548fb02eb6fac9a895bb6d0716d066f39005433a2dc27fd3d70aa8b56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:06:35.221 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:35 smithi157 podman[62899]: 2024-04-26 20:06:35.220836986 +0000 UTC m=+0.388600736 container init d6aba73548fb02eb6fac9a895bb6d0716d066f39005433a2dc27fd3d70aa8b56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T20:06:35.480 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:35 smithi157 podman[62899]: 2024-04-26 20:06:35.223740055 +0000 UTC m=+0.391503810 container start d6aba73548fb02eb6fac9a895bb6d0716d066f39005433a2dc27fd3d70aa8b56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T20:06:35.480 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:35 smithi157 ceph-mds[62918]: -- 172.21.15.157:0/4117272574 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b13461a40 con 0x563b13435000 2024-04-26T20:06:35.480 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l[62914]: starting mds.l at 2024-04-26T20:06:35.480 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:35 smithi157 bash[62899]: d6aba73548fb02eb6fac9a895bb6d0716d066f39005433a2dc27fd3d70aa8b56 2024-04-26T20:06:35.480 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:06:35 smithi157 systemd[1]: Started Ceph mds.l for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:35.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:36.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 -- 172.21.15.131:0/1725999251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3241031f0 msgr2=0x7fa3241055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 --2- 172.21.15.131:0/1725999251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3241031f0 0x7fa3241055e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa318009900 tx=0x7fa31802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 -- 172.21.15.131:0/1725999251 shutdown_connections 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 --2- 172.21.15.131:0/1725999251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa324108450 0x7fa32410a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 --2- 172.21.15.131:0/1725999251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa324105b20 0x7fa324107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 --2- 172.21.15.131:0/1725999251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3241031f0 0x7fa3241055e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 -- 172.21.15.131:0/1725999251 >> 172.21.15.131:0/1725999251 conn(0x7fa3240fcbc0 msgr2=0x7fa3240ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 -- 172.21.15.131:0/1725999251 shutdown_connections 2024-04-26T20:06:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.114+0000 7fa32a600640 1 -- 172.21.15.131:0/1725999251 wait complete. 2024-04-26T20:06:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.115+0000 7fa32a600640 1 Processor -- start 2024-04-26T20:06:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.115+0000 7fa32a600640 1 -- start start 2024-04-26T20:06:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa32a600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3241031f0 0x7fa324198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa32a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324105b20 0x7fa324198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa32a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa324108450 0x7fa32419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa32a600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410e310 con 0x7fa3241031f0 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa32a600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410e190 con 0x7fa324105b20 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa32a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410e490 con 0x7fa324108450 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa329200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3241031f0 0x7fa324198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa329200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3241031f0 0x7fa324198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33636/0 (socket says 172.21.15.131:33636) 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa329200640 1 -- 172.21.15.131:0/1870466361 learned_addr learned my addr 172.21.15.131:0/1870466361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa323e00640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324105b20 0x7fa324198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa329c00640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa324108450 0x7fa32419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa329200640 1 -- 172.21.15.131:0/1870466361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa324108450 msgr2=0x7fa32419ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa329200640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa324108450 0x7fa32419ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.116+0000 7fa329200640 1 -- 172.21.15.131:0/1870466361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324105b20 msgr2=0x7fa324198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa329200640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324105b20 0x7fa324198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa329200640 1 -- 172.21.15.131:0/1870466361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3241a05b0 con 0x7fa3241031f0 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa323e00640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324105b20 0x7fa324198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa329200640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3241031f0 0x7fa324198990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa318005ab0 tx=0x7fa318003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa321600640 1 -- 172.21.15.131:0/1870466361 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31803f070 con 0x7fa3241031f0 2024-04-26T20:06:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241a0840 con 0x7fa3241031f0 2024-04-26T20:06:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa321600640 1 -- 172.21.15.131:0/1870466361 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa318031c50 con 0x7fa3241031f0 2024-04-26T20:06:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa321600640 1 -- 172.21.15.131:0/1870466361 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318035030 con 0x7fa3241031f0 2024-04-26T20:06:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.117+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a0d80 con 0x7fa3241031f0 2024-04-26T20:06:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.118+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3240fe6b0 con 0x7fa3241031f0 2024-04-26T20:06:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.120+0000 7fa321600640 1 -- 172.21.15.131:0/1870466361 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa318042070 con 0x7fa3241031f0 2024-04-26T20:06:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.122+0000 7fa321600640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa30c0a3bb0 0x7fa30c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.122+0000 7fa321600640 1 -- 172.21.15.131:0/1870466361 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa3180f07d0 con 0x7fa3241031f0 2024-04-26T20:06:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.122+0000 7fa323e00640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa30c0a3bb0 0x7fa30c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.123+0000 7fa323e00640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa30c0a3bb0 0x7fa30c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa324199ea0 tx=0x7fa310005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:36.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.126+0000 7fa321600640 1 -- 172.21.15.131:0/1870466361 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa3180bcf60 con 0x7fa3241031f0 2024-04-26T20:06:36.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.157:6832/1138993646,v1:172.21.15.157:6833/1138993646] up:boot 2024-04-26T20:06:36.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: fsmap 1 up:standby 2024-04-26T20:06:36.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: Deploying daemon mds.k on smithi148 2024-04-26T20:06:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:36 smithi148 ceph-mon[32281]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:36.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.371+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa32402dad0 con 0x7fa3241031f0 2024-04-26T20:06:36.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.372+0000 7fa321600640 1 -- 172.21.15.131:0/1870466361 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fa3180c1260 con 0x7fa3241031f0 2024-04-26T20:06:36.373 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:36.373 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T20:06:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 2 2024-04-26T20:06:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa30c0a3bb0 msgr2=0x7fa30c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa30c0a3bb0 0x7fa30c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa324199ea0 tx=0x7fa310005f90 comp rx=0 tx=0).stop 2024-04-26T20:06:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3241031f0 msgr2=0x7fa324198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3241031f0 0x7fa324198990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa318005ab0 tx=0x7fa318003b90 comp rx=0 tx=0).stop 2024-04-26T20:06:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 shutdown_connections 2024-04-26T20:06:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa30c0a3bb0 0x7fa30c0a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa324108450 0x7fa32419ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324105b20 0x7fa324198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 --2- 172.21.15.131:0/1870466361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3241031f0 0x7fa324198990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.374+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 >> 172.21.15.131:0/1870466361 conn(0x7fa3240fcbc0 msgr2=0x7fa324103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.375+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 shutdown_connections 2024-04-26T20:06:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:36.375+0000 7fa32a600640 1 -- 172.21.15.131:0/1870466361 wait complete. 2024-04-26T20:06:36.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.157:6832/1138993646,v1:172.21.15.157:6833/1138993646] up:boot 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: fsmap 1 up:standby 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: Deploying daemon mds.k on smithi148 2024-04-26T20:06:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:36 smithi157 ceph-mon[32703]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.157:6832/1138993646,v1:172.21.15.157:6833/1138993646] up:boot 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: fsmap 1 up:standby 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: Deploying daemon mds.k on smithi148 2024-04-26T20:06:36.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:36 smithi131 ceph-mon[27919]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:37.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:37 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1870466361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:37.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:37 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1870466361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:37 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1870466361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:37.817 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:37 smithi148 systemd[1]: Starting Ceph mds.k for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:06:38.186 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:37 smithi148 podman[62040]: 2024-04-26 20:06:37.81186533 +0000 UTC m=+0.018289401 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:06:38.187 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:37 smithi148 podman[62040]: 2024-04-26 20:06:37.95039282 +0000 UTC m=+0.156816866 container create fbec80a25b19be1f692d062d9e65d3eadf53f54b2df6a663211358aac35220c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:06:38.187 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:38 smithi148 podman[62040]: 2024-04-26 20:06:38.186470733 +0000 UTC m=+0.392894782 container init fbec80a25b19be1f692d062d9e65d3eadf53f54b2df6a663211358aac35220c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T20:06:38.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:38.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:38 smithi148 ceph-mon[32281]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:38.567 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:38 smithi148 podman[62040]: 2024-04-26 20:06:38.18909118 +0000 UTC m=+0.395515227 container start fbec80a25b19be1f692d062d9e65d3eadf53f54b2df6a663211358aac35220c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T20:06:38.567 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:38 smithi148 ceph-mds[62059]: -- 172.21.15.148:0/3992102755 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593cc9a1a40 con 0x5593cc974800 2024-04-26T20:06:38.567 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k[62055]: starting mds.k at 2024-04-26T20:06:38.567 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:38 smithi148 bash[62040]: fbec80a25b19be1f692d062d9e65d3eadf53f54b2df6a663211358aac35220c1 2024-04-26T20:06:38.567 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:06:38 smithi148 systemd[1]: Started Ceph mds.k for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.587+0000 7f9119600640 1 -- 172.21.15.131:0/1738784446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9114104360 msgr2=0x7f911410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.587+0000 7f9119600640 1 --2- 172.21.15.131:0/1738784446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9114104360 0x7f911410abf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f910c009ca0 tx=0x7f910c02a8c0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.588+0000 7f9119600640 1 -- 172.21.15.131:0/1738784446 shutdown_connections 2024-04-26T20:06:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.588+0000 7f9119600640 1 --2- 172.21.15.131:0/1738784446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9114104360 0x7f911410abf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.588+0000 7f9119600640 1 --2- 172.21.15.131:0/1738784446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91141039a0 0x7f9114103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.588+0000 7f9119600640 1 --2- 172.21.15.131:0/1738784446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91141027b0 0x7f9114102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.588+0000 7f9119600640 1 -- 172.21.15.131:0/1738784446 >> 172.21.15.131:0/1738784446 conn(0x7f91140fdd40 msgr2=0x7f9114100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.588+0000 7f9119600640 1 -- 172.21.15.131:0/1738784446 shutdown_connections 2024-04-26T20:06:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.588+0000 7f9119600640 1 -- 172.21.15.131:0/1738784446 wait complete. 2024-04-26T20:06:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 Processor -- start 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 -- start start 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91141027b0 0x7f911419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91141039a0 0x7f911419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9114104360 0x7f91141a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e310 con 0x7f91141039a0 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e190 con 0x7f91141027b0 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9119600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e490 con 0x7f9114104360 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9113400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91141039a0 0x7f911419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9113e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91141027b0 0x7f911419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9113400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91141039a0 0x7f911419b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33658/0 (socket says 172.21.15.131:33658) 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.589+0000 7f9113400640 1 -- 172.21.15.131:0/2193221765 learned_addr learned my addr 172.21.15.131:0/2193221765 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9118c00640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9114104360 0x7f91141a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9113e00640 1 -- 172.21.15.131:0/2193221765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9114104360 msgr2=0x7f91141a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9113e00640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9114104360 0x7f91141a20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9113e00640 1 -- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91141039a0 msgr2=0x7f911419b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9113e00640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91141039a0 0x7f911419b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9113e00640 1 -- 172.21.15.131:0/2193221765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141a2710 con 0x7f91141027b0 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9113e00640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91141027b0 0x7f911419aaf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f910400e9c0 tx=0x7f910400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9110c00640 1 -- 172.21.15.131:0/2193221765 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910400ccc0 con 0x7f91141027b0 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9110c00640 1 -- 172.21.15.131:0/2193221765 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91040035f0 con 0x7f91141027b0 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a2a00 con 0x7f91141027b0 2024-04-26T20:06:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.590+0000 7f9110c00640 1 -- 172.21.15.131:0/2193221765 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910400ce60 con 0x7f91141027b0 2024-04-26T20:06:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.591+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a2f50 con 0x7f91141027b0 2024-04-26T20:06:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.592+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9114069670 con 0x7f91141027b0 2024-04-26T20:06:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.593+0000 7f9110c00640 1 -- 172.21.15.131:0/2193221765 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9104004000 con 0x7f91141027b0 2024-04-26T20:06:38.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.595+0000 7f9110c00640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f90f80a3bb0 0x7f90f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:38.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.595+0000 7f9110c00640 1 -- 172.21.15.131:0/2193221765 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f91040d0420 con 0x7f91141027b0 2024-04-26T20:06:38.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.595+0000 7f9113400640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f90f80a3bb0 0x7f90f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:38.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.596+0000 7f9113400640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f90f80a3bb0 0x7f90f80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f911419c000 tx=0x7f9100005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:38.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.599+0000 7f9110c00640 1 -- 172.21.15.131:0/2193221765 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f910409d200 con 0x7f91141027b0 2024-04-26T20:06:38.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:38 smithi157 ceph-mon[32703]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:38.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:38 smithi131 ceph-mon[27919]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:38.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.844+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91141a31e0 con 0x7f91141027b0 2024-04-26T20:06:38.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.845+0000 7f9110c00640 1 -- 172.21.15.131:0/2193221765 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f910409cb00 con 0x7f91141027b0 2024-04-26T20:06:38.846 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:38.846 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:06:38.846 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f90f80a3bb0 msgr2=0x7f90f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f90f80a3bb0 0x7f90f80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f911419c000 tx=0x7f9100005f90 comp rx=0 tx=0).stop 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91141027b0 msgr2=0x7f911419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91141027b0 0x7f911419aaf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f910400e9c0 tx=0x7f910400ee80 comp rx=0 tx=0).stop 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 shutdown_connections 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f90f80a3bb0 0x7f90f80a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9114104360 0x7f91141a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91141039a0 0x7f911419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 --2- 172.21.15.131:0/2193221765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91141027b0 0x7f911419aaf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 >> 172.21.15.131:0/2193221765 conn(0x7f91140fdd40 msgr2=0x7f9114074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 shutdown_connections 2024-04-26T20:06:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:38.849+0000 7f9119600640 1 -- 172.21.15.131:0/2193221765 wait complete. 2024-04-26T20:06:39.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.148:6832/248659002,v1:172.21.15.148:6833/248659002] up:boot 2024-04-26T20:06:39.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: fsmap 2 up:standby 2024-04-26T20:06:39.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T20:06:39.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:39.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:39.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:39.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: Deploying daemon mds.j on smithi131 2024-04-26T20:06:39.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:39 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2193221765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:39.574 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.148:6832/248659002,v1:172.21.15.148:6833/248659002] up:boot 2024-04-26T20:06:39.574 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: fsmap 2 up:standby 2024-04-26T20:06:39.574 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: Deploying daemon mds.j on smithi131 2024-04-26T20:06:39.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:39 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2193221765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:39.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.148:6832/248659002,v1:172.21.15.148:6833/248659002] up:boot 2024-04-26T20:06:39.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: fsmap 2 up:standby 2024-04-26T20:06:39.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T20:06:39.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: Deploying daemon mds.j on smithi131 2024-04-26T20:06:39.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:39 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2193221765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:40.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:40 smithi148 ceph-mon[32281]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:40.586 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:40 smithi131 ceph-mon[27919]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:40.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:40 smithi157 ceph-mon[32703]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:40.875 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:41.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.195+0000 7f0565600640 1 -- 172.21.15.131:0/3283351415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 msgr2=0x7f05601080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:41.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.196+0000 7f0565600640 1 --2- 172.21.15.131:0/3283351415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 0x7f05601080f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f054c009900 tx=0x7f054c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.196+0000 7f0565600640 1 -- 172.21.15.131:0/3283351415 shutdown_connections 2024-04-26T20:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.196+0000 7f0565600640 1 --2- 172.21.15.131:0/3283351415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0560108630 0x7f056010aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.196+0000 7f0565600640 1 --2- 172.21.15.131:0/3283351415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 0x7f05601080f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.197+0000 7f0565600640 1 --2- 172.21.15.131:0/3283351415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05601033d0 0x7f05601057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.197+0000 7f0565600640 1 -- 172.21.15.131:0/3283351415 >> 172.21.15.131:0/3283351415 conn(0x7f05600fcdc0 msgr2=0x7f05600ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.197+0000 7f0565600640 1 -- 172.21.15.131:0/3283351415 shutdown_connections 2024-04-26T20:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.197+0000 7f0565600640 1 -- 172.21.15.131:0/3283351415 wait complete. 2024-04-26T20:06:41.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.198+0000 7f0565600640 1 Processor -- start 2024-04-26T20:06:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.199+0000 7f0565600640 1 -- start start 2024-04-26T20:06:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.199+0000 7f0565600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05601033d0 0x7f056019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.199+0000 7f0565600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 0x7f056019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.199+0000 7f0565600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0560108630 0x7f05601a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.199+0000 7f0565600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0560102c90 con 0x7f05601033d0 2024-04-26T20:06:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.199+0000 7f0565600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0560102b10 con 0x7f0560108630 2024-04-26T20:06:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.199+0000 7f0565600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0560102e10 con 0x7f0560105d00 2024-04-26T20:06:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.201+0000 7f055ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05601033d0 0x7f056019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.201+0000 7f055e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 0x7f056019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.201+0000 7f055f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0560108630 0x7f05601a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.201+0000 7f055f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0560108630 0x7f05601a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:52678/0 (socket says 172.21.15.131:52678) 2024-04-26T20:06:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.201+0000 7f055f400640 1 -- 172.21.15.131:0/3267087127 learned_addr learned my addr 172.21.15.131:0/3267087127 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.202+0000 7f055ea00640 1 -- 172.21.15.131:0/3267087127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 msgr2=0x7f056019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.202+0000 7f055ea00640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 0x7f056019f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.202+0000 7f055ea00640 1 -- 172.21.15.131:0/3267087127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0560108630 msgr2=0x7f05601a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.202+0000 7f055ea00640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0560108630 0x7f05601a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.202+0000 7f055ea00640 1 -- 172.21.15.131:0/3267087127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0560110f20 con 0x7f05601033d0 2024-04-26T20:06:41.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.203+0000 7f055ea00640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05601033d0 0x7f056019efe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9f0 tx=0x7f055000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:41.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.204+0000 7f0557400640 1 -- 172.21.15.131:0/3267087127 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000ccc0 con 0x7f05601033d0 2024-04-26T20:06:41.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.204+0000 7f0557400640 1 -- 172.21.15.131:0/3267087127 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0550005020 con 0x7f05601033d0 2024-04-26T20:06:41.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.204+0000 7f0557400640 1 -- 172.21.15.131:0/3267087127 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550004a00 con 0x7f05601033d0 2024-04-26T20:06:41.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.204+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0560111210 con 0x7f05601033d0 2024-04-26T20:06:41.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.205+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05601116e0 con 0x7f05601033d0 2024-04-26T20:06:41.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.206+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05600697c0 con 0x7f05601033d0 2024-04-26T20:06:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.211+0000 7f0557400640 1 -- 172.21.15.131:0/3267087127 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0550004ba0 con 0x7f05601033d0 2024-04-26T20:06:41.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.214+0000 7f0557400640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f05440a3d00 0x7f05440a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:41.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.214+0000 7f0557400640 1 -- 172.21.15.131:0/3267087127 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f05500cffc0 con 0x7f05601033d0 2024-04-26T20:06:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.214+0000 7f055e000640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f05440a3d00 0x7f05440a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.215+0000 7f0557400640 1 -- 172.21.15.131:0/3267087127 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f055009ceb0 con 0x7f05601033d0 2024-04-26T20:06:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.215+0000 7f055e000640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f05440a3d00 0x7f05440a61c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f054c02a910 tx=0x7f054c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:41.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.464+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0560063530 con 0x7f05601033d0 2024-04-26T20:06:41.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.464+0000 7f0557400640 1 -- 172.21.15.131:0/3267087127 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f055009c7b0 con 0x7f05601033d0 2024-04-26T20:06:41.465 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:41.465 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:06:41.466 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f05440a3d00 msgr2=0x7f05440a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f05440a3d00 0x7f05440a61c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f054c02a910 tx=0x7f054c000bc0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05601033d0 msgr2=0x7f056019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05601033d0 0x7f056019efe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9f0 tx=0x7f055000eeb0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 shutdown_connections 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0560108630 0x7f05601a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f05440a3d00 0x7f05440a61c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0560105d00 0x7f056019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 --2- 172.21.15.131:0/3267087127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05601033d0 0x7f056019efe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 >> 172.21.15.131:0/3267087127 conn(0x7f05600fcdc0 msgr2=0x7f0560103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 shutdown_connections 2024-04-26T20:06:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:41.467+0000 7f0565600640 1 -- 172.21.15.131:0/3267087127 wait complete. 2024-04-26T20:06:41.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:41 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3267087127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:41 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3267087127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:41.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:41 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3267087127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:42.087 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:41 smithi131 podman[67941]: 2024-04-26 20:06:41.804244874 +0000 UTC m=+0.144289235 container create c68ace96567d22b1633ed884d9e6ba9326b755cd1232e876763072f0787a062c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T20:06:42.088 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:42 smithi131 podman[67941]: 2024-04-26 20:06:42.086125335 +0000 UTC m=+0.426169700 container init c68ace96567d22b1633ed884d9e6ba9326b755cd1232e876763072f0787a062c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T20:06:42.506 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:42 smithi131 podman[67941]: 2024-04-26 20:06:42.097688997 +0000 UTC m=+0.437733364 container start c68ace96567d22b1633ed884d9e6ba9326b755cd1232e876763072f0787a062c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:06:42.506 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mds[67960]: -- 172.21.15.131:0/2332286635 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56227770c4e0 con 0x5622776e2800 2024-04-26T20:06:42.507 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mds[67960]: -- 172.21.15.131:0/2332286635 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56227770fa40 con 0x5622776e2800 2024-04-26T20:06:42.507 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j[67956]: starting mds.j at 2024-04-26T20:06:42.507 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:42 smithi131 bash[67941]: c68ace96567d22b1633ed884d9e6ba9326b755cd1232e876763072f0787a062c 2024-04-26T20:06:42.507 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:06:42 smithi131 systemd[1]: Started Ceph mds.j for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:42.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:42 smithi148 ceph-mon[32281]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:42.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:42 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:42.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:42 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:42.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:42 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:42.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:42 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:42.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:42 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:42.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:42 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:42.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:42 smithi157 ceph-mon[32703]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:42.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:42 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:42.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:42 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:42.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:42 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:42.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:42 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:42.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:42 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:42.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:42 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:43.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mon[27919]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:43.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:43.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:43.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:43.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:43.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:43.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:42 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:43.486 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 -- 172.21.15.131:0/1571466814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 msgr2=0x7ff1401091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 --2- 172.21.15.131:0/1571466814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff1401091b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009900 tx=0x7ff12c02a410 comp rx=0 tx=0).stop 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 -- 172.21.15.131:0/1571466814 shutdown_connections 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 --2- 172.21.15.131:0/1571466814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14010a940 0x7ff1401111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 --2- 172.21.15.131:0/1571466814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff140109f80 0x7ff14010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 --2- 172.21.15.131:0/1571466814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff1401091b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 -- 172.21.15.131:0/1571466814 >> 172.21.15.131:0/1571466814 conn(0x7ff140077d30 msgr2=0x7ff14007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 -- 172.21.15.131:0/1571466814 shutdown_connections 2024-04-26T20:06:43.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.691+0000 7ff144a00640 1 -- 172.21.15.131:0/1571466814 wait complete. 2024-04-26T20:06:43.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 Processor -- start 2024-04-26T20:06:43.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 -- start start 2024-04-26T20:06:43.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff14019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:43.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff140109f80 0x7ff14019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:43.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff14010a940 0x7ff1401a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:43.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401148e0 con 0x7ff140108d90 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff140114760 con 0x7ff14010a940 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff144a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff140114a60 con 0x7ff140109f80 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff13f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff14019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff13f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff14019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33714/0 (socket says 172.21.15.131:33714) 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff13f400640 1 -- 172.21.15.131:0/2146654317 learned_addr learned my addr 172.21.15.131:0/2146654317 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff13fe00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff14010a940 0x7ff1401a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.692+0000 7ff13ea00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff140109f80 0x7ff14019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff13f400640 1 -- 172.21.15.131:0/2146654317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff140109f80 msgr2=0x7ff14019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff13f400640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff140109f80 0x7ff14019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff13f400640 1 -- 172.21.15.131:0/2146654317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff14010a940 msgr2=0x7ff1401a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:43.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff13f400640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff14010a940 0x7ff1401a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff13f400640 1 -- 172.21.15.131:0/2146654317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1401a6bd0 con 0x7ff140108d90 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff13f400640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff14019efb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff12c031500 tx=0x7ff12c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff137e00640 1 -- 172.21.15.131:0/2146654317 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c03f070 con 0x7ff140108d90 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff137e00640 1 -- 172.21.15.131:0/2146654317 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c031c50 con 0x7ff140108d90 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff137e00640 1 -- 172.21.15.131:0/2146654317 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c035030 con 0x7ff140108d90 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.693+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1401a6e60 con 0x7ff140108d90 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.694+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401a72c0 con 0x7ff140108d90 2024-04-26T20:06:43.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.695+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff140110180 con 0x7ff140108d90 2024-04-26T20:06:43.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.696+0000 7ff137e00640 1 -- 172.21.15.131:0/2146654317 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff12c042070 con 0x7ff140108d90 2024-04-26T20:06:43.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.698+0000 7ff137e00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1240a3c30 0x7ff1240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:43.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.698+0000 7ff137e00640 1 -- 172.21.15.131:0/2146654317 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff12c0f0650 con 0x7ff140108d90 2024-04-26T20:06:43.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.698+0000 7ff13ea00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1240a3c30 0x7ff1240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:43.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.699+0000 7ff13ea00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1240a3c30 0x7ff1240a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1401a04c0 tx=0x7ff130005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:43.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.703+0000 7ff137e00640 1 -- 172.21.15.131:0/2146654317 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff12c0bcde0 con 0x7ff140108d90 2024-04-26T20:06:43.747 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:43 smithi131 ceph-mon[27919]: Deploying daemon mds.d on smithi131 2024-04-26T20:06:43.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:43 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6834/3820661643,v1:172.21.15.131:6835/3820661643] up:boot 2024-04-26T20:06:43.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:43 smithi131 ceph-mon[27919]: fsmap 3 up:standby 2024-04-26T20:06:43.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:43 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T20:06:43.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:43 smithi148 ceph-mon[32281]: Deploying daemon mds.d on smithi131 2024-04-26T20:06:43.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:43 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6834/3820661643,v1:172.21.15.131:6835/3820661643] up:boot 2024-04-26T20:06:43.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:43 smithi148 ceph-mon[32281]: fsmap 3 up:standby 2024-04-26T20:06:43.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:43 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T20:06:43.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.959+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff140063680 con 0x7ff140108d90 2024-04-26T20:06:43.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.960+0000 7ff137e00640 1 -- 172.21.15.131:0/2146654317 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7ff12c0c10e0 con 0x7ff140108d90 2024-04-26T20:06:43.962 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:43.962 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:06:43.962 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 4 2024-04-26T20:06:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1240a3c30 msgr2=0x7ff1240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1240a3c30 0x7ff1240a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1401a04c0 tx=0x7ff130005f90 comp rx=0 tx=0).stop 2024-04-26T20:06:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 msgr2=0x7ff14019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff14019efb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff12c031500 tx=0x7ff12c003c10 comp rx=0 tx=0).stop 2024-04-26T20:06:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 shutdown_connections 2024-04-26T20:06:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff14010a940 0x7ff1401a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1240a3c30 0x7ff1240a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff140109f80 0x7ff14019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 --2- 172.21.15.131:0/2146654317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff140108d90 0x7ff14019efb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 >> 172.21.15.131:0/2146654317 conn(0x7ff140077d30 msgr2=0x7ff14010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 shutdown_connections 2024-04-26T20:06:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:43.963+0000 7ff144a00640 1 -- 172.21.15.131:0/2146654317 wait complete. 2024-04-26T20:06:43.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:43 smithi157 ceph-mon[32703]: Deploying daemon mds.d on smithi131 2024-04-26T20:06:43.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:43 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6834/3820661643,v1:172.21.15.131:6835/3820661643] up:boot 2024-04-26T20:06:43.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:43 smithi157 ceph-mon[32703]: fsmap 3 up:standby 2024-04-26T20:06:43.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:43 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T20:06:44.693 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:44 smithi131 ceph-mon[27919]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:44.694 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:44 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2146654317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:44.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:44 smithi148 ceph-mon[32281]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:44.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:44 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2146654317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:44.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:44 smithi157 ceph-mon[32703]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:44.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:44 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2146654317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:45.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:45 smithi131 ceph-mon[27919]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:45.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:45 smithi157 ceph-mon[32703]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:45.984 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:46.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:45 smithi148 ceph-mon[32281]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:46.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 -- 172.21.15.131:0/2097636562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e241032b0 msgr2=0x7f8e241056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:46.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 --2- 172.21.15.131:0/2097636562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e241032b0 0x7f8e241056a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009900 tx=0x7f8e1402a400 comp rx=0 tx=0).stop 2024-04-26T20:06:46.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 -- 172.21.15.131:0/2097636562 shutdown_connections 2024-04-26T20:06:46.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 --2- 172.21.15.131:0/2097636562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e24108510 0x7f8e2410a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 --2- 172.21.15.131:0/2097636562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e24105be0 0x7f8e24107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 --2- 172.21.15.131:0/2097636562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e241032b0 0x7f8e241056a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 -- 172.21.15.131:0/2097636562 >> 172.21.15.131:0/2097636562 conn(0x7f8e240fcbc0 msgr2=0x7f8e240ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:46.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 -- 172.21.15.131:0/2097636562 shutdown_connections 2024-04-26T20:06:46.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.195+0000 7f8e23200640 1 -- 172.21.15.131:0/2097636562 wait complete. 2024-04-26T20:06:46.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.196+0000 7f8e23200640 1 Processor -- start 2024-04-26T20:06:46.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.196+0000 7f8e23200640 1 -- start start 2024-04-26T20:06:46.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e23200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e241032b0 0x7f8e2419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:46.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e23200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e24105be0 0x7f8e2419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e23200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e24108510 0x7f8e241a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e23200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410e2f0 con 0x7f8e24105be0 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e23200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410e170 con 0x7f8e241032b0 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e23200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410e470 con 0x7f8e24108510 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e241032b0 0x7f8e2419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e241032b0 0x7f8e2419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:33244/0 (socket says 172.21.15.131:33244) 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 -- 172.21.15.131:0/3320692560 learned_addr learned my addr 172.21.15.131:0/3320692560 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21400640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e24105be0 0x7f8e2419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e22800640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e24108510 0x7f8e241a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 -- 172.21.15.131:0/3320692560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e24108510 msgr2=0x7f8e241a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e24108510 0x7f8e241a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 -- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e24105be0 msgr2=0x7f8e2419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e24105be0 0x7f8e2419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e21e00640 1 -- 172.21.15.131:0/3320692560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e241a2800 con 0x7f8e241032b0 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.197+0000 7f8e22800640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e24108510 0x7f8e241a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.198+0000 7f8e21e00640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e241032b0 0x7f8e2419abe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005ab0 tx=0x7f8e14003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.198+0000 7f8e1aa00640 1 -- 172.21.15.131:0/3320692560 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1403f070 con 0x7f8e241032b0 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.198+0000 7f8e1aa00640 1 -- 172.21.15.131:0/3320692560 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e14031c50 con 0x7f8e241032b0 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.198+0000 7f8e1aa00640 1 -- 172.21.15.131:0/3320692560 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14035030 con 0x7f8e241032b0 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.198+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e241a2a90 con 0x7f8e241032b0 2024-04-26T20:06:46.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.198+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e241a2fd0 con 0x7f8e241032b0 2024-04-26T20:06:46.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.200+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2410a000 con 0x7f8e241032b0 2024-04-26T20:06:46.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.201+0000 7f8e1aa00640 1 -- 172.21.15.131:0/3320692560 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8e14042070 con 0x7f8e241032b0 2024-04-26T20:06:46.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.203+0000 7f8e1aa00640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e080a3c30 0x7f8e080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:46.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.203+0000 7f8e1aa00640 1 -- 172.21.15.131:0/3320692560 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8e140f0710 con 0x7f8e241032b0 2024-04-26T20:06:46.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.203+0000 7f8e21400640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e080a3c30 0x7f8e080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:46.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.204+0000 7f8e21400640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e080a3c30 0x7f8e080a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e100095f0 tx=0x7f8e10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:46.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.207+0000 7f8e1aa00640 1 -- 172.21.15.131:0/3320692560 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8e140bd4f0 con 0x7f8e241032b0 2024-04-26T20:06:46.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.462+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e2402dad0 con 0x7f8e241032b0 2024-04-26T20:06:46.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.463+0000 7f8e1aa00640 1 -- 172.21.15.131:0/3320692560 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f8e140bcdf0 con 0x7f8e241032b0 2024-04-26T20:06:46.463 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:46.463 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:06:46.464 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 4 2024-04-26T20:06:46.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e080a3c30 msgr2=0x7f8e080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:46.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e080a3c30 0x7f8e080a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e100095f0 tx=0x7f8e10005f90 comp rx=0 tx=0).stop 2024-04-26T20:06:46.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e241032b0 msgr2=0x7f8e2419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:46.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e241032b0 0x7f8e2419abe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005ab0 tx=0x7f8e14003b90 comp rx=0 tx=0).stop 2024-04-26T20:06:46.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 shutdown_connections 2024-04-26T20:06:46.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e080a3c30 0x7f8e080a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e24108510 0x7f8e241a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e24105be0 0x7f8e2419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 --2- 172.21.15.131:0/3320692560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e241032b0 0x7f8e2419abe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 >> 172.21.15.131:0/3320692560 conn(0x7f8e240fcbc0 msgr2=0x7f8e24102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 shutdown_connections 2024-04-26T20:06:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:46.466+0000 7f8e23200640 1 -- 172.21.15.131:0/3320692560 wait complete. 2024-04-26T20:06:46.484 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:06:46 smithi131 systemd[1]: Started Ceph mds.d for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:47.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: Deploying daemon mds.e on smithi148 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3320692560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6836/3688291148,v1:172.21.15.131:6837/3688291148] up:boot 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: fsmap 4 up:standby 2024-04-26T20:06:47.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:47 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T20:06:47.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: Deploying daemon mds.e on smithi148 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3320692560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6836/3688291148,v1:172.21.15.131:6837/3688291148] up:boot 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: fsmap 4 up:standby 2024-04-26T20:06:47.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:47 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: Deploying daemon mds.e on smithi148 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3320692560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:47.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6836/3688291148,v1:172.21.15.131:6837/3688291148] up:boot 2024-04-26T20:06:47.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: fsmap 4 up:standby 2024-04-26T20:06:47.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:47 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T20:06:48.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:48 smithi148 ceph-mon[32281]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:48.488 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:48.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 -- 172.21.15.131:0/2495212446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 msgr2=0x7f38f4105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:48.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 --2- 172.21.15.131:0/2495212446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 0x7f38f4105ff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009860 tx=0x7f38dc02a340 comp rx=0 tx=0).stop 2024-04-26T20:06:48.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 -- 172.21.15.131:0/2495212446 shutdown_connections 2024-04-26T20:06:48.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 --2- 172.21.15.131:0/2495212446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4106530 0x7f38f410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 --2- 172.21.15.131:0/2495212446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 0x7f38f4105ff0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 --2- 172.21.15.131:0/2495212446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f4104980 0x7f38f4104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 -- 172.21.15.131:0/2495212446 >> 172.21.15.131:0/2495212446 conn(0x7f38f40fff10 msgr2=0x7f38f4102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 -- 172.21.15.131:0/2495212446 shutdown_connections 2024-04-26T20:06:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.700+0000 7f38f8a00640 1 -- 172.21.15.131:0/2495212446 wait complete. 2024-04-26T20:06:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.701+0000 7f38f8a00640 1 Processor -- start 2024-04-26T20:06:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f8a00640 1 -- start start 2024-04-26T20:06:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4104980 0x7f38f419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f8a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 0x7f38f419b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f8a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f4106530 0x7f38f41a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f8a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41104b0 con 0x7f38f4104980 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f8a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38f4110330 con 0x7f38f4105b70 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f8a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38f4110630 con 0x7f38f4106530 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4104980 0x7f38f419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4104980 0x7f38f419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54640/0 (socket says 172.21.15.131:54640) 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38f2800640 1 -- 172.21.15.131:0/1966342750 learned_addr learned my addr 172.21.15.131:0/1966342750 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.702+0000 7f38e9e00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 0x7f38f419b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f3200640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f4106530 0x7f38f41a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f2800640 1 -- 172.21.15.131:0/1966342750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f4106530 msgr2=0x7f38f41a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f2800640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f4106530 0x7f38f41a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f2800640 1 -- 172.21.15.131:0/1966342750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 msgr2=0x7f38f419b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:48.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f2800640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 0x7f38f419b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f2800640 1 -- 172.21.15.131:0/1966342750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f41a2830 con 0x7f38f4104980 2024-04-26T20:06:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f2800640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4104980 0x7f38f419ac20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e400e9c0 tx=0x7f38e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38ebe00640 1 -- 172.21.15.131:0/1966342750 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e400cd80 con 0x7f38f4104980 2024-04-26T20:06:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f41a2b20 con 0x7f38f4104980 2024-04-26T20:06:48.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38ebe00640 1 -- 172.21.15.131:0/1966342750 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e40035f0 con 0x7f38f4104980 2024-04-26T20:06:48.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.703+0000 7f38ebe00640 1 -- 172.21.15.131:0/1966342750 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4010690 con 0x7f38f4104980 2024-04-26T20:06:48.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.704+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f41a3070 con 0x7f38f4104980 2024-04-26T20:06:48.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.705+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f410bd70 con 0x7f38f4104980 2024-04-26T20:06:48.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.707+0000 7f38ebe00640 1 -- 172.21.15.131:0/1966342750 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f38e4004000 con 0x7f38f4104980 2024-04-26T20:06:48.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.708+0000 7f38ebe00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38d80a3c30 0x7f38d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:48.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.708+0000 7f38ebe00640 1 -- 172.21.15.131:0/1966342750 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38e40d05e0 con 0x7f38f4104980 2024-04-26T20:06:48.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.708+0000 7f38e9e00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38d80a3c30 0x7f38d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:48.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.709+0000 7f38e9e00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38d80a3c30 0x7f38d80a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000c00 tx=0x7f38dc030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:48.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.713+0000 7f38ebe00640 1 -- 172.21.15.131:0/1966342750 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f38e409d3c0 con 0x7f38f4104980 2024-04-26T20:06:48.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:48 smithi157 ceph-mon[32703]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:48.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:48 smithi131 ceph-mon[27919]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:48.817 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:48 smithi148 systemd[1]: Starting Ceph mds.e for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:06:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.955+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38f402dad0 con 0x7f38f4104980 2024-04-26T20:06:48.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.956+0000 7f38ebe00640 1 -- 172.21.15.131:0/1966342750 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f38e409ccc0 con 0x7f38f4104980 2024-04-26T20:06:48.957 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:48.957 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:06:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 5 2024-04-26T20:06:48.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38d80a3c30 msgr2=0x7f38d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38d80a3c30 0x7f38d80a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000c00 tx=0x7f38dc030470 comp rx=0 tx=0).stop 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4104980 msgr2=0x7f38f419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4104980 0x7f38f419ac20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e400e9c0 tx=0x7f38e400ee80 comp rx=0 tx=0).stop 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 shutdown_connections 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38d80a3c30 0x7f38d80a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f4106530 0x7f38f41a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f4105b70 0x7f38f419b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 --2- 172.21.15.131:0/1966342750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38f4104980 0x7f38f419ac20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 >> 172.21.15.131:0/1966342750 conn(0x7f38f40fff10 msgr2=0x7f38f4108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:48.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.959+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 shutdown_connections 2024-04-26T20:06:48.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:48.960+0000 7f38f8a00640 1 -- 172.21.15.131:0/1966342750 wait complete. 2024-04-26T20:06:49.110 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:48 smithi148 podman[62359]: 2024-04-26 20:06:48.762483472 +0000 UTC m=+0.017790731 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:06:49.110 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:48 smithi148 podman[62359]: 2024-04-26 20:06:48.871552665 +0000 UTC m=+0.126859956 container create e6a0e595490d00a4538297bd825624492dd5a081669c5131ce313ab35bff56bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T20:06:49.110 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:49 smithi148 podman[62359]: 2024-04-26 20:06:49.107516669 +0000 UTC m=+0.362823915 container init e6a0e595490d00a4538297bd825624492dd5a081669c5131ce313ab35bff56bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T20:06:49.110 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:49 smithi148 podman[62359]: 2024-04-26 20:06:49.110134295 +0000 UTC m=+0.365441543 container start e6a0e595490d00a4538297bd825624492dd5a081669c5131ce313ab35bff56bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T20:06:49.426 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mds[62378]: -- 172.21.15.148:0/783227119 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a79391a40 con 0x561a79364800 2024-04-26T20:06:49.426 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e[62374]: starting mds.e at 2024-04-26T20:06:49.426 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:49 smithi148 bash[62359]: e6a0e595490d00a4538297bd825624492dd5a081669c5131ce313ab35bff56bc 2024-04-26T20:06:49.426 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:06:49 smithi148 systemd[1]: Started Ceph mds.e for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:49 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1966342750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:49.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:49 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1966342750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:49.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:49.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:49.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:49.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1966342750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:49.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:50 smithi157 ceph-mon[32703]: Deploying daemon mds.f on smithi157 2024-04-26T20:06:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:50 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.148:6834/2202047184,v1:172.21.15.148:6835/2202047184] up:boot 2024-04-26T20:06:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:50 smithi157 ceph-mon[32703]: fsmap 5 up:standby 2024-04-26T20:06:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T20:06:50.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:50 smithi157 ceph-mon[32703]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:50 smithi131 ceph-mon[27919]: Deploying daemon mds.f on smithi157 2024-04-26T20:06:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:50 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.148:6834/2202047184,v1:172.21.15.148:6835/2202047184] up:boot 2024-04-26T20:06:50.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:50 smithi131 ceph-mon[27919]: fsmap 5 up:standby 2024-04-26T20:06:50.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T20:06:50.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:50 smithi131 ceph-mon[27919]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:50 smithi148 ceph-mon[32281]: Deploying daemon mds.f on smithi157 2024-04-26T20:06:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:50 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.148:6834/2202047184,v1:172.21.15.148:6835/2202047184] up:boot 2024-04-26T20:06:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:50 smithi148 ceph-mon[32281]: fsmap 5 up:standby 2024-04-26T20:06:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T20:06:50.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:50 smithi148 ceph-mon[32281]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:50.982 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:51.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 -- 172.21.15.131:0/2008669153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 msgr2=0x7f51381093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:51.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 --2- 172.21.15.131:0/2008669153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 0x7f51381093b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5128009ca0 tx=0x7f512802a8c0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 -- 172.21.15.131:0/2008669153 shutdown_connections 2024-04-26T20:06:51.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 --2- 172.21.15.131:0/2008669153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f513810ab40 0x7f51381113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 --2- 172.21.15.131:0/2008669153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f513810a180 0x7f513810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 --2- 172.21.15.131:0/2008669153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 0x7f51381093b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 -- 172.21.15.131:0/2008669153 >> 172.21.15.131:0/2008669153 conn(0x7f5138077f40 msgr2=0x7f513807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:51.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 -- 172.21.15.131:0/2008669153 shutdown_connections 2024-04-26T20:06:51.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.194+0000 7f513f000640 1 -- 172.21.15.131:0/2008669153 wait complete. 2024-04-26T20:06:51.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 Processor -- start 2024-04-26T20:06:51.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 -- start start 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 0x7f513819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f513810a180 0x7f513819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f513810ab40 0x7f51381a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5138114ad0 con 0x7f513810a180 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5138114950 con 0x7f513810ab40 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.195+0000 7f513f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5138114c50 con 0x7f5138108f90 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f5137e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 0x7f513819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f512f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f513810a180 0x7f513819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f5137e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 0x7f513819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59936/0 (socket says 172.21.15.131:59936) 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f5137e00640 1 -- 172.21.15.131:0/2055818281 learned_addr learned my addr 172.21.15.131:0/2055818281 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f513d000640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f513810ab40 0x7f51381a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:51.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f512f400640 1 -- 172.21.15.131:0/2055818281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 msgr2=0x7f513819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f512f400640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 0x7f513819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f512f400640 1 -- 172.21.15.131:0/2055818281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f513810ab40 msgr2=0x7f51381a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f512f400640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f513810ab40 0x7f51381a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f512f400640 1 -- 172.21.15.131:0/2055818281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51381a6dd0 con 0x7f513810a180 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.196+0000 7f512f400640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f513810a180 0x7f513819f6f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f512000e9f0 tx=0x7f512000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.197+0000 7f5135600640 1 -- 172.21.15.131:0/2055818281 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512000ccc0 con 0x7f513810a180 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.197+0000 7f5135600640 1 -- 172.21.15.131:0/2055818281 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5120005020 con 0x7f513810a180 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.197+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51381a70c0 con 0x7f513810a180 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.197+0000 7f5135600640 1 -- 172.21.15.131:0/2055818281 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120004a00 con 0x7f513810a180 2024-04-26T20:06:51.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.197+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51381a75c0 con 0x7f513810a180 2024-04-26T20:06:51.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.198+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5138110300 con 0x7f513810a180 2024-04-26T20:06:51.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.200+0000 7f5135600640 1 -- 172.21.15.131:0/2055818281 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5120004ba0 con 0x7f513810a180 2024-04-26T20:06:51.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.201+0000 7f5135600640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f511c0a3bb0 0x7f511c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:51.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.202+0000 7f5135600640 1 -- 172.21.15.131:0/2055818281 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f51200cffb0 con 0x7f513810a180 2024-04-26T20:06:51.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.202+0000 7f5137e00640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f511c0a3bb0 0x7f511c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:51.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.202+0000 7f5137e00640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f511c0a3bb0 0x7f511c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5128005e30 tx=0x7f5128005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:51.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.206+0000 7f5135600640 1 -- 172.21.15.131:0/2055818281 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f512009cd90 con 0x7f513810a180 2024-04-26T20:06:51.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.448+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f513802dad0 con 0x7f513810a180 2024-04-26T20:06:51.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.449+0000 7f5135600640 1 -- 172.21.15.131:0/2055818281 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f512009c690 con 0x7f513810a180 2024-04-26T20:06:51.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:51.541 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T20:06:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2024-04-26T20:06:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f511c0a3bb0 msgr2=0x7f511c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f511c0a3bb0 0x7f511c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5128005e30 tx=0x7f5128005dc0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f513810a180 msgr2=0x7f513819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f513810a180 0x7f513819f6f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f512000e9f0 tx=0x7f512000eeb0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 shutdown_connections 2024-04-26T20:06:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f513810ab40 0x7f51381a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f513810a180 0x7f513819f6f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f511c0a3bb0 0x7f511c0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 --2- 172.21.15.131:0/2055818281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5138108f90 0x7f513819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 >> 172.21.15.131:0/2055818281 conn(0x7f5138077f40 msgr2=0x7f513810d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 shutdown_connections 2024-04-26T20:06:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:51.452+0000 7f513f000640 1 -- 172.21.15.131:0/2055818281 wait complete. 2024-04-26T20:06:51.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:51 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2055818281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:51.699 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:51 smithi157 systemd[1]: Starting Ceph mds.f for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:06:51.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:51 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2055818281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:51.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:51 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2055818281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:51.980 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:51 smithi157 podman[63205]: 2024-04-26 20:06:51.819982422 +0000 UTC m=+0.018449999 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:06:51.980 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:51 smithi157 podman[63205]: 2024-04-26 20:06:51.939773206 +0000 UTC m=+0.138240782 container create 270914e0e0f8ec8f134404e34af2217396782ec7822addc243c2d83c27952558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T20:06:52.480 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:52 smithi157 podman[63205]: 2024-04-26 20:06:52.192976997 +0000 UTC m=+0.391444575 container init 270914e0e0f8ec8f134404e34af2217396782ec7822addc243c2d83c27952558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T20:06:52.480 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:52 smithi157 podman[63205]: 2024-04-26 20:06:52.195538375 +0000 UTC m=+0.394005963 container start 270914e0e0f8ec8f134404e34af2217396782ec7822addc243c2d83c27952558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T20:06:52.480 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mds[63224]: -- 172.21.15.157:0/2664302021 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b85ea77a40 con 0x55b85ea4ac00 2024-04-26T20:06:52.480 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f[63220]: starting mds.f at 2024-04-26T20:06:52.480 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:52 smithi157 bash[63205]: 270914e0e0f8ec8f134404e34af2217396782ec7822addc243c2d83c27952558 2024-04-26T20:06:52.480 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:06:52 smithi157 systemd[1]: Started Ceph mds.f for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:52 smithi148 ceph-mon[32281]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:52.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mon[32703]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:52.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:52.981 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:53.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:52 smithi131 ceph-mon[27919]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:53.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:53.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:53.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:53.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:53.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:53.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:53.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:53.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:53 smithi157 ceph-mon[32703]: Deploying daemon mds.c on smithi157 2024-04-26T20:06:53.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:53 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.157:6834/1007065155,v1:172.21.15.157:6835/1007065155] up:boot 2024-04-26T20:06:53.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:53 smithi157 ceph-mon[32703]: fsmap 6 up:standby 2024-04-26T20:06:53.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.753+0000 7fa6c7a00640 1 -- 172.21.15.131:0/3962993247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0104b90 msgr2=0x7fa6c0104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.753+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/3962993247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0104b90 0x7fa6c0104fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009900 tx=0x7fa6a802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.753+0000 7fa6c7a00640 1 -- 172.21.15.131:0/3962993247 shutdown_connections 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.753+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/3962993247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0106740 0x7fa6c010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.753+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/3962993247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c0105d80 0x7fa6c0106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.753+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/3962993247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0104b90 0x7fa6c0104fb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.754+0000 7fa6c7a00640 1 -- 172.21.15.131:0/3962993247 >> 172.21.15.131:0/3962993247 conn(0x7fa6c0100120 msgr2=0x7fa6c0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.754+0000 7fa6c7a00640 1 -- 172.21.15.131:0/3962993247 shutdown_connections 2024-04-26T20:06:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.754+0000 7fa6c7a00640 1 -- 172.21.15.131:0/3962993247 wait complete. 2024-04-26T20:06:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 Processor -- start 2024-04-26T20:06:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 -- start start 2024-04-26T20:06:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0104b90 0x7fa6c019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0105d80 0x7fa6c019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c0106740 0x7fa6c01a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0110700 con 0x7fa6c0104b90 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0110580 con 0x7fa6c0105d80 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c7a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0110880 con 0x7fa6c0106740 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0104b90 0x7fa6c019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0104b90 0x7fa6c019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54676/0 (socket says 172.21.15.131:54676) 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.755+0000 7fa6c5000640 1 -- 172.21.15.131:0/2076484926 learned_addr learned my addr 172.21.15.131:0/2076484926 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c0106740 0x7fa6c01a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6b7e00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0105d80 0x7fa6c019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5a00640 1 -- 172.21.15.131:0/2076484926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0105d80 msgr2=0x7fa6c019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0105d80 0x7fa6c019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5a00640 1 -- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0104b90 msgr2=0x7fa6c019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0104b90 0x7fa6c019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5a00640 1 -- 172.21.15.131:0/2076484926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01a29d0 con 0x7fa6c0106740 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5000640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0104b90 0x7fa6c019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6c5a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c0106740 0x7fa6c01a2330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.756+0000 7fa6be000640 1 -- 172.21.15.131:0/2076484926 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800ce20 con 0x7fa6c0106740 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.757+0000 7fa6be000640 1 -- 172.21.15.131:0/2076484926 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8005020 con 0x7fa6c0106740 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.757+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c01a2cc0 con 0x7fa6c0106740 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.757+0000 7fa6be000640 1 -- 172.21.15.131:0/2076484926 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8010690 con 0x7fa6c0106740 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.757+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c01a31c0 con 0x7fa6c0106740 2024-04-26T20:06:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.758+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c010beb0 con 0x7fa6c0106740 2024-04-26T20:06:53.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.760+0000 7fa6be000640 1 -- 172.21.15.131:0/2076484926 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa6b80054d0 con 0x7fa6c0106740 2024-04-26T20:06:53.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.761+0000 7fa6be000640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6a00a3bb0 0x7fa6a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:53.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.761+0000 7fa6be000640 1 -- 172.21.15.131:0/2076484926 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa6b80d0750 con 0x7fa6c0106740 2024-04-26T20:06:53.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.761+0000 7fa6c5000640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6a00a3bb0 0x7fa6a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:53.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.762+0000 7fa6c5000640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6a00a3bb0 0x7fa6a00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8006010 tx=0x7fa6a8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:53.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:53.765+0000 7fa6be000640 1 -- 172.21.15.131:0/2076484926 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa6b809d530 con 0x7fa6c0106740 2024-04-26T20:06:53.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:53 smithi148 ceph-mon[32281]: Deploying daemon mds.c on smithi157 2024-04-26T20:06:53.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:53 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.157:6834/1007065155,v1:172.21.15.157:6835/1007065155] up:boot 2024-04-26T20:06:53.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:53 smithi148 ceph-mon[32281]: fsmap 6 up:standby 2024-04-26T20:06:53.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T20:06:54.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:53 smithi131 ceph-mon[27919]: Deploying daemon mds.c on smithi157 2024-04-26T20:06:54.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:53 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.157:6834/1007065155,v1:172.21.15.157:6835/1007065155] up:boot 2024-04-26T20:06:54.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:53 smithi131 ceph-mon[27919]: fsmap 6 up:standby 2024-04-26T20:06:54.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T20:06:54.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.009+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6c002dad0 con 0x7fa6c0106740 2024-04-26T20:06:54.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.010+0000 7fa6be000640 1 -- 172.21.15.131:0/2076484926 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7fa6b809ce30 con 0x7fa6c0106740 2024-04-26T20:06:54.010 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:54.011 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-26T20:06:54.011 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-04-26T20:06:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6a00a3bb0 msgr2=0x7fa6a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6a00a3bb0 0x7fa6a00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8006010 tx=0x7fa6a8005a20 comp rx=0 tx=0).stop 2024-04-26T20:06:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c0106740 msgr2=0x7fa6c01a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c0106740 0x7fa6c01a2330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b800ef10 comp rx=0 tx=0).stop 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 shutdown_connections 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6a00a3bb0 0x7fa6a00a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c0106740 0x7fa6c01a2330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c0105d80 0x7fa6c019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 --2- 172.21.15.131:0/2076484926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c0104b90 0x7fa6c019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.012+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 >> 172.21.15.131:0/2076484926 conn(0x7fa6c0100120 msgr2=0x7fa6c0109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.013+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 shutdown_connections 2024-04-26T20:06:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:54.013+0000 7fa6c7a00640 1 -- 172.21.15.131:0/2076484926 wait complete. 2024-04-26T20:06:54.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:54 smithi157 ceph-mon[32703]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:54.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:54 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2076484926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:54.731 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:54 smithi157 systemd[1]: Starting Ceph mds.c for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:06:54.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:54 smithi148 ceph-mon[32281]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:54.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:54 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2076484926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:55.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:54 smithi131 ceph-mon[27919]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:55.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:54 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2076484926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:55.141 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:54 smithi157 podman[63512]: 2024-04-26 20:06:54.769032368 +0000 UTC m=+0.017936282 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:06:55.141 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:54 smithi157 podman[63512]: 2024-04-26 20:06:54.880311214 +0000 UTC m=+0.129215114 container create dafe6d5c07503300fa3f5f8002289a0f4a3fcdaa0b21b46e9f52658e4e93c999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:06:55.141 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:55 smithi157 podman[63512]: 2024-04-26 20:06:55.140913031 +0000 UTC m=+0.389816938 container init dafe6d5c07503300fa3f5f8002289a0f4a3fcdaa0b21b46e9f52658e4e93c999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T20:06:55.480 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:55 smithi157 podman[63512]: 2024-04-26 20:06:55.143710905 +0000 UTC m=+0.392614814 container start dafe6d5c07503300fa3f5f8002289a0f4a3fcdaa0b21b46e9f52658e4e93c999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-04-26T20:06:55.480 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:55 smithi157 ceph-mds[63531]: -- 172.21.15.157:0/411017222 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eca7fcba40 con 0x55eca7f9e800 2024-04-26T20:06:55.480 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c[63527]: starting mds.c at 2024-04-26T20:06:55.480 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:55 smithi157 bash[63512]: dafe6d5c07503300fa3f5f8002289a0f4a3fcdaa0b21b46e9f52658e4e93c999 2024-04-26T20:06:55.480 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:06:55 smithi157 systemd[1]: Started Ceph mds.c for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:56.035 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 -- 172.21.15.131:0/571352484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d40103930 msgr2=0x7f8d40103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 --2- 172.21.15.131:0/571352484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d40103930 0x7f8d40103db0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009900 tx=0x7f8d2c02a410 comp rx=0 tx=0).stop 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 -- 172.21.15.131:0/571352484 shutdown_connections 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 --2- 172.21.15.131:0/571352484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d401042f0 0x7f8d4010ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 --2- 172.21.15.131:0/571352484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d40103930 0x7f8d40103db0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 --2- 172.21.15.131:0/571352484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 0x7f8d40102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 -- 172.21.15.131:0/571352484 >> 172.21.15.131:0/571352484 conn(0x7f8d400fdcb0 msgr2=0x7f8d401000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 -- 172.21.15.131:0/571352484 shutdown_connections 2024-04-26T20:06:56.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.245+0000 7f8d46a00640 1 -- 172.21.15.131:0/571352484 wait complete. 2024-04-26T20:06:56.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.246+0000 7f8d46a00640 1 Processor -- start 2024-04-26T20:06:56.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.246+0000 7f8d46a00640 1 -- start start 2024-04-26T20:06:56.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d46a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 0x7f8d40198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:56.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d46a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d40103930 0x7f8d40198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:56.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d46a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d401042f0 0x7f8d4019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d46a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010e2a0 con 0x7f8d40102740 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d46a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010e120 con 0x7f8d40103930 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d46a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010e420 con 0x7f8d401042f0 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 0x7f8d40198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 0x7f8d40198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49486/0 (socket says 172.21.15.131:49486) 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 -- 172.21.15.131:0/3109758236 learned_addr learned my addr 172.21.15.131:0/3109758236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d44c00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d40103930 0x7f8d40198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d46000640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d401042f0 0x7f8d4019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 -- 172.21.15.131:0/3109758236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d401042f0 msgr2=0x7f8d4019ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d401042f0 0x7f8d4019ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 -- 172.21.15.131:0/3109758236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d40103930 msgr2=0x7f8d40198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d40103930 0x7f8d40198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.247+0000 7f8d45600640 1 -- 172.21.15.131:0/3109758236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d401a0580 con 0x7f8d40102740 2024-04-26T20:06:56.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.248+0000 7f8d45600640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 0x7f8d40198960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e9c0 tx=0x7f8d3400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:56.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.248+0000 7f8d3e000640 1 -- 172.21.15.131:0/3109758236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400ccc0 con 0x7f8d40102740 2024-04-26T20:06:56.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.248+0000 7f8d3e000640 1 -- 172.21.15.131:0/3109758236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d340035f0 con 0x7f8d40102740 2024-04-26T20:06:56.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.248+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d401a0870 con 0x7f8d40102740 2024-04-26T20:06:56.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.248+0000 7f8d3e000640 1 -- 172.21.15.131:0/3109758236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400ce60 con 0x7f8d40102740 2024-04-26T20:06:56.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.248+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401a0d90 con 0x7f8d40102740 2024-04-26T20:06:56.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.249+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d40109a60 con 0x7f8d40102740 2024-04-26T20:06:56.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.252+0000 7f8d3e000640 1 -- 172.21.15.131:0/3109758236 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8d340037a0 con 0x7f8d40102740 2024-04-26T20:06:56.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.253+0000 7f8d3e000640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d280a3bb0 0x7f8d280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:56.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.254+0000 7f8d3e000640 1 -- 172.21.15.131:0/3109758236 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8d340d03b0 con 0x7f8d40102740 2024-04-26T20:06:56.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.254+0000 7f8d44c00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d280a3bb0 0x7f8d280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:56.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.255+0000 7f8d44c00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d280a3bb0 0x7f8d280a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c031500 tx=0x7f8d2c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.260+0000 7f8d3e000640 1 -- 172.21.15.131:0/3109758236 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8d3409d190 con 0x7f8d40102740 2024-04-26T20:06:56.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: Deploying daemon mds.h on smithi148 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.157:6836/3509306790,v1:172.21.15.157:6837/3509306790] up:boot 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: fsmap 7 up:standby 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T20:06:56.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:56 smithi131 ceph-mon[27919]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.505+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d4002dad0 con 0x7f8d40102740 2024-04-26T20:06:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.506+0000 7f8d3e000640 1 -- 172.21.15.131:0/3109758236 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7f8d3409ca90 con 0x7f8d40102740 2024-04-26T20:06:56.508 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:56.508 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T20:06:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 8 2024-04-26T20:06:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d280a3bb0 msgr2=0x7f8d280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d280a3bb0 0x7f8d280a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c031500 tx=0x7f8d2c005a20 comp rx=0 tx=0).stop 2024-04-26T20:06:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 msgr2=0x7f8d40198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 0x7f8d40198960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e9c0 tx=0x7f8d3400edc0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 shutdown_connections 2024-04-26T20:06:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d280a3bb0 0x7f8d280a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d401042f0 0x7f8d4019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d40103930 0x7f8d40198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 --2- 172.21.15.131:0/3109758236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40102740 0x7f8d40198960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 >> 172.21.15.131:0/3109758236 conn(0x7f8d400fdcb0 msgr2=0x7f8d40106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 shutdown_connections 2024-04-26T20:06:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:56.509+0000 7f8d46a00640 1 -- 172.21.15.131:0/3109758236 wait complete. 2024-04-26T20:06:56.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: Deploying daemon mds.h on smithi148 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.157:6836/3509306790,v1:172.21.15.157:6837/3509306790] up:boot 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: fsmap 7 up:standby 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T20:06:56.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:56 smithi148 ceph-mon[32281]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:56.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: Deploying daemon mds.h on smithi148 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.157:6836/3509306790,v1:172.21.15.157:6837/3509306790] up:boot 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: fsmap 7 up:standby 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T20:06:56.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:56 smithi157 ceph-mon[32703]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:57.509 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:57 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3109758236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:57.631 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:57 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3109758236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:57.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:57 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3109758236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:58.066 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:06:57 smithi148 systemd[1]: Started Ceph mds.h for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:06:58.531 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:06:58.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:58 smithi131 ceph-mon[27919]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:58.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:58.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:58.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:58.693 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:58 smithi157 ceph-mon[32703]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:58.693 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.693 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.693 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.693 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:58.693 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:58.694 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:58.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 -- 172.21.15.131:0/1051601605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00105b80 msgr2=0x7f8e00106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:58.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 --2- 172.21.15.131:0/1051601605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00105b80 0x7f8e00106000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009860 tx=0x7f8de802a340 comp rx=0 tx=0).stop 2024-04-26T20:06:58.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 -- 172.21.15.131:0/1051601605 shutdown_connections 2024-04-26T20:06:58.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 --2- 172.21.15.131:0/1051601605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00106540 0x7f8e0010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:58.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 --2- 172.21.15.131:0/1051601605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00105b80 0x7f8e00106000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:58.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 --2- 172.21.15.131:0/1051601605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e00104990 0x7f8e00104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:58.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 -- 172.21.15.131:0/1051601605 >> 172.21.15.131:0/1051601605 conn(0x7f8e000fff00 msgr2=0x7f8e00102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:58.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 -- 172.21.15.131:0/1051601605 shutdown_connections 2024-04-26T20:06:58.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.739+0000 7f8dff200640 1 -- 172.21.15.131:0/1051601605 wait complete. 2024-04-26T20:06:58.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.740+0000 7f8dff200640 1 Processor -- start 2024-04-26T20:06:58.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.740+0000 7f8dff200640 1 -- start start 2024-04-26T20:06:58.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dff200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00104990 0x7f8e0019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:58.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dff200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e00105b80 0x7f8e0019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:58.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dff200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00106540 0x7f8e001a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:58.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dff200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001104a0 con 0x7f8e00106540 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dff200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e00110320 con 0x7f8e00105b80 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dff200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e00110620 con 0x7f8e00104990 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00104990 0x7f8e0019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00104990 0x7f8e0019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44712/0 (socket says 172.21.15.131:44712) 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 -- 172.21.15.131:0/2791778355 learned_addr learned my addr 172.21.15.131:0/2791778355 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfe800640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00106540 0x7f8e001a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8df7e00640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e00105b80 0x7f8e0019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 -- 172.21.15.131:0/2791778355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e00105b80 msgr2=0x7f8e0019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e00105b80 0x7f8e0019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 -- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00106540 msgr2=0x7f8e001a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00106540 0x7f8e001a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.741+0000 7f8dfde00640 1 -- 172.21.15.131:0/2791778355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e001a2790 con 0x7f8e00104990 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.742+0000 7f8dfde00640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00104990 0x7f8e0019ab70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e9a0 tx=0x7f8df000eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.742+0000 7f8df6a00640 1 -- 172.21.15.131:0/2791778355 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000ccc0 con 0x7f8e00104990 2024-04-26T20:06:58.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.742+0000 7f8df6a00640 1 -- 172.21.15.131:0/2791778355 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df00108e0 con 0x7f8e00104990 2024-04-26T20:06:58.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.742+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e001a2a80 con 0x7f8e00104990 2024-04-26T20:06:58.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.742+0000 7f8df6a00640 1 -- 172.21.15.131:0/2791778355 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0018980 con 0x7f8e00104990 2024-04-26T20:06:58.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.742+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e001a2fa0 con 0x7f8e00104990 2024-04-26T20:06:58.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.744+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0010bdd0 con 0x7f8e00104990 2024-04-26T20:06:58.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.745+0000 7f8df6a00640 1 -- 172.21.15.131:0/2791778355 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8df001f020 con 0x7f8e00104990 2024-04-26T20:06:58.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.747+0000 7f8df6a00640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8de40a3bb0 0x7f8de40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:06:58.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.747+0000 7f8df6a00640 1 -- 172.21.15.131:0/2791778355 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8df00d0100 con 0x7f8e00104990 2024-04-26T20:06:58.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.747+0000 7f8df7e00640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8de40a3bb0 0x7f8de40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:06:58.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.748+0000 7f8df7e00640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8de40a3bb0 0x7f8de40a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8de802a930 tx=0x7f8de8030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:06:58.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.751+0000 7f8df6a00640 1 -- 172.21.15.131:0/2791778355 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8df009cee0 con 0x7f8e00104990 2024-04-26T20:06:58.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:58 smithi148 ceph-mon[32281]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:06:58.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:06:58.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:06:58.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:06:58.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.996+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e0002dad0 con 0x7f8e00104990 2024-04-26T20:06:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:58.998+0000 7f8df6a00640 1 -- 172.21.15.131:0/2791778355 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7f8df009c7e0 con 0x7f8e00104990 2024-04-26T20:06:59.000 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:06:59.000 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T20:06:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8de40a3bb0 msgr2=0x7f8de40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8de40a3bb0 0x7f8de40a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8de802a930 tx=0x7f8de8030450 comp rx=0 tx=0).stop 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00104990 msgr2=0x7f8e0019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00104990 0x7f8e0019ab70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e9a0 tx=0x7f8df000eda0 comp rx=0 tx=0).stop 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 shutdown_connections 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e00106540 0x7f8e001a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e00105b80 0x7f8e0019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8de40a3bb0 0x7f8de40a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 --2- 172.21.15.131:0/2791778355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e00104990 0x7f8e0019ab70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:06:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 >> 172.21.15.131:0/2791778355 conn(0x7f8e000fff00 msgr2=0x7f8e00108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:06:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 shutdown_connections 2024-04-26T20:06:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:06:59.001+0000 7f8dff200640 1 -- 172.21.15.131:0/2791778355 wait complete. 2024-04-26T20:06:59.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:59 smithi157 ceph-mon[32703]: Deploying daemon mds.i on smithi157 2024-04-26T20:06:59.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:59 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.148:6836/1158905219,v1:172.21.15.148:6837/1158905219] up:boot 2024-04-26T20:06:59.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:59 smithi157 ceph-mon[32703]: fsmap 8 up:standby 2024-04-26T20:06:59.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:59 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T20:06:59.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:06:59 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2791778355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:59.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:59 smithi131 ceph-mon[27919]: Deploying daemon mds.i on smithi157 2024-04-26T20:06:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:59 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.148:6836/1158905219,v1:172.21.15.148:6837/1158905219] up:boot 2024-04-26T20:06:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:59 smithi131 ceph-mon[27919]: fsmap 8 up:standby 2024-04-26T20:06:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:59 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T20:06:59.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:06:59 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2791778355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:06:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:59 smithi148 ceph-mon[32281]: Deploying daemon mds.i on smithi157 2024-04-26T20:06:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:59 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.148:6836/1158905219,v1:172.21.15.148:6837/1158905219] up:boot 2024-04-26T20:06:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:59 smithi148 ceph-mon[32281]: fsmap 8 up:standby 2024-04-26T20:06:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:59 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T20:06:59.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:06:59 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2791778355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:00.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:00 smithi157 ceph-mon[32703]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:00.481 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 systemd[1]: Starting Ceph mds.i for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:07:00.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:00 smithi131 ceph-mon[27919]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:00.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:00 smithi148 ceph-mon[32281]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:00.863 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 podman[63831]: 2024-04-26 20:07:00.450630345 +0000 UTC m=+0.018007688 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:07:00.887 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 podman[63831]: 2024-04-26 20:07:00.579628169 +0000 UTC m=+0.147005554 container create 7ea454b9e86acfc35cbb60a7fe49479d9b3d28d23042a2ec79795ef64ac47592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T20:07:01.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:07:01.230 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 podman[63831]: 2024-04-26 20:07:00.863272264 +0000 UTC m=+0.430649606 container init 7ea454b9e86acfc35cbb60a7fe49479d9b3d28d23042a2ec79795ef64ac47592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T20:07:01.297 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 podman[63831]: 2024-04-26 20:07:00.865909574 +0000 UTC m=+0.433286919 container start 7ea454b9e86acfc35cbb60a7fe49479d9b3d28d23042a2ec79795ef64ac47592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T20:07:01.331 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 ceph-mds[63850]: -- 172.21.15.157:0/2171389944 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561556ea3a40 con 0x561556e76c00 2024-04-26T20:07:01.364 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i[63846]: starting mds.i at 2024-04-26T20:07:01.397 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 bash[63831]: 7ea454b9e86acfc35cbb60a7fe49479d9b3d28d23042a2ec79795ef64ac47592 2024-04-26T20:07:01.431 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:07:00 smithi157 systemd[1]: Started Ceph mds.i for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:07:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 -- 172.21.15.131:0/1163477241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 msgr2=0x7fc804074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:01.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 --2- 172.21.15.131:0/1163477241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 0x7fc804074dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009930 tx=0x7fc7f402a420 comp rx=0 tx=0).stop 2024-04-26T20:07:01.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 -- 172.21.15.131:0/1163477241 shutdown_connections 2024-04-26T20:07:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 --2- 172.21.15.131:0/1163477241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc80410ef20 0x7fc804111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:01.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 --2- 172.21.15.131:0/1163477241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804075390 0x7fc804075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:01.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 --2- 172.21.15.131:0/1163477241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 0x7fc804074dc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:01.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 -- 172.21.15.131:0/1163477241 >> 172.21.15.131:0/1163477241 conn(0x7fc8040fdee0 msgr2=0x7fc804100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:01.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 -- 172.21.15.131:0/1163477241 shutdown_connections 2024-04-26T20:07:01.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.230+0000 7fc80a800640 1 -- 172.21.15.131:0/1163477241 wait complete. 2024-04-26T20:07:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.231+0000 7fc80a800640 1 Processor -- start 2024-04-26T20:07:01.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc80a800640 1 -- start start 2024-04-26T20:07:01.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc80a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804075390 0x7fc80419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:02.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc80a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 0x7fc80419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:02.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc80a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc80410ef20 0x7fc8041a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc80a800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc804114a80 con 0x7fc80410ef20 2024-04-26T20:07:02.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc80a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc804114900 con 0x7fc804076970 2024-04-26T20:07:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc80a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc804114c00 con 0x7fc804075390 2024-04-26T20:07:02.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc803400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804075390 0x7fc80419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:02.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc803400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804075390 0x7fc80419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44726/0 (socket says 172.21.15.131:44726) 2024-04-26T20:07:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc803400640 1 -- 172.21.15.131:0/393086046 learned_addr learned my addr 172.21.15.131:0/393086046 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:02.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc803e00640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc80410ef20 0x7fc8041a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:02.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.232+0000 7fc802a00640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 0x7fc80419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:02.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc803400640 1 -- 172.21.15.131:0/393086046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 msgr2=0x7fc80419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:02.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc803400640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 0x7fc80419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:02.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc803400640 1 -- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc80410ef20 msgr2=0x7fc8041a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:02.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc803400640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc80410ef20 0x7fc8041a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc803400640 1 -- 172.21.15.131:0/393086046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8041a6dc0 con 0x7fc804075390 2024-04-26T20:07:02.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc803400640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804075390 0x7fc80419f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009a50 tx=0x7fc7f40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:02.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc7fbe00640 1 -- 172.21.15.131:0/393086046 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f403f030 con 0x7fc804075390 2024-04-26T20:07:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc7fbe00640 1 -- 172.21.15.131:0/393086046 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f4031910 con 0x7fc804075390 2024-04-26T20:07:02.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.233+0000 7fc7fbe00640 1 -- 172.21.15.131:0/393086046 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4035070 con 0x7fc804075390 2024-04-26T20:07:02.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.234+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8041a7050 con 0x7fc804075390 2024-04-26T20:07:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.234+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8041a7590 con 0x7fc804075390 2024-04-26T20:07:02.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.235+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8041050f0 con 0x7fc804075390 2024-04-26T20:07:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.237+0000 7fc7fbe00640 1 -- 172.21.15.131:0/393086046 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc7f4042030 con 0x7fc804075390 2024-04-26T20:07:02.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.238+0000 7fc7fbe00640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc7e80a3c30 0x7fc7e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:03.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.238+0000 7fc7fbe00640 1 -- 172.21.15.131:0/393086046 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc7f40f0490 con 0x7fc804075390 2024-04-26T20:07:03.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.238+0000 7fc802a00640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc7e80a3c30 0x7fc7e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:03.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.239+0000 7fc802a00640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc7e80a3c30 0x7fc7e80a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00095f0 tx=0x7fc7f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.242+0000 7fc7fbe00640 1 -- 172.21.15.131:0/393086046 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc7f40bcc20 con 0x7fc804075390 2024-04-26T20:07:03.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:03.326 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:03.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:03.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:03.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:03.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:03.531 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: Deploying daemon mds.g on smithi131 2024-04-26T20:07:03.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.157:6838/1960857281,v1:172.21.15.157:6839/1960857281] up:boot 2024-04-26T20:07:03.599 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: fsmap 9 up:standby 2024-04-26T20:07:03.633 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T20:07:03.666 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/393086046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:03.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:02 smithi157 ceph-mon[32703]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:03.733 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:03.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:03.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:03.842 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:03.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:03.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:03.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: Deploying daemon mds.g on smithi131 2024-04-26T20:07:04.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.157:6838/1960857281,v1:172.21.15.157:6839/1960857281] up:boot 2024-04-26T20:07:04.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: fsmap 9 up:standby 2024-04-26T20:07:04.100 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T20:07:04.133 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/393086046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:04.166 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:02 smithi148 ceph-mon[32281]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:04.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.488+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc804063530 con 0x7fc804075390 2024-04-26T20:07:04.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.489+0000 7fc7fbe00640 1 -- 172.21.15.131:0/393086046 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7fc7f40c0f20 con 0x7fc804075390 2024-04-26T20:07:04.267 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:04.300 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T20:07:04.335 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 10 2024-04-26T20:07:04.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc7e80a3c30 msgr2=0x7fc7e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:04.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc7e80a3c30 0x7fc7e80a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00095f0 tx=0x7fc7f0005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:04.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804075390 msgr2=0x7fc80419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804075390 0x7fc80419f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009a50 tx=0x7fc7f40042f0 comp rx=0 tx=0).stop 2024-04-26T20:07:04.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 shutdown_connections 2024-04-26T20:07:04.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc80410ef20 0x7fc8041a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:04.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804076970 0x7fc80419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:04.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc7e80a3c30 0x7fc7e80a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 --2- 172.21.15.131:0/393086046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804075390 0x7fc80419f1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 >> 172.21.15.131:0/393086046 conn(0x7fc8040fdee0 msgr2=0x7fc8041022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 shutdown_connections 2024-04-26T20:07:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:01.492+0000 7fc80a800640 1 -- 172.21.15.131:0/393086046 wait complete. 2024-04-26T20:07:04.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:04.846 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:04.875 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:04.908 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:04.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:04.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:05.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: Deploying daemon mds.g on smithi131 2024-04-26T20:07:05.057 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.157:6838/1960857281,v1:172.21.15.157:6839/1960857281] up:boot 2024-04-26T20:07:05.092 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: fsmap 9 up:standby 2024-04-26T20:07:05.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T20:07:05.226 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/393086046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:05.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:02 smithi131 ceph-mon[27919]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:05.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:03 smithi131 ceph-mon[27919]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:05.343 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:04 smithi131 systemd[1]: Starting Ceph mds.g for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:07:05.376 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:04 smithi131 podman[68885]: 2024-04-26 20:07:04.489192118 +0000 UTC m=+0.032587929 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:07:05.409 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:04 smithi131 podman[68885]: 2024-04-26 20:07:04.635944272 +0000 UTC m=+0.179340151 container create cb8a2959c13291de06aaa57053997e9911d8277b7d1c28529559f019bdab73c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T20:07:05.442 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:04 smithi131 podman[68885]: 2024-04-26 20:07:04.940150392 +0000 UTC m=+0.483546264 container init cb8a2959c13291de06aaa57053997e9911d8277b7d1c28529559f019bdab73c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:07:05.484 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:04 smithi131 podman[68885]: 2024-04-26 20:07:04.948803891 +0000 UTC m=+0.492199770 container start cb8a2959c13291de06aaa57053997e9911d8277b7d1c28529559f019bdab73c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T20:07:05.534 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:04 smithi131 ceph-mds[68912]: -- 172.21.15.131:0/1881790094 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562b780ba40 con 0x5562b77de800 2024-04-26T20:07:05.584 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g[68903]: starting mds.g at 2024-04-26T20:07:05.662 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:05 smithi131 bash[68885]: cb8a2959c13291de06aaa57053997e9911d8277b7d1c28529559f019bdab73c4 2024-04-26T20:07:05.701 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:07:05 smithi131 systemd[1]: Started Ceph mds.g for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:07:05.751 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:03 smithi157 ceph-mon[32703]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:05.793 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:03 smithi148 ceph-mon[32281]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:06.161 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] up:boot 2024-04-26T20:07:06.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: fsmap 10 up:standby 2024-04-26T20:07:06.226 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T20:07:06.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:06.356 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:06.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:06.426 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:06.459 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:06.493 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:06.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: Deploying daemon mds.a on smithi131 2024-04-26T20:07:06.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:06 smithi131 ceph-mon[27919]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:06.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] up:boot 2024-04-26T20:07:06.634 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: fsmap 10 up:standby 2024-04-26T20:07:06.668 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T20:07:06.701 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:06.735 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:06.768 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:06.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:06.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:06.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:06.960 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: Deploying daemon mds.a on smithi131 2024-04-26T20:07:07.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:06 smithi157 ceph-mon[32703]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:07.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] up:boot 2024-04-26T20:07:07.093 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: fsmap 10 up:standby 2024-04-26T20:07:07.136 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T20:07:07.178 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:07.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:07.243 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:07.277 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:07.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:07.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:07.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: Deploying daemon mds.a on smithi131 2024-04-26T20:07:07.443 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:06 smithi148 ceph-mon[32281]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:07.477 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:07:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 -- 172.21.15.131:0/2554581208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078ed0 msgr2=0x7f49f8079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 --2- 172.21.15.131:0/2554581208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078ed0 0x7f49f8079770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009be0 tx=0x7f49f002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:07:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 -- 172.21.15.131:0/2554581208 shutdown_connections 2024-04-26T20:07:07.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 --2- 172.21.15.131:0/2554581208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078ed0 0x7f49f8079770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:07.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 --2- 172.21.15.131:0/2554581208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49f8078530 0x7f49f8078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:08.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 --2- 172.21.15.131:0/2554581208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 0x7f49f8077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:08.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 -- 172.21.15.131:0/2554581208 >> 172.21.15.131:0/2554581208 conn(0x7f49f80fff10 msgr2=0x7f49f8102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:08.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 -- 172.21.15.131:0/2554581208 shutdown_connections 2024-04-26T20:07:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.773+0000 7f49f7200640 1 -- 172.21.15.131:0/2554581208 wait complete. 2024-04-26T20:07:08.145 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:07 smithi157 ceph-mon[32703]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:08.177 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:07 smithi148 ceph-mon[32281]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:08.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.774+0000 7f49f7200640 1 Processor -- start 2024-04-26T20:07:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.774+0000 7f49f7200640 1 -- start start 2024-04-26T20:07:08.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 0x7f49f819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:08.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078530 0x7f49f819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:08.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49f8078ed0 0x7f49f81a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:08.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f7200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81148c0 con 0x7f49f80772f0 2024-04-26T20:07:08.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8114740 con 0x7f49f8078ed0 2024-04-26T20:07:08.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8114a40 con 0x7f49f8078530 2024-04-26T20:07:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 0x7f49f819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:08.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 0x7f49f819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36400/0 (socket says 172.21.15.131:36400) 2024-04-26T20:07:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f5e00640 1 -- 172.21.15.131:0/928912639 learned_addr learned my addr 172.21.15.131:0/928912639 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:08.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f5400640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078530 0x7f49f819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:09.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.775+0000 7f49f6800640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49f8078ed0 0x7f49f81a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:09.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f5400640 1 -- 172.21.15.131:0/928912639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49f8078ed0 msgr2=0x7f49f81a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f5400640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49f8078ed0 0x7f49f81a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f5400640 1 -- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 msgr2=0x7f49f819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f5400640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 0x7f49f819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f5400640 1 -- 172.21.15.131:0/928912639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81a6b60 con 0x7f49f8078530 2024-04-26T20:07:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f6800640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49f8078ed0 0x7f49f81a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:07:09.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f5e00640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 0x7f49f819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:07:09.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f5400640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078530 0x7f49f819f480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009fd0 tx=0x7f49e400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:09.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49eea00640 1 -- 172.21.15.131:0/928912639 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400ccc0 con 0x7f49f8078530 2024-04-26T20:07:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49eea00640 1 -- 172.21.15.131:0/928912639 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e400ee70 con 0x7f49f8078530 2024-04-26T20:07:09.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49eea00640 1 -- 172.21.15.131:0/928912639 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4004a00 con 0x7f49f8078530 2024-04-26T20:07:09.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81a6e50 con 0x7f49f8078530 2024-04-26T20:07:09.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.776+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81a72c0 con 0x7f49f8078530 2024-04-26T20:07:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.777+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f81112f0 con 0x7f49f8078530 2024-04-26T20:07:09.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.780+0000 7f49eea00640 1 -- 172.21.15.131:0/928912639 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f49e4004ba0 con 0x7f49f8078530 2024-04-26T20:07:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.782+0000 7f49eea00640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49dc0a3c30 0x7f49dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:09.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.782+0000 7f49eea00640 1 -- 172.21.15.131:0/928912639 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f49e40cff50 con 0x7f49f8078530 2024-04-26T20:07:09.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.782+0000 7f49f5e00640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49dc0a3c30 0x7f49dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:09.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.783+0000 7f49f5e00640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49dc0a3c30 0x7f49dc0a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:07.783+0000 7f49eea00640 1 -- 172.21.15.131:0/928912639 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f49e409cd30 con 0x7f49f8078530 2024-04-26T20:07:09.929 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:07 smithi131 ceph-mon[27919]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.052+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49f802dad0 con 0x7f49f8078530 2024-04-26T20:07:10.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.053+0000 7f49eea00640 1 -- 172.21.15.131:0/928912639 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8848 (secure 0 0 0) 0x7f49e409c630 con 0x7f49f8078530 2024-04-26T20:07:10.062 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:10.129 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T20:07:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 11 2024-04-26T20:07:10.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49dc0a3c30 msgr2=0x7f49dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49dc0a3c30 0x7f49dc0a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e800b480 comp rx=0 tx=0).stop 2024-04-26T20:07:10.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078530 msgr2=0x7f49f819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:10.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078530 0x7f49f819f480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009fd0 tx=0x7f49e400ec20 comp rx=0 tx=0).stop 2024-04-26T20:07:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 shutdown_connections 2024-04-26T20:07:10.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49f8078ed0 0x7f49f81a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:10.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49dc0a3c30 0x7f49dc0a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:10.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f8078530 0x7f49f819f480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:10.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 --2- 172.21.15.131:0/928912639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49f80772f0 0x7f49f819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:10.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 >> 172.21.15.131:0/928912639 conn(0x7f49f80fff10 msgr2=0x7f49f8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:10.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 shutdown_connections 2024-04-26T20:07:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:08.056+0000 7f49f7200640 1 -- 172.21.15.131:0/928912639 wait complete. 2024-04-26T20:07:10.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:08 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/928912639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:10.763 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:07:08 smithi131 podman[69242]: 2024-04-26 20:07:08.876801272 +0000 UTC m=+0.208976566 container create dd30dc3a6547d5475444bb4e5d70e6c6f76e4f0f96b1b486950144bd10a2a5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T20:07:10.846 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:07:09 smithi131 podman[69242]: 2024-04-26 20:07:09.139392794 +0000 UTC m=+0.471568094 container init dd30dc3a6547d5475444bb4e5d70e6c6f76e4f0f96b1b486950144bd10a2a5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T20:07:10.879 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:07:09 smithi131 podman[69242]: 2024-04-26 20:07:09.149786705 +0000 UTC m=+0.481962025 container start dd30dc3a6547d5475444bb4e5d70e6c6f76e4f0f96b1b486950144bd10a2a5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T20:07:10.912 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:07:09 smithi131 ceph-mds[69260]: -- 172.21.15.131:0/1713330260 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b5b9c1a40 con 0x555b5b994800 2024-04-26T20:07:10.987 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:07:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a[69256]: starting mds.a at 2024-04-26T20:07:11.021 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:07:09 smithi131 bash[69242]: dd30dc3a6547d5475444bb4e5d70e6c6f76e4f0f96b1b486950144bd10a2a5a4 2024-04-26T20:07:11.054 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:07:09 smithi131 systemd[1]: Started Ceph mds.a for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:07:11.088 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:11.125 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:11.163 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:11.223 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:11.254 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:11.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:11.321 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: Deploying daemon mds.b on smithi148 2024-04-26T20:07:11.354 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:11.388 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] up:boot 2024-04-26T20:07:11.421 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: fsmap 11 up:standby 2024-04-26T20:07:11.554 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T20:07:11.588 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:08 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/928912639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:11.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:11.654 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:11.711 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:11.746 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:11.780 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:11.813 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:11.864 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: Deploying daemon mds.b on smithi148 2024-04-26T20:07:11.904 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:11.980 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] up:boot 2024-04-26T20:07:12.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: fsmap 11 up:standby 2024-04-26T20:07:12.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T20:07:12.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:08 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/928912639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:12.114 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:12.147 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:12.214 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:12.247 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T20:07:12.280 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T20:07:12.314 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:12.347 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: Deploying daemon mds.b on smithi148 2024-04-26T20:07:12.401 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:12.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] up:boot 2024-04-26T20:07:12.464 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: fsmap 11 up:standby 2024-04-26T20:07:12.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T20:07:12.555 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:11 smithi131 ceph-mon[27919]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:12.596 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:11 smithi148 ceph-mon[32281]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:12.651 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:11 smithi157 ceph-mon[32703]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:12.681 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:07:12 smithi148 systemd[1]: Started Ceph mds.b for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:07:12.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:07:12.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 -- 172.21.15.131:0/3451864539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744102980 msgr2=0x7f1744102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 --2- 172.21.15.131:0/3451864539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744102980 0x7f1744102da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1738009900 tx=0x7f173802a410 comp rx=0 tx=0).stop 2024-04-26T20:07:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 -- 172.21.15.131:0/3451864539 shutdown_connections 2024-04-26T20:07:13.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 --2- 172.21.15.131:0/3451864539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1744104530 0x7f174410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:13.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 --2- 172.21.15.131:0/3451864539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1744103b70 0x7f1744103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 --2- 172.21.15.131:0/3451864539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744102980 0x7f1744102da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:13.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 -- 172.21.15.131:0/3451864539 >> 172.21.15.131:0/3451864539 conn(0x7f17440fdf10 msgr2=0x7f1744100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:13.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 -- 172.21.15.131:0/3451864539 shutdown_connections 2024-04-26T20:07:13.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.943+0000 7f174c600640 1 -- 172.21.15.131:0/3451864539 wait complete. 2024-04-26T20:07:13.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.944+0000 7f174c600640 1 Processor -- start 2024-04-26T20:07:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.944+0000 7f174c600640 1 -- start start 2024-04-26T20:07:13.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174c600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1744102980 0x7f174419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174c600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1744103b70 0x7f174419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:13.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744104530 0x7f17441a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:13.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174c600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f174410e500 con 0x7f1744103b70 2024-04-26T20:07:13.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174c600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f174410e380 con 0x7f1744102980 2024-04-26T20:07:13.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f174410e680 con 0x7f1744104530 2024-04-26T20:07:13.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744104530 0x7f17441a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:13.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f1749c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1744102980 0x7f174419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:13.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744104530 0x7f17441a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40562/0 (socket says 172.21.15.131:40562) 2024-04-26T20:07:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f1749c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1744102980 0x7f174419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:37390/0 (socket says 172.21.15.131:37390) 2024-04-26T20:07:13.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 -- 172.21.15.131:0/4075685332 learned_addr learned my addr 172.21.15.131:0/4075685332 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:14.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f1749200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1744103b70 0x7f174419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:14.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 -- 172.21.15.131:0/4075685332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1744102980 msgr2=0x7f174419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:14.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1744102980 0x7f174419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:14.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 -- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1744103b70 msgr2=0x7f174419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:14.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1744103b70 0x7f174419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:14.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 -- 172.21.15.131:0/4075685332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17441a2a30 con 0x7f1744104530 2024-04-26T20:07:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.945+0000 7f174a600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744104530 0x7f17441a23c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f173c00eb10 tx=0x7f173c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.946+0000 7f1742000640 1 -- 172.21.15.131:0/4075685332 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c004b20 con 0x7f1744104530 2024-04-26T20:07:14.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.946+0000 7f1742000640 1 -- 172.21.15.131:0/4075685332 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173c004cc0 con 0x7f1744104530 2024-04-26T20:07:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.946+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17441a2d20 con 0x7f1744104530 2024-04-26T20:07:14.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.946+0000 7f1742000640 1 -- 172.21.15.131:0/4075685332 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c010670 con 0x7f1744104530 2024-04-26T20:07:14.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.946+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17441a3210 con 0x7f1744104530 2024-04-26T20:07:14.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.947+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17441a34a0 con 0x7f1744104530 2024-04-26T20:07:14.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.949+0000 7f1742000640 1 -- 172.21.15.131:0/4075685332 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f173c005020 con 0x7f1744104530 2024-04-26T20:07:14.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.950+0000 7f1742000640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f172c0a3bb0 0x7f172c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:14.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.950+0000 7f1742000640 1 -- 172.21.15.131:0/4075685332 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f173c0d0520 con 0x7f1744104530 2024-04-26T20:07:14.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.951+0000 7f1749c00640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f172c0a3bb0 0x7f172c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:14.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.951+0000 7f1749c00640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f172c0a3bb0 0x7f172c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1738005ab0 tx=0x7f1738005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:14.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:12.954+0000 7f1742000640 1 -- 172.21.15.131:0/4075685332 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f173c09d300 con 0x7f1744104530 2024-04-26T20:07:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.204+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17441a3800 con 0x7f1744104530 2024-04-26T20:07:14.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.205+0000 7f1742000640 1 -- 172.21.15.131:0/4075685332 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10506 (secure 0 0 0) 0x7f173c09cc00 con 0x7f1744104530 2024-04-26T20:07:14.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:14.874 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T20:07:14.908 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:14.956 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.064 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.099 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.174 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:15.207 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] up:boot 2024-04-26T20:07:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: fsmap 12 up:standby 2024-04-26T20:07:15.337 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T20:07:15.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:13 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4075685332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:15.399 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.465 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.499 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.556 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:15.591 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:15.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] up:boot 2024-04-26T20:07:15.683 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: fsmap 12 up:standby 2024-04-26T20:07:15.735 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T20:07:15.786 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:13 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4075685332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:15.849 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2024-04-26T20:07:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f172c0a3bb0 msgr2=0x7f172c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:16.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f172c0a3bb0 0x7f172c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1738005ab0 tx=0x7f1738005a20 comp rx=0 tx=0).stop 2024-04-26T20:07:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744104530 msgr2=0x7f17441a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:16.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744104530 0x7f17441a23c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f173c00eb10 tx=0x7f173c00c2d0 comp rx=0 tx=0).stop 2024-04-26T20:07:16.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 shutdown_connections 2024-04-26T20:07:16.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f172c0a3bb0 0x7f172c0a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1744104530 0x7f17441a23c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1744103b70 0x7f174419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:16.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 --2- 172.21.15.131:0/4075685332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1744102980 0x7f174419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.207+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 >> 172.21.15.131:0/4075685332 conn(0x7f17440fdf10 msgr2=0x7f1744074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:16.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.208+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 shutdown_connections 2024-04-26T20:07:16.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:13.208+0000 7f174c600640 1 -- 172.21.15.131:0/4075685332 wait complete. 2024-04-26T20:07:16.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:07:16.542 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:16.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:16.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:16.673 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:16.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:16.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:16.783 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] up:boot 2024-04-26T20:07:16.816 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: fsmap 12 up:standby 2024-04-26T20:07:16.850 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T20:07:16.883 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:13 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4075685332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:16.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:14 smithi131 ceph-mon[27919]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:16.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:16.983 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.137 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.175 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.242 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:17.350 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:17.422 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.458 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:16 smithi131 ceph-mon[27919]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:17.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:17.525 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:14 smithi157 ceph-mon[32703]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:17.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.592 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.625 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.659 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.692 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.725 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.821 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:17.859 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:17.892 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:17.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:16 smithi157 ceph-mon[32703]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:18.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:18.128 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:14 smithi148 ceph-mon[32281]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:18.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.234 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.293 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.476 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:18.559 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:18.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.643 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:16 smithi148 ceph-mon[32281]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:18.703 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:18.744 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:18.851 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:18 smithi157 ceph-mon[32703]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:18.896 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:18 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1700283583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:18.947 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:19.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:19.063 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:19.093 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:19.126 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:18 smithi148 ceph-mon[32281]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:19.160 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:18 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1700283583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:19.193 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:19.227 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 -- 172.21.15.131:0/1035430632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4105d80 msgr2=0x7fcbe4106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1035430632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4105d80 0x7fcbe4106200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd002a410 comp rx=0 tx=0).stop 2024-04-26T20:07:19.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 -- 172.21.15.131:0/1035430632 shutdown_connections 2024-04-26T20:07:19.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1035430632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe4106740 0x7fcbe410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:19.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1035430632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4105d80 0x7fcbe4106200 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1035430632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbe4104b90 0x7fcbe4104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:19.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 -- 172.21.15.131:0/1035430632 >> 172.21.15.131:0/1035430632 conn(0x7fcbe4100120 msgr2=0x7fcbe4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:19.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 -- 172.21.15.131:0/1035430632 shutdown_connections 2024-04-26T20:07:19.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.404+0000 7fcbe9400640 1 -- 172.21.15.131:0/1035430632 wait complete. 2024-04-26T20:07:19.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.405+0000 7fcbe9400640 1 Processor -- start 2024-04-26T20:07:19.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe9400640 1 -- start start 2024-04-26T20:07:19.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4104b90 0x7fcbe419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:19.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe4105d80 0x7fcbe419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:19.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbe4106740 0x7fcbe41a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:19.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe9400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe41106f0 con 0x7fcbe4105d80 2024-04-26T20:07:19.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe9400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4110570 con 0x7fcbe4106740 2024-04-26T20:07:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4110870 con 0x7fcbe4104b90 2024-04-26T20:07:19.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4104b90 0x7fcbe419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4104b90 0x7fcbe419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47640/0 (socket says 172.21.15.131:47640) 2024-04-26T20:07:20.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 -- 172.21.15.131:0/1700283583 learned_addr learned my addr 172.21.15.131:0/1700283583 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:20.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe1e00640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe4105d80 0x7fcbe419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:20.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe3200640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbe4106740 0x7fcbe41a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:20.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 -- 172.21.15.131:0/1700283583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbe4106740 msgr2=0x7fcbe41a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbe4106740 0x7fcbe41a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 -- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe4105d80 msgr2=0x7fcbe419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe4105d80 0x7fcbe419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:20.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.406+0000 7fcbe2800640 1 -- 172.21.15.131:0/1700283583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe41a29d0 con 0x7fcbe4104b90 2024-04-26T20:07:20.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.407+0000 7fcbe2800640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4104b90 0x7fcbe419adc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400e9c0 tx=0x7fcbd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:20.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.407+0000 7fcbdb400640 1 -- 172.21.15.131:0/1700283583 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400cd80 con 0x7fcbe4104b90 2024-04-26T20:07:20.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.407+0000 7fcbdb400640 1 -- 172.21.15.131:0/1700283583 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd40035f0 con 0x7fcbe4104b90 2024-04-26T20:07:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.407+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe41a2cc0 con 0x7fcbe4104b90 2024-04-26T20:07:20.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.407+0000 7fcbdb400640 1 -- 172.21.15.131:0/1700283583 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4010690 con 0x7fcbe4104b90 2024-04-26T20:07:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.407+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe41a31c0 con 0x7fcbe4104b90 2024-04-26T20:07:20.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.408+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe410beb0 con 0x7fcbe4104b90 2024-04-26T20:07:20.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.410+0000 7fcbdb400640 1 -- 172.21.15.131:0/1700283583 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcbd4004020 con 0x7fcbe4104b90 2024-04-26T20:07:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.411+0000 7fcbdb400640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcbc80a3bb0 0x7fcbc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:20.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.411+0000 7fcbdb400640 1 -- 172.21.15.131:0/1700283583 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcbd40d03f0 con 0x7fcbe4104b90 2024-04-26T20:07:20.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.411+0000 7fcbe1e00640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcbc80a3bb0 0x7fcbc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.412+0000 7fcbe1e00640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcbc80a3bb0 0x7fcbc80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0031500 tx=0x7fcbd0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.415+0000 7fcbdb400640 1 -- 172.21.15.131:0/1700283583 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcbd409d1d0 con 0x7fcbe4104b90 2024-04-26T20:07:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.622+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbe40635d0 con 0x7fcbe4104b90 2024-04-26T20:07:21.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.624+0000 7fcbdb400640 1 -- 172.21.15.131:0/1700283583 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18515 (secure 0 0 0) 0x7fcbd409cad0 con 0x7fcbe4104b90 2024-04-26T20:07:21.218 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:21.272 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:06:19.818466+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:07:21.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcbc80a3bb0 msgr2=0x7fcbc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:21.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcbc80a3bb0 0x7fcbc80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0031500 tx=0x7fcbd0005a20 comp rx=0 tx=0).stop 2024-04-26T20:07:21.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4104b90 msgr2=0x7fcbe419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:21.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4104b90 0x7fcbe419adc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400e9c0 tx=0x7fcbd400ee80 comp rx=0 tx=0).stop 2024-04-26T20:07:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 shutdown_connections 2024-04-26T20:07:21.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbe4106740 0x7fcbe41a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe4105d80 0x7fcbe419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:21.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcbc80a3bb0 0x7fcbc80a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:21.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 --2- 172.21.15.131:0/1700283583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe4104b90 0x7fcbe419adc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:21.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 >> 172.21.15.131:0/1700283583 conn(0x7fcbe4100120 msgr2=0x7fcbe4109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:21.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 shutdown_connections 2024-04-26T20:07:21.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:17.627+0000 7fcbe9400640 1 -- 172.21.15.131:0/1700283583 wait complete. 2024-04-26T20:07:21.874 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:07:21.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:07:22.000 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.054 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.090 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:18 smithi131 ceph-mon[27919]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:22.157 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:18 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1700283583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:22.199 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.240 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.324 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:22.408 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:22.469 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:19 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:20 smithi131 ceph-mon[27919]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:22.574 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.616 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.677 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:22.716 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:22.757 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:19 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.799 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:20 smithi148 ceph-mon[32281]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:22.858 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.909 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:22.956 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:22.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:23.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:19 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:07:23.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:20 smithi157 ceph-mon[32703]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:23.116 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:21 smithi157 ceph-mon[32703]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:23.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:21 smithi131 ceph-mon[27919]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:23.200 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:21 smithi148 ceph-mon[32281]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:23.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 -- 172.21.15.131:0/1528322843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d4078530 msgr2=0x7f68d4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 --2- 172.21.15.131:0/1528322843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d4078530 0x7f68d4078990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009900 tx=0x7f68c002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:07:23.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 -- 172.21.15.131:0/1528322843 shutdown_connections 2024-04-26T20:07:23.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 --2- 172.21.15.131:0/1528322843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 0x7f68d4079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:23.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 --2- 172.21.15.131:0/1528322843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d4078530 0x7f68d4078990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:23.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 --2- 172.21.15.131:0/1528322843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d40772f0 0x7f68d4077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 -- 172.21.15.131:0/1528322843 >> 172.21.15.131:0/1528322843 conn(0x7f68d40fffe0 msgr2=0x7f68d4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 -- 172.21.15.131:0/1528322843 shutdown_connections 2024-04-26T20:07:23.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.791+0000 7f68d9600640 1 -- 172.21.15.131:0/1528322843 wait complete. 2024-04-26T20:07:23.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.792+0000 7f68d9600640 1 Processor -- start 2024-04-26T20:07:23.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.792+0000 7f68d9600640 1 -- start start 2024-04-26T20:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d9600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40772f0 0x7f68d419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:23.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d4078530 0x7f68d419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:23.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d9600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 0x7f68d41a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:23.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d9600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41149f0 con 0x7f68d4078530 2024-04-26T20:07:23.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d9600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4114870 con 0x7f68d40772f0 2024-04-26T20:07:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d9600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4114b70 con 0x7f68d4078ed0 2024-04-26T20:07:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40772f0 0x7f68d419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:24.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 0x7f68d41a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:24.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40772f0 0x7f68d419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:49728/0 (socket says 172.21.15.131:49728) 2024-04-26T20:07:24.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 0x7f68d41a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47664/0 (socket says 172.21.15.131:47664) 2024-04-26T20:07:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d3e00640 1 -- 172.21.15.131:0/3633184859 learned_addr learned my addr 172.21.15.131:0/3633184859 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d3400640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d4078530 0x7f68d419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:24.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d8c00640 1 -- 172.21.15.131:0/3633184859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40772f0 msgr2=0x7f68d419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:24.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d8c00640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40772f0 0x7f68d419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d8c00640 1 -- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d4078530 msgr2=0x7f68d419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:24.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d8c00640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d4078530 0x7f68d419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:24.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d8c00640 1 -- 172.21.15.131:0/3633184859 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d41a6ca0 con 0x7f68d4078ed0 2024-04-26T20:07:24.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.793+0000 7f68d3e00640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40772f0 0x7f68d419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:07:24.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.794+0000 7f68d8c00640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 0x7f68d41a6630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009fd0 tx=0x7f68cc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:24.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.794+0000 7f68d0c00640 1 -- 172.21.15.131:0/3633184859 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc00ce20 con 0x7f68d4078ed0 2024-04-26T20:07:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.794+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d41a6f90 con 0x7f68d4078ed0 2024-04-26T20:07:24.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.794+0000 7f68d0c00640 1 -- 172.21.15.131:0/3633184859 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc005020 con 0x7f68d4078ed0 2024-04-26T20:07:24.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.794+0000 7f68d0c00640 1 -- 172.21.15.131:0/3633184859 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc010690 con 0x7f68d4078ed0 2024-04-26T20:07:24.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.794+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d41a7450 con 0x7f68d4078ed0 2024-04-26T20:07:24.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.795+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d41113e0 con 0x7f68d4078ed0 2024-04-26T20:07:24.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.798+0000 7f68d0c00640 1 -- 172.21.15.131:0/3633184859 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f68cc0054d0 con 0x7f68d4078ed0 2024-04-26T20:07:24.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.799+0000 7f68d0c00640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f68b80a3bb0 0x7f68b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:25.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.799+0000 7f68d0c00640 1 -- 172.21.15.131:0/3633184859 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f68cc0d0640 con 0x7f68d4078ed0 2024-04-26T20:07:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.800+0000 7f68d3e00640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f68b80a3bb0 0x7f68b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:25.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.800+0000 7f68d3e00640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f68b80a3bb0 0x7f68b80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68c40095f0 tx=0x7f68c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:25.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:22.801+0000 7f68d0c00640 1 -- 172.21.15.131:0/3633184859 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f68cc09d420 con 0x7f68d4078ed0 2024-04-26T20:07:25.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.001+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f68d402dad0 con 0x7f68d4078ed0 2024-04-26T20:07:25.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.002+0000 7f68d0c00640 1 -- 172.21.15.131:0/3633184859 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f68cc09cd20 con 0x7f68d4078ed0 2024-04-26T20:07:25.367 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:07:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f68b80a3bb0 msgr2=0x7f68b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f68b80a3bb0 0x7f68b80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68c40095f0 tx=0x7f68c4005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:25.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 msgr2=0x7f68d41a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:25.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 0x7f68d41a6630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009fd0 tx=0x7f68cc00ef10 comp rx=0 tx=0).stop 2024-04-26T20:07:25.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 shutdown_connections 2024-04-26T20:07:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f68b80a3bb0 0x7f68b80a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68d4078ed0 0x7f68d41a6630 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:25.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d4078530 0x7f68d419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 --2- 172.21.15.131:0/3633184859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40772f0 0x7f68d419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:25.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.005+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 >> 172.21.15.131:0/3633184859 conn(0x7f68d40fffe0 msgr2=0x7f68d4101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:25.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.006+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 shutdown_connections 2024-04-26T20:07:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:23.006+0000 7f68d9600640 1 -- 172.21.15.131:0/3633184859 wait complete. 2024-04-26T20:07:25.960 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:07:26.002 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:23 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3633184859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:07:26.063 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:24 smithi131 ceph-mon[27919]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:26.096 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:23 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3633184859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:07:26.139 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:24 smithi157 ceph-mon[32703]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:26.176 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:25 smithi157 ceph-mon[32703]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:26.218 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:23 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3633184859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:07:26.259 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:24 smithi148 ceph-mon[32281]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:26.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:25 smithi148 ceph-mon[32281]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:26.436 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:25 smithi131 ceph-mon[27919]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:26.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.640+0000 7f9779a00640 1 -- 172.21.15.131:0/2930707997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97740772f0 msgr2=0x7f9774077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:26.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.640+0000 7f9779a00640 1 --2- 172.21.15.131:0/2930707997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97740772f0 0x7f9774077710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f976c009900 tx=0x7f976c02a410 comp rx=0 tx=0).stop 2024-04-26T20:07:26.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.641+0000 7f9779a00640 1 -- 172.21.15.131:0/2930707997 shutdown_connections 2024-04-26T20:07:26.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.641+0000 7f9779a00640 1 --2- 172.21.15.131:0/2930707997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774078ed0 0x7f9774079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:26.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.641+0000 7f9779a00640 1 --2- 172.21.15.131:0/2930707997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9774078530 0x7f9774078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:26.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.641+0000 7f9779a00640 1 --2- 172.21.15.131:0/2930707997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97740772f0 0x7f9774077710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:26.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.641+0000 7f9779a00640 1 -- 172.21.15.131:0/2930707997 >> 172.21.15.131:0/2930707997 conn(0x7f9774100120 msgr2=0x7f9774102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:26.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.641+0000 7f9779a00640 1 -- 172.21.15.131:0/2930707997 shutdown_connections 2024-04-26T20:07:26.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.641+0000 7f9779a00640 1 -- 172.21.15.131:0/2930707997 wait complete. 2024-04-26T20:07:27.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 Processor -- start 2024-04-26T20:07:27.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 -- start start 2024-04-26T20:07:27.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97740772f0 0x7f977419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9774078530 0x7f977419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9774078ed0 0x7f97741a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:27.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9774114b10 con 0x7f9774078ed0 2024-04-26T20:07:27.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9774114990 con 0x7f9774078530 2024-04-26T20:07:27.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.642+0000 7f9779a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9774114c90 con 0x7f97740772f0 2024-04-26T20:07:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9772a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97740772f0 0x7f977419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:27.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9773400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9774078ed0 0x7f97741a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:27.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9772000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9774078530 0x7f977419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:27.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9772a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97740772f0 0x7f977419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56610/0 (socket says 172.21.15.131:56610) 2024-04-26T20:07:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9772a00640 1 -- 172.21.15.131:0/93433186 learned_addr learned my addr 172.21.15.131:0/93433186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:27.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9773400640 1 -- 172.21.15.131:0/93433186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97740772f0 msgr2=0x7f977419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:27.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9773400640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97740772f0 0x7f977419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9773400640 1 -- 172.21.15.131:0/93433186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9774078530 msgr2=0x7f977419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:27.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9773400640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9774078530 0x7f977419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9773400640 1 -- 172.21.15.131:0/93433186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97741a6dc0 con 0x7f9774078ed0 2024-04-26T20:07:27.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9772a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97740772f0 0x7f977419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:07:27.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9772000640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9774078530 0x7f977419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:07:27.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.643+0000 7f9773400640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9774078ed0 0x7f97741a6750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f976400d680 tx=0x7f976400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:28.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.644+0000 7f976b400640 1 -- 172.21.15.131:0/93433186 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764017070 con 0x7f9774078ed0 2024-04-26T20:07:28.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.644+0000 7f976b400640 1 -- 172.21.15.131:0/93433186 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9764009820 con 0x7f9774078ed0 2024-04-26T20:07:28.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.644+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741a70b0 con 0x7f9774078ed0 2024-04-26T20:07:28.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.644+0000 7f976b400640 1 -- 172.21.15.131:0/93433186 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764004ce0 con 0x7f9774078ed0 2024-04-26T20:07:28.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.644+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741a75b0 con 0x7f9774078ed0 2024-04-26T20:07:28.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.645+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9774111700 con 0x7f9774078ed0 2024-04-26T20:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.647+0000 7f976b400640 1 -- 172.21.15.131:0/93433186 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f97640099d0 con 0x7f9774078ed0 2024-04-26T20:07:28.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.648+0000 7f976b400640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97580a3bb0 0x7f97580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.649+0000 7f976b400640 1 -- 172.21.15.131:0/93433186 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f97640cf840 con 0x7f9774078ed0 2024-04-26T20:07:28.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.649+0000 7f9772a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97580a3bb0 0x7f97580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:28.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.650+0000 7f9772a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97580a3bb0 0x7f97580a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f976c0096c0 tx=0x7f976c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:28.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.653+0000 7f976b400640 1 -- 172.21.15.131:0/93433186 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f976409bfd0 con 0x7f9774078ed0 2024-04-26T20:07:28.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.854+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9774063530 con 0x7f9774078ed0 2024-04-26T20:07:28.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.855+0000 7f976b400640 1 -- 172.21.15.131:0/93433186 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f97640a02d0 con 0x7f9774078ed0 2024-04-26T20:07:28.687 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:28.730 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-04-26T20:07:28.795 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:07:28.836 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-04-26T20:07:28.878 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-04-26T20:07:28.920 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:07:28.961 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-04-26T20:07:29.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:26 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/93433186' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T20:07:29.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:27 smithi148 ceph-mon[32281]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:29.086 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:26 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/93433186' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T20:07:29.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:27 smithi157 ceph-mon[32703]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97580a3bb0 msgr2=0x7f97580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:29.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97580a3bb0 0x7f97580a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f976c0096c0 tx=0x7f976c005a20 comp rx=0 tx=0).stop 2024-04-26T20:07:29.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9774078ed0 msgr2=0x7f97741a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9774078ed0 0x7f97741a6750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f976400d680 tx=0x7f976400db40 comp rx=0 tx=0).stop 2024-04-26T20:07:29.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 shutdown_connections 2024-04-26T20:07:29.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9774078ed0 0x7f97741a6750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:29.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9774078530 0x7f977419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97580a3bb0 0x7f97580a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:29.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.857+0000 7f9779a00640 1 --2- 172.21.15.131:0/93433186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97740772f0 0x7f977419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:29.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.858+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 >> 172.21.15.131:0/93433186 conn(0x7f9774100120 msgr2=0x7f9774101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:29.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.858+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 shutdown_connections 2024-04-26T20:07:29.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:26.858+0000 7f9779a00640 1 -- 172.21.15.131:0/93433186 wait complete. 2024-04-26T20:07:29.754 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:07:29.798 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:07:29.838 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:26 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/93433186' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T20:07:29.879 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:27 smithi131 ceph-mon[27919]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:29.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:29 smithi157 ceph-mon[32703]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:29.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:29 smithi131 ceph-mon[27919]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:30.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:29 smithi148 ceph-mon[32281]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:30.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 -- 172.21.15.131:0/1466397910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c0772f0 msgr2=0x7f189c077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:30.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 --2- 172.21.15.131:0/1466397910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c0772f0 0x7f189c077710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f188c009be0 tx=0x7f188c02a6c0 comp rx=0 tx=0).stop 2024-04-26T20:07:30.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 -- 172.21.15.131:0/1466397910 shutdown_connections 2024-04-26T20:07:30.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 --2- 172.21.15.131:0/1466397910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c078ed0 0x7f189c079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:30.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 --2- 172.21.15.131:0/1466397910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f189c078530 0x7f189c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:30.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 --2- 172.21.15.131:0/1466397910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c0772f0 0x7f189c077710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:30.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 -- 172.21.15.131:0/1466397910 >> 172.21.15.131:0/1466397910 conn(0x7f189c100120 msgr2=0x7f189c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:30.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 -- 172.21.15.131:0/1466397910 shutdown_connections 2024-04-26T20:07:30.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.137+0000 7f18a1600640 1 -- 172.21.15.131:0/1466397910 wait complete. 2024-04-26T20:07:30.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.138+0000 7f18a1600640 1 Processor -- start 2024-04-26T20:07:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f18a1600640 1 -- start start 2024-04-26T20:07:30.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f18a1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c0772f0 0x7f189c19f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:30.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f18a1600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f189c078530 0x7f189c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f18a1600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c078ed0 0x7f189c1a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:30.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f18a1600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f189c114ad0 con 0x7f189c0772f0 2024-04-26T20:07:30.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f18a1600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f189c114950 con 0x7f189c078ed0 2024-04-26T20:07:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f18a1600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f189c114c50 con 0x7f189c078530 2024-04-26T20:07:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f189aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c0772f0 0x7f189c19f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:30.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f189aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c0772f0 0x7f189c19f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51124/0 (socket says 172.21.15.131:51124) 2024-04-26T20:07:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f189aa00640 1 -- 172.21.15.131:0/1590131530 learned_addr learned my addr 172.21.15.131:0/1590131530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:31.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f1892000640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f189c078530 0x7f189c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:31.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.139+0000 7f189b400640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c078ed0 0x7f189c1a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:31.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f189aa00640 1 -- 172.21.15.131:0/1590131530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f189c078530 msgr2=0x7f189c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:31.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f189aa00640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f189c078530 0x7f189c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:31.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f189aa00640 1 -- 172.21.15.131:0/1590131530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c078ed0 msgr2=0x7f189c1a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:31.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f189aa00640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c078ed0 0x7f189c1a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:31.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f189aa00640 1 -- 172.21.15.131:0/1590131530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1a6e60 con 0x7f189c0772f0 2024-04-26T20:07:31.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f189aa00640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c0772f0 0x7f189c19f240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f188c034040 tx=0x7f188c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:31.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f1893e00640 1 -- 172.21.15.131:0/1590131530 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c041020 con 0x7f189c0772f0 2024-04-26T20:07:31.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a70f0 con 0x7f189c0772f0 2024-04-26T20:07:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f1893e00640 1 -- 172.21.15.131:0/1590131530 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f188c0318f0 con 0x7f189c0772f0 2024-04-26T20:07:31.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f1893e00640 1 -- 172.21.15.131:0/1590131530 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c036040 con 0x7f189c0772f0 2024-04-26T20:07:31.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.140+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a7630 con 0x7f189c0772f0 2024-04-26T20:07:31.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.142+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c111500 con 0x7f189c0772f0 2024-04-26T20:07:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.144+0000 7f1893e00640 1 -- 172.21.15.131:0/1590131530 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f188c044020 con 0x7f189c0772f0 2024-04-26T20:07:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.146+0000 7f1893e00640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.147+0000 7f1893e00640 1 -- 172.21.15.131:0/1590131530 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f188c0f1080 con 0x7f189c0772f0 2024-04-26T20:07:31.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.147+0000 7f1892000640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:32.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.147+0000 7f1892000640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18800a3bb0 0x7f18800a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f189c1a0750 tx=0x7f1884005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:32.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.149+0000 7f1893e00640 1 -- 172.21.15.131:0/1590131530 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f188c04b020 con 0x7f189c0772f0 2024-04-26T20:07:32.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.352+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f189c02dad0 con 0x7f189c0772f0 2024-04-26T20:07:32.148 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:30 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1590131530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T20:07:32.188 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:31 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1590131530' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T20:07:32.222 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:31 smithi148 ceph-mon[32281]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T20:07:32.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:31 smithi148 ceph-mon[32281]: pgmap v321: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:32.299 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:30 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1590131530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T20:07:32.338 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:31 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1590131530' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T20:07:32.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:31 smithi157 ceph-mon[32703]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T20:07:32.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:31 smithi157 ceph-mon[32703]: pgmap v321: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:32.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.678+0000 7f1893e00640 1 -- 172.21.15.131:0/1590131530 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f188c0bde60 con 0x7f189c0772f0 2024-04-26T20:07:32.520 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-04-26T20:07:32.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18800a3bb0 msgr2=0x7f18800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:32.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18800a3bb0 0x7f18800a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f189c1a0750 tx=0x7f1884005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:32.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c0772f0 msgr2=0x7f189c19f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c0772f0 0x7f189c19f240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f188c034040 tx=0x7f188c030d00 comp rx=0 tx=0).stop 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 shutdown_connections 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c078ed0 0x7f189c1a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f189c078530 0x7f189c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 --2- 172.21.15.131:0/1590131530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c0772f0 0x7f189c19f240 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f188c034040 tx=0x7f188c030d00 comp rx=0 tx=0).stop 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.684+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 >> 172.21.15.131:0/1590131530 conn(0x7f189c100120 msgr2=0x7f189c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.685+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 shutdown_connections 2024-04-26T20:07:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:30.686+0000 7f18a1600640 1 -- 172.21.15.131:0/1590131530 wait complete. 2024-04-26T20:07:32.658 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T20:07:32.659 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:30 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1590131530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T20:07:32.659 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:31 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1590131530' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T20:07:32.659 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:31 smithi131 ceph-mon[27919]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T20:07:32.659 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:31 smithi131 ceph-mon[27919]: pgmap v321: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 -- 172.21.15.131:0/3381679255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 msgr2=0x7fcd5c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 --2- 172.21.15.131:0/3381679255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 0x7fcd5c102da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009860 tx=0x7fcd4c02a340 comp rx=0 tx=0).stop 2024-04-26T20:07:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 -- 172.21.15.131:0/3381679255 shutdown_connections 2024-04-26T20:07:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 --2- 172.21.15.131:0/3381679255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd5c104530 0x7fcd5c10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 --2- 172.21.15.131:0/3381679255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 0x7fcd5c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 --2- 172.21.15.131:0/3381679255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 0x7fcd5c102da0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 -- 172.21.15.131:0/3381679255 >> 172.21.15.131:0/3381679255 conn(0x7fcd5c0fdf10 msgr2=0x7fcd5c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 -- 172.21.15.131:0/3381679255 shutdown_connections 2024-04-26T20:07:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.873+0000 7fcd61800640 1 -- 172.21.15.131:0/3381679255 wait complete. 2024-04-26T20:07:32.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.874+0000 7fcd61800640 1 Processor -- start 2024-04-26T20:07:32.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.874+0000 7fcd61800640 1 -- start start 2024-04-26T20:07:32.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd61800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 0x7fcd5c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:32.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd61800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 0x7fcd5c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:32.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd61800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd5c104530 0x7fcd5c1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd61800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10e4b0 con 0x7fcd5c103b70 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd61800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10e330 con 0x7fcd5c104530 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd61800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10e630 con 0x7fcd5c102980 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd52000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 0x7fcd5c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd52000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 0x7fcd5c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51134/0 (socket says 172.21.15.131:51134) 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd52000640 1 -- 172.21.15.131:0/3699848863 learned_addr learned my addr 172.21.15.131:0/3699848863 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd5aa00640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 0x7fcd5c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.875+0000 7fcd5b400640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd5c104530 0x7fcd5c1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd5aa00640 1 -- 172.21.15.131:0/3699848863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd5c104530 msgr2=0x7fcd5c1a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd5aa00640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd5c104530 0x7fcd5c1a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd5aa00640 1 -- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 msgr2=0x7fcd5c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd5aa00640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 0x7fcd5c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd5aa00640 1 -- 172.21.15.131:0/3699848863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c1a2a20 con 0x7fcd5c102980 2024-04-26T20:07:32.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd52000640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 0x7fcd5c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:07:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd5aa00640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 0x7fcd5c19ae00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02a860 tx=0x7fcd4c032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd53e00640 1 -- 172.21.15.131:0/3699848863 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c037040 con 0x7fcd5c102980 2024-04-26T20:07:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd53e00640 1 -- 172.21.15.131:0/3699848863 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd4c030bd0 con 0x7fcd5c102980 2024-04-26T20:07:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd53e00640 1 -- 172.21.15.131:0/3699848863 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c003850 con 0x7fcd5c102980 2024-04-26T20:07:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c1a2cb0 con 0x7fcd5c102980 2024-04-26T20:07:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.876+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c1a30f0 con 0x7fcd5c102980 2024-04-26T20:07:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.878+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c1a32c0 con 0x7fcd5c102980 2024-04-26T20:07:32.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.879+0000 7fcd53e00640 1 -- 172.21.15.131:0/3699848863 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcd4c043020 con 0x7fcd5c102980 2024-04-26T20:07:32.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.881+0000 7fcd53e00640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcd400a3c30 0x7fcd400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:32.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.881+0000 7fcd53e00640 1 -- 172.21.15.131:0/3699848863 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7fcd4c0ef2e0 con 0x7fcd5c102980 2024-04-26T20:07:32.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.881+0000 7fcd52000640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcd400a3c30 0x7fcd400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.882+0000 7fcd52000640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcd400a3c30 0x7fcd400a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd440096c0 tx=0x7fcd44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:32.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:32.885+0000 7fcd53e00640 1 -- 172.21.15.131:0/3699848863 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcd4c04a060 con 0x7fcd5c102980 2024-04-26T20:07:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:32 smithi157 ceph-mon[32703]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T20:07:32.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:32 smithi157 ceph-mon[32703]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T20:07:33.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:32 smithi131 ceph-mon[27919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T20:07:33.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:32 smithi131 ceph-mon[27919]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T20:07:33.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:32 smithi148 ceph-mon[32281]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T20:07:33.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:32 smithi148 ceph-mon[32281]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T20:07:33.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.093+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fcd5c1a3490 con 0x7fcd5c102980 2024-04-26T20:07:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.725+0000 7fcd53e00640 1 -- 172.21.15.131:0/3699848863 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) ==== 171+0+0 (secure 0 0 0) 0x7fcd4c0bbf50 con 0x7fcd5c102980 2024-04-26T20:07:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-04-26T20:07:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcd400a3c30 msgr2=0x7fcd400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcd400a3c30 0x7fcd400a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd440096c0 tx=0x7fcd44009260 comp rx=0 tx=0).stop 2024-04-26T20:07:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 msgr2=0x7fcd5c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 0x7fcd5c19ae00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02a860 tx=0x7fcd4c032dc0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 shutdown_connections 2024-04-26T20:07:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd5c104530 0x7fcd5c1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd5c103b70 0x7fcd5c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fcd400a3c30 0x7fcd400a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 --2- 172.21.15.131:0/3699848863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5c102980 0x7fcd5c19ae00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.732+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 >> 172.21.15.131:0/3699848863 conn(0x7fcd5c0fdf10 msgr2=0x7fcd5c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.733+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 shutdown_connections 2024-04-26T20:07:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.733+0000 7fcd61800640 1 -- 172.21.15.131:0/3699848863 wait complete. 2024-04-26T20:07:33.761 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:07:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 -- 172.21.15.131:0/2353470242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8678108f90 msgr2=0x7f86781093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 --2- 172.21.15.131:0/2353470242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8678108f90 0x7f86781093b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8668009900 tx=0x7f866802a410 comp rx=0 tx=0).stop 2024-04-26T20:07:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 -- 172.21.15.131:0/2353470242 shutdown_connections 2024-04-26T20:07:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 --2- 172.21.15.131:0/2353470242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867810ab40 0x7f86781113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 --2- 172.21.15.131:0/2353470242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867810a180 0x7f867810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 --2- 172.21.15.131:0/2353470242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8678108f90 0x7f86781093b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 -- 172.21.15.131:0/2353470242 >> 172.21.15.131:0/2353470242 conn(0x7f8678077f30 msgr2=0x7f867807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:33.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 -- 172.21.15.131:0/2353470242 shutdown_connections 2024-04-26T20:07:33.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.940+0000 7f867ec00640 1 -- 172.21.15.131:0/2353470242 wait complete. 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.941+0000 7f867ec00640 1 Processor -- start 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867ec00640 1 -- start start 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8678108f90 0x7f867819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867810a180 0x7f867819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f867810ab40 0x7f86781a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8678114ae0 con 0x7f867810a180 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8678114960 con 0x7f867810ab40 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8678114c60 con 0x7f8678108f90 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f8677e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8678108f90 0x7f867819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f8677e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8678108f90 0x7f867819f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56210/0 (socket says 172.21.15.131:56210) 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f8677e00640 1 -- 172.21.15.131:0/1942094683 learned_addr learned my addr 172.21.15.131:0/1942094683 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f866fe00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867810a180 0x7f867819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.942+0000 7f867cc00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f867810ab40 0x7f86781a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8677e00640 1 -- 172.21.15.131:0/1942094683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f867810ab40 msgr2=0x7f86781a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8677e00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f867810ab40 0x7f86781a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8677e00640 1 -- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867810a180 msgr2=0x7f867819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8677e00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867810a180 0x7f867819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8677e00640 1 -- 172.21.15.131:0/1942094683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86781a6e30 con 0x7f8678108f90 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8677e00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8678108f90 0x7f867819f210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8668031500 tx=0x7f8668003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8675600640 1 -- 172.21.15.131:0/1942094683 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866803f070 con 0x7f8678108f90 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86781a70c0 con 0x7f8678108f90 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8675600640 1 -- 172.21.15.131:0/1942094683 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8668031c50 con 0x7f8678108f90 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f8675600640 1 -- 172.21.15.131:0/1942094683 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668035030 con 0x7f8678108f90 2024-04-26T20:07:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.943+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86781a7600 con 0x7f8678108f90 2024-04-26T20:07:33.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.944+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8678110320 con 0x7f8678108f90 2024-04-26T20:07:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.947+0000 7f8675600640 1 -- 172.21.15.131:0/1942094683 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8668042070 con 0x7f8678108f90 2024-04-26T20:07:33.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.948+0000 7f8675600640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f865c0a3bb0 0x7f865c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:33.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.948+0000 7f8675600640 1 -- 172.21.15.131:0/1942094683 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f86680f1110 con 0x7f8678108f90 2024-04-26T20:07:33.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.948+0000 7f866fe00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f865c0a3bb0 0x7f865c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:33.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.949+0000 7f866fe00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f865c0a3bb0 0x7f865c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86781a0720 tx=0x7f8660005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:33.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:33.950+0000 7f8675600640 1 -- 172.21.15.131:0/1942094683 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f86680bd5b0 con 0x7f8678108f90 2024-04-26T20:07:33.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:33 smithi157 ceph-mon[32703]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T20:07:33.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:33 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3699848863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T20:07:33.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:33 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T20:07:33.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:33 smithi157 ceph-mon[32703]: pgmap v324: 33 pgs: 3 creating+peering, 22 active+clean, 8 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:34.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:33 smithi131 ceph-mon[27919]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T20:07:34.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:33 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3699848863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T20:07:34.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:33 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T20:07:34.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:33 smithi131 ceph-mon[27919]: pgmap v324: 33 pgs: 3 creating+peering, 22 active+clean, 8 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:34.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:33 smithi148 ceph-mon[32281]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T20:07:34.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:33 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3699848863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T20:07:34.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:33 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T20:07:34.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:33 smithi148 ceph-mon[32281]: pgmap v324: 33 pgs: 3 creating+peering, 22 active+clean, 8 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:34.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.191+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f867802dad0 con 0x7f8678108f90 2024-04-26T20:07:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.727+0000 7f8675600640 1 -- 172.21.15.131:0/1942094683 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f86680c18b0 con 0x7f8678108f90 2024-04-26T20:07:34.729 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T20:07:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.733+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f865c0a3bb0 msgr2=0x7f865c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.733+0000 7f867ec00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f865c0a3bb0 0x7f865c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86781a0720 tx=0x7f8660005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.733+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8678108f90 msgr2=0x7f867819f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.733+0000 7f867ec00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8678108f90 0x7f867819f210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8668031500 tx=0x7f8668003c10 comp rx=0 tx=0).stop 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.734+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 shutdown_connections 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.734+0000 7f867ec00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f867810ab40 0x7f86781a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.734+0000 7f867ec00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867810a180 0x7f867819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.734+0000 7f867ec00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f865c0a3bb0 0x7f865c0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.734+0000 7f867ec00640 1 --2- 172.21.15.131:0/1942094683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8678108f90 0x7f867819f210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.734+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 >> 172.21.15.131:0/1942094683 conn(0x7f8678077f30 msgr2=0x7f867810d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.735+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 shutdown_connections 2024-04-26T20:07:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.735+0000 7f867ec00640 1 -- 172.21.15.131:0/1942094683 wait complete. 2024-04-26T20:07:34.775 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T20:07:34.776 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T20:07:34.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.953+0000 7f784f400640 1 -- 172.21.15.131:0/1980969495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 msgr2=0x7f784810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.953+0000 7f784f400640 1 --2- 172.21.15.131:0/1980969495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f784810cfd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7840009ca0 tx=0x7f784002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.953+0000 7f784f400640 1 -- 172.21.15.131:0/1980969495 shutdown_connections 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.953+0000 7f784f400640 1 --2- 172.21.15.131:0/1980969495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f784810cfd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.953+0000 7f784f400640 1 --2- 172.21.15.131:0/1980969495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7848105d80 0x7f7848106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.953+0000 7f784f400640 1 --2- 172.21.15.131:0/1980969495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7848104b90 0x7f7848104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.953+0000 7f784f400640 1 -- 172.21.15.131:0/1980969495 >> 172.21.15.131:0/1980969495 conn(0x7f7848100120 msgr2=0x7f7848102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.954+0000 7f784f400640 1 -- 172.21.15.131:0/1980969495 shutdown_connections 2024-04-26T20:07:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.954+0000 7f784f400640 1 -- 172.21.15.131:0/1980969495 wait complete. 2024-04-26T20:07:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 Processor -- start 2024-04-26T20:07:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 -- start start 2024-04-26T20:07:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7848104b90 0x7f784819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7848105d80 0x7f784819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f78481a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7848110710 con 0x7f7848106740 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7848110590 con 0x7f7848105d80 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7848110890 con 0x7f7848104b90 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f78481a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f78481a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36060/0 (socket says 172.21.15.131:36060) 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784d400640 1 -- 172.21.15.131:0/393838832 learned_addr learned my addr 172.21.15.131:0/393838832 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f784ca00640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7848104b90 0x7f784819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.955+0000 7f7847e00640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7848105d80 0x7f784819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f784d400640 1 -- 172.21.15.131:0/393838832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7848104b90 msgr2=0x7f784819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f784d400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7848104b90 0x7f784819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f784d400640 1 -- 172.21.15.131:0/393838832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7848105d80 msgr2=0x7f784819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f784d400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7848105d80 0x7f784819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f784d400640 1 -- 172.21.15.131:0/393838832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481a2950 con 0x7f7848106740 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f7847e00640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7848105d80 0x7f784819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f784d400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f78481a22b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7840033040 tx=0x7f78400316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f7845600640 1 -- 172.21.15.131:0/393838832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840003b70 con 0x7f7848106740 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78481a2be0 con 0x7f7848106740 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f7845600640 1 -- 172.21.15.131:0/393838832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840003d10 con 0x7f7848106740 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.956+0000 7f7845600640 1 -- 172.21.15.131:0/393838832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784002fb40 con 0x7f7848106740 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.957+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a3130 con 0x7f7848106740 2024-04-26T20:07:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.958+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784810beb0 con 0x7f7848106740 2024-04-26T20:07:34.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.960+0000 7f7845600640 1 -- 172.21.15.131:0/393838832 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7840004610 con 0x7f7848106740 2024-04-26T20:07:34.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.962+0000 7f7845600640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78300a3bb0 0x7f78300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:34.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.962+0000 7f7845600640 1 -- 172.21.15.131:0/393838832 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f78400f17a0 con 0x7f7848106740 2024-04-26T20:07:34.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.962+0000 7f784ca00640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78300a3bb0 0x7f78300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:34.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.963+0000 7f784ca00640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78300a3bb0 0x7f78300a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f784819c210 tx=0x7f783c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:34.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:34.967+0000 7f7845600640 1 -- 172.21.15.131:0/393838832 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f78400be230 con 0x7f7848106740 2024-04-26T20:07:35.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:34 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T20:07:35.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:34 smithi131 ceph-mon[27919]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T20:07:35.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:34 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1942094683' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T20:07:35.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:34 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T20:07:35.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:34 smithi131 ceph-mon[27919]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T20:07:35.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:34 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T20:07:35.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:34 smithi148 ceph-mon[32281]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T20:07:35.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:34 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1942094683' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T20:07:35.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:34 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T20:07:35.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:34 smithi148 ceph-mon[32281]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T20:07:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.179+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f7848063530 con 0x7f7848106740 2024-04-26T20:07:35.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:34 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T20:07:35.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:34 smithi157 ceph-mon[32703]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T20:07:35.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:34 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1942094683' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T20:07:35.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:34 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T20:07:35.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:34 smithi157 ceph-mon[32703]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T20:07:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.757+0000 7f7845600640 1 -- 172.21.15.131:0/393838832 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) ==== 157+0+0 (secure 0 0 0) 0x7f78400bdb30 con 0x7f7848106740 2024-04-26T20:07:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78300a3bb0 msgr2=0x7f78300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78300a3bb0 0x7f78300a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f784819c210 tx=0x7f783c005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 msgr2=0x7f78481a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f78481a22b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7840033040 tx=0x7f78400316f0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 shutdown_connections 2024-04-26T20:07:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7848106740 0x7f78481a22b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7848105d80 0x7f784819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78300a3bb0 0x7f78300a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 --2- 172.21.15.131:0/393838832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7848104b90 0x7f784819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 >> 172.21.15.131:0/393838832 conn(0x7f7848100120 msgr2=0x7f7848109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.761+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 shutdown_connections 2024-04-26T20:07:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.762+0000 7f784f400640 1 -- 172.21.15.131:0/393838832 wait complete. 2024-04-26T20:07:35.784 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:07:35.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 -- 172.21.15.131:0/55257364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc1027a0 msgr2=0x7f48cc102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:35.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 --2- 172.21.15.131:0/55257364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc1027a0 0x7f48cc102bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009be0 tx=0x7f48c402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 -- 172.21.15.131:0/55257364 shutdown_connections 2024-04-26T20:07:35.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 --2- 172.21.15.131:0/55257364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc104350 0x7f48cc10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 --2- 172.21.15.131:0/55257364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc103990 0x7f48cc103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 --2- 172.21.15.131:0/55257364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc1027a0 0x7f48cc102bc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 -- 172.21.15.131:0/55257364 >> 172.21.15.131:0/55257364 conn(0x7f48cc0fdd10 msgr2=0x7f48cc100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:35.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 -- 172.21.15.131:0/55257364 shutdown_connections 2024-04-26T20:07:35.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.993+0000 7f48d2200640 1 -- 172.21.15.131:0/55257364 wait complete. 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.994+0000 7f48d2200640 1 Processor -- start 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d2200640 1 -- start start 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc1027a0 0x7f48cc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d2200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc103990 0x7f48cc19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d2200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc104350 0x7f48cc1a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d2200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10e2a0 con 0x7f48cc104350 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d2200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10e120 con 0x7f48cc103990 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10e420 con 0x7f48cc1027a0 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc104350 0x7f48cc1a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc1027a0 0x7f48cc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc1027a0 0x7f48cc19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56240/0 (socket says 172.21.15.131:56240) 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48d0e00640 1 -- 172.21.15.131:0/1231072735 learned_addr learned my addr 172.21.15.131:0/1231072735 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.995+0000 7f48cbe00640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc103990 0x7f48cc19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d0e00640 1 -- 172.21.15.131:0/1231072735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc103990 msgr2=0x7f48cc19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d0e00640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc103990 0x7f48cc19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d0e00640 1 -- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc104350 msgr2=0x7f48cc1a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d0e00640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc104350 0x7f48cc1a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d0e00640 1 -- 172.21.15.131:0/1231072735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc1a2800 con 0x7f48cc1027a0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d0e00640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc1027a0 0x7f48cc19abe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000f80 tx=0x7f48c4030d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48c9600640 1 -- 172.21.15.131:0/1231072735 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4041020 con 0x7f48cc1027a0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc1a2a90 con 0x7f48cc1027a0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48c9600640 1 -- 172.21.15.131:0/1231072735 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c40318f0 con 0x7f48cc1027a0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48c9600640 1 -- 172.21.15.131:0/1231072735 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4036040 con 0x7f48cc1027a0 2024-04-26T20:07:35.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.996+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1a2f80 con 0x7f48cc1027a0 2024-04-26T20:07:35.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.997+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc1a31b0 con 0x7f48cc1027a0 2024-04-26T20:07:36.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:35.999+0000 7f48c9600640 1 -- 172.21.15.131:0/1231072735 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f48c4044020 con 0x7f48cc1027a0 2024-04-26T20:07:36.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:36.001+0000 7f48c9600640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f48b00a3bb0 0x7f48b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:36.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:36.001+0000 7f48c9600640 1 -- 172.21.15.131:0/1231072735 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8277+0+0 (secure 0 0 0) 0x7f48c40f1080 con 0x7f48cc1027a0 2024-04-26T20:07:36.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:36.001+0000 7f48cbe00640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f48b00a3bb0 0x7f48b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:36.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:36.002+0000 7f48cbe00640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f48b00a3bb0 0x7f48b00a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48cc19c0f0 tx=0x7f48b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:36.005+0000 7f48c9600640 1 -- 172.21.15.131:0/1231072735 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f48c404b020 con 0x7f48cc1027a0 2024-04-26T20:07:36.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:35 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T20:07:36.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:35 smithi148 ceph-mon[32281]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T20:07:36.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:35 smithi148 ceph-mon[32281]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T20:07:36.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:35 smithi148 ceph-mon[32281]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T20:07:36.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:35 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/393838832' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T20:07:36.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:35 smithi148 ceph-mon[32281]: pgmap v327: 97 pgs: 11 creating+peering, 41 active+clean, 45 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-26T20:07:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:36.209+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f48cc1a3380 con 0x7f48cc1027a0 2024-04-26T20:07:36.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:35 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T20:07:36.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:35 smithi157 ceph-mon[32703]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T20:07:36.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:35 smithi157 ceph-mon[32703]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T20:07:36.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:35 smithi157 ceph-mon[32703]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T20:07:36.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:35 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/393838832' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T20:07:36.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:35 smithi157 ceph-mon[32703]: pgmap v327: 97 pgs: 11 creating+peering, 41 active+clean, 45 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-26T20:07:36.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:35 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T20:07:36.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:35 smithi131 ceph-mon[27919]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T20:07:36.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:35 smithi131 ceph-mon[27919]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T20:07:36.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:35 smithi131 ceph-mon[27919]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T20:07:36.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:35 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/393838832' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T20:07:36.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:35 smithi131 ceph-mon[27919]: pgmap v327: 97 pgs: 11 creating+peering, 41 active+clean, 45 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-26T20:07:37.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: Cluster is now healthy 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] up:active 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/393838832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1231072735' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T20:07:37.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:36 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T20:07:37.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T20:07:37.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: Cluster is now healthy 2024-04-26T20:07:37.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] up:active 2024-04-26T20:07:37.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:37.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/393838832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T20:07:37.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T20:07:37.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:37.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1231072735' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T20:07:37.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:36 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T20:07:37.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T20:07:37.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: Cluster is now healthy 2024-04-26T20:07:37.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] up:active 2024-04-26T20:07:37.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:37.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/393838832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T20:07:37.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T20:07:37.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:37.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1231072735' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T20:07:37.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:36 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T20:07:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.789+0000 7f48c9600640 1 -- 172.21.15.131:0/1231072735 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) ==== 228+0+0 (secure 0 0 0) 0x7f48c40bd9d0 con 0x7f48cc1027a0 2024-04-26T20:07:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' created 2024-04-26T20:07:37.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.797+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f48b00a3bb0 msgr2=0x7f48b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:37.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.797+0000 7f48d2200640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f48b00a3bb0 0x7f48b00a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48cc19c0f0 tx=0x7f48b4005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:37.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.797+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc1027a0 msgr2=0x7f48cc19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:37.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.797+0000 7f48d2200640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc1027a0 0x7f48cc19abe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000f80 tx=0x7f48c4030d10 comp rx=0 tx=0).stop 2024-04-26T20:07:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.799+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 shutdown_connections 2024-04-26T20:07:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.799+0000 7f48d2200640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc104350 0x7f48cc1a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.799+0000 7f48d2200640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48cc103990 0x7f48cc19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:37.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.799+0000 7f48d2200640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f48b00a3bb0 0x7f48b00a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:37.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.799+0000 7f48d2200640 1 --2- 172.21.15.131:0/1231072735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48cc1027a0 0x7f48cc19abe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:37.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.799+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 >> 172.21.15.131:0/1231072735 conn(0x7f48cc0fdd10 msgr2=0x7f48cc109160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:37.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.800+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 shutdown_connections 2024-04-26T20:07:37.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:37.800+0000 7f48d2200640 1 -- 172.21.15.131:0/1231072735 wait complete. 2024-04-26T20:07:37.826 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T20:07:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 -- 172.21.15.131:0/2264551430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98075390 msgr2=0x7fbb98075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/2264551430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98075390 0x7fbb98075810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009930 tx=0x7fbb8402a420 comp rx=0 tx=0).stop 2024-04-26T20:07:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 -- 172.21.15.131:0/2264551430 shutdown_connections 2024-04-26T20:07:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/2264551430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9810ee90 0x7fbb98111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/2264551430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98075390 0x7fbb98075810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/2264551430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb98076970 0x7fbb98074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 -- 172.21.15.131:0/2264551430 >> 172.21.15.131:0/2264551430 conn(0x7fbb980fde30 msgr2=0x7fbb98100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:38.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 -- 172.21.15.131:0/2264551430 shutdown_connections 2024-04-26T20:07:38.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.008+0000 7fbb9ee00640 1 -- 172.21.15.131:0/2264551430 wait complete. 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.009+0000 7fbb9ee00640 1 Processor -- start 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.009+0000 7fbb9ee00640 1 -- start start 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb98075390 0x7fbb9819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98076970 0x7fbb9819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9ee00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9810ee90 0x7fbb981a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9ee00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981149a0 con 0x7fbb9810ee90 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98114820 con 0x7fbb98076970 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98114b20 con 0x7fbb98075390 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98076970 0x7fbb9819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98076970 0x7fbb9819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36484/0 (socket says 172.21.15.131:36484) 2024-04-26T20:07:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 -- 172.21.15.131:0/1295224655 learned_addr learned my addr 172.21.15.131:0/1295224655 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9da00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb98075390 0x7fbb9819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9e400640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9810ee90 0x7fbb981a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 -- 172.21.15.131:0/1295224655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb98075390 msgr2=0x7fbb9819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb98075390 0x7fbb9819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 -- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9810ee90 msgr2=0x7fbb981a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9810ee90 0x7fbb981a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9d000640 1 -- 172.21.15.131:0/1295224655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb981a6cf0 con 0x7fbb98076970 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.010+0000 7fbb9da00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb98075390 0x7fbb9819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.011+0000 7fbb9d000640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98076970 0x7fbb9819f610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb84031500 tx=0x7fbb84003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.011+0000 7fbb96000640 1 -- 172.21.15.131:0/1295224655 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8403f070 con 0x7fbb98076970 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.011+0000 7fbb96000640 1 -- 172.21.15.131:0/1295224655 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb84003e70 con 0x7fbb98076970 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.011+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb981a6f80 con 0x7fbb98076970 2024-04-26T20:07:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.011+0000 7fbb96000640 1 -- 172.21.15.131:0/1295224655 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84035030 con 0x7fbb98076970 2024-04-26T20:07:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.011+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb981a73e0 con 0x7fbb98076970 2024-04-26T20:07:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.012+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb98105060 con 0x7fbb98076970 2024-04-26T20:07:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.014+0000 7fbb96000640 1 -- 172.21.15.131:0/1295224655 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbb84042070 con 0x7fbb98076970 2024-04-26T20:07:38.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.015+0000 7fbb96000640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbb800a3bb0 0x7fbb800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:38.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.016+0000 7fbb96000640 1 -- 172.21.15.131:0/1295224655 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fbb840f1380 con 0x7fbb98076970 2024-04-26T20:07:38.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.016+0000 7fbb9da00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbb800a3bb0 0x7fbb800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:38.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.016+0000 7fbb9da00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbb800a3bb0 0x7fbb800a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0058c0 tx=0x7fbb8c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.019+0000 7fbb96000640 1 -- 172.21.15.131:0/1295224655 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbb840bdae0 con 0x7fbb98076970 2024-04-26T20:07:38.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:37 smithi148 ceph-mon[32281]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T20:07:38.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:37 smithi148 ceph-mon[32281]: pgmap v330: 97 pgs: 8 creating+peering, 48 active+clean, 41 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-04-26T20:07:38.067 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:07:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b[63000]: 2024-04-26T20:07:37.666+0000 7f71fa892700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:07:38.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:37 smithi131 ceph-mon[27919]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T20:07:38.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:37 smithi131 ceph-mon[27919]: pgmap v330: 97 pgs: 8 creating+peering, 48 active+clean, 41 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-04-26T20:07:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.226+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fbb980635a0 con 0x7fbb98076970 2024-04-26T20:07:38.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:37 smithi157 ceph-mon[32703]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T20:07:38.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:37 smithi157 ceph-mon[32703]: pgmap v330: 97 pgs: 8 creating+peering, 48 active+clean, 41 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-04-26T20:07:38.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.797+0000 7fbb96000640 1 -- 172.21.15.131:0/1295224655 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v90) ==== 169+0+0 (secure 0 0 0) 0x7fbb840bd3e0 con 0x7fbb98076970 2024-04-26T20:07:38.800 INFO:teuthology.orchestra.run.smithi131.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T20:07:38.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.804+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbb800a3bb0 msgr2=0x7fbb800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:38.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.804+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbb800a3bb0 0x7fbb800a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0058c0 tx=0x7fbb8c005850 comp rx=0 tx=0).stop 2024-04-26T20:07:38.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.804+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98076970 msgr2=0x7fbb9819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:38.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.804+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98076970 0x7fbb9819f610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb84031500 tx=0x7fbb84003c80 comp rx=0 tx=0).stop 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 shutdown_connections 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9810ee90 0x7fbb981a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb98076970 0x7fbb9819f610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbb800a3bb0 0x7fbb800a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 --2- 172.21.15.131:0/1295224655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb98075390 0x7fbb9819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 >> 172.21.15.131:0/1295224655 conn(0x7fbb980fde30 msgr2=0x7fbb98102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 shutdown_connections 2024-04-26T20:07:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:38.807+0000 7fbb9ee00640 1 -- 172.21.15.131:0/1295224655 wait complete. 2024-04-26T20:07:38.839 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T20:07:39.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 -- 172.21.15.131:0/3393199036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101280 msgr2=0x7f16401016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:39.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 --2- 172.21.15.131:0/3393199036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101280 0x7f16401016a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1630009900 tx=0x7f163002a480 comp rx=0 tx=0).stop 2024-04-26T20:07:39.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 -- 172.21.15.131:0/3393199036 shutdown_connections 2024-04-26T20:07:39.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 --2- 172.21.15.131:0/3393199036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1640102580 0x7f1640102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 --2- 172.21.15.131:0/3393199036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101be0 0x7f1640102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 --2- 172.21.15.131:0/3393199036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101280 0x7f16401016a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 -- 172.21.15.131:0/3393199036 >> 172.21.15.131:0/3393199036 conn(0x7f16400fce20 msgr2=0x7f16400ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:39.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 -- 172.21.15.131:0/3393199036 shutdown_connections 2024-04-26T20:07:39.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.005+0000 7f1644a00640 1 -- 172.21.15.131:0/3393199036 wait complete. 2024-04-26T20:07:39.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.006+0000 7f1644a00640 1 Processor -- start 2024-04-26T20:07:39.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.006+0000 7f1644a00640 1 -- start start 2024-04-26T20:07:39.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f1644a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101280 0x7f164019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:39.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f1644a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101be0 0x7f164019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:39.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f1644a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1640102580 0x7f16401a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:39.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f1644a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16401106c0 con 0x7f1640101be0 2024-04-26T20:07:39.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f1644a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1640110540 con 0x7f1640102580 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f1644a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1640110840 con 0x7f1640101280 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101280 0x7f164019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101280 0x7f164019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56274/0 (socket says 172.21.15.131:56274) 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 -- 172.21.15.131:0/623160328 learned_addr learned my addr 172.21.15.131:0/623160328 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f1637e00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101be0 0x7f164019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163e800640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1640102580 0x7f16401a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 -- 172.21.15.131:0/623160328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1640102580 msgr2=0x7f16401a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1640102580 0x7f16401a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 -- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101be0 msgr2=0x7f164019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101be0 0x7f164019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163de00640 1 -- 172.21.15.131:0/623160328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16401a2a10 con 0x7f1640101280 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.007+0000 7f163e800640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1640102580 0x7f16401a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.008+0000 7f163de00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101280 0x7f164019adf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1630005940 tx=0x7f16300042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.008+0000 7f1636a00640 1 -- 172.21.15.131:0/623160328 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163003f030 con 0x7f1640101280 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.008+0000 7f1636a00640 1 -- 172.21.15.131:0/623160328 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1630003cb0 con 0x7f1640101280 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.008+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16401a2ca0 con 0x7f1640101280 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.008+0000 7f1636a00640 1 -- 172.21.15.131:0/623160328 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630035070 con 0x7f1640101280 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.008+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16401a3190 con 0x7f1640101280 2024-04-26T20:07:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.009+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164010bfd0 con 0x7f1640101280 2024-04-26T20:07:39.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.011+0000 7f1636a00640 1 -- 172.21.15.131:0/623160328 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1630042030 con 0x7f1640101280 2024-04-26T20:07:39.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.012+0000 7f1636a00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:39.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.012+0000 7f1636a00640 1 -- 172.21.15.131:0/623160328 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8770+0+0 (secure 0 0 0) 0x7f16300f0ae0 con 0x7f1640101280 2024-04-26T20:07:39.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.012+0000 7f1637e00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:39.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.013+0000 7f1637e00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f16240a3bb0 0x7f16240a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f164019c300 tx=0x7f1628005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.017+0000 7f1636a00640 1 -- 172.21.15.131:0/623160328 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f16300bd240 con 0x7f1640101280 2024-04-26T20:07:39.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:38 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T20:07:39.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:38 smithi148 ceph-mon[32281]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T20:07:39.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:38 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1295224655' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T20:07:39.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:38 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T20:07:39.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:38 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T20:07:39.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:38 smithi157 ceph-mon[32703]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T20:07:39.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:38 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1295224655' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T20:07:39.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:38 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T20:07:39.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.248+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f1640063560 con 0x7f1640101280 2024-04-26T20:07:39.255 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:38 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T20:07:39.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:38 smithi131 ceph-mon[27919]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T20:07:39.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:38 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1295224655' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T20:07:39.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:38 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T20:07:39.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.829+0000 7f1636a00640 1 -- 172.21.15.131:0/623160328 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f1630003e60 con 0x7f1640101280 2024-04-26T20:07:39.831 INFO:teuthology.orchestra.run.smithi131.stderr:added data pool 4 to fsmap 2024-04-26T20:07:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f16240a3bb0 msgr2=0x7f16240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f16240a3bb0 0x7f16240a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f164019c300 tx=0x7f1628005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101280 msgr2=0x7f164019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101280 0x7f164019adf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1630005940 tx=0x7f16300042f0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 shutdown_connections 2024-04-26T20:07:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1640102580 0x7f16401a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1640101be0 0x7f164019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 --2- 172.21.15.131:0/623160328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1640101280 0x7f164019adf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.833+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 >> 172.21.15.131:0/623160328 conn(0x7f16400fce20 msgr2=0x7f1640100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.834+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 shutdown_connections 2024-04-26T20:07:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:39.834+0000 7f1644a00640 1 -- 172.21.15.131:0/623160328 wait complete. 2024-04-26T20:07:39.859 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.046+0000 7f7e9b200640 1 -- 172.21.15.131:0/1335724766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c1031f0 msgr2=0x7f7e9c1055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.046+0000 7f7e9b200640 1 --2- 172.21.15.131:0/1335724766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c1031f0 0x7f7e9c1055e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009900 tx=0x7f7e9402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.047+0000 7f7e9b200640 1 -- 172.21.15.131:0/1335724766 shutdown_connections 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.047+0000 7f7e9b200640 1 --2- 172.21.15.131:0/1335724766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9c108450 0x7f7e9c10a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.047+0000 7f7e9b200640 1 --2- 172.21.15.131:0/1335724766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e9c105b20 0x7f7e9c107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.047+0000 7f7e9b200640 1 --2- 172.21.15.131:0/1335724766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c1031f0 0x7f7e9c1055e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.047+0000 7f7e9b200640 1 -- 172.21.15.131:0/1335724766 >> 172.21.15.131:0/1335724766 conn(0x7f7e9c0fcbc0 msgr2=0x7f7e9c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.047+0000 7f7e9b200640 1 -- 172.21.15.131:0/1335724766 shutdown_connections 2024-04-26T20:07:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.047+0000 7f7e9b200640 1 -- 172.21.15.131:0/1335724766 wait complete. 2024-04-26T20:07:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 Processor -- start 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 -- start start 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e9c1031f0 0x7f7e9c102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9c105b20 0x7f7e9c1011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c108450 0x7f7e9c1016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c10e2f0 con 0x7f7e9c108450 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c10e170 con 0x7f7e9c1031f0 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c10e470 con 0x7f7e9c105b20 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c108450 0x7f7e9c1016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c108450 0x7f7e9c1016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36118/0 (socket says 172.21.15.131:36118) 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e9a800640 1 -- 172.21.15.131:0/3655847365 learned_addr learned my addr 172.21.15.131:0/3655847365 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e99400640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9c105b20 0x7f7e9c1011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.048+0000 7f7e99e00640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e9c1031f0 0x7f7e9c102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9a800640 1 -- 172.21.15.131:0/3655847365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9c105b20 msgr2=0x7f7e9c1011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9a800640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9c105b20 0x7f7e9c1011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9a800640 1 -- 172.21.15.131:0/3655847365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e9c1031f0 msgr2=0x7f7e9c102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9a800640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e9c1031f0 0x7f7e9c102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9a800640 1 -- 172.21.15.131:0/3655847365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c101f70 con 0x7f7e9c108450 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9a800640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c108450 0x7f7e9c1016e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d950 tx=0x7f7e8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e92a00640 1 -- 172.21.15.131:0/3655847365 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c017030 con 0x7f7e9c108450 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e92a00640 1 -- 172.21.15.131:0/3655847365 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c004020 con 0x7f7e9c108450 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e92a00640 1 -- 172.21.15.131:0/3655847365 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c003b50 con 0x7f7e9c108450 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c1a2ac0 con 0x7f7e9c108450 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.049+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1a2fc0 con 0x7f7e9c108450 2024-04-26T20:07:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.050+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c0fe6b0 con 0x7f7e9c108450 2024-04-26T20:07:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.052+0000 7f7e92a00640 1 -- 172.21.15.131:0/3655847365 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7e8c01a070 con 0x7f7e9c108450 2024-04-26T20:07:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.054+0000 7f7e92a00640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7e800a3bb0 0x7f7e800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.054+0000 7f7e92a00640 1 -- 172.21.15.131:0/3655847365 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f7e8c0d0180 con 0x7f7e9c108450 2024-04-26T20:07:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.054+0000 7f7e99e00640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7e800a3bb0 0x7f7e800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.055+0000 7f7e99e00640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7e800a3bb0 0x7f7e800a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7e94031500 tx=0x7f7e94005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.058+0000 7f7e92a00640 1 -- 172.21.15.131:0/3655847365 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e8c021070 con 0x7f7e9c108450 2024-04-26T20:07:40.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:39 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T20:07:40.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:39 smithi148 ceph-mon[32281]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T20:07:40.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:39 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/623160328' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T20:07:40.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:39 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T20:07:40.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:39 smithi148 ceph-mon[32281]: pgmap v333: 129 pgs: 4 creating+peering, 99 active+clean, 26 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-26T20:07:40.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:39 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T20:07:40.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:39 smithi157 ceph-mon[32703]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T20:07:40.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:39 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/623160328' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T20:07:40.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:39 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T20:07:40.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:39 smithi157 ceph-mon[32703]: pgmap v333: 129 pgs: 4 creating+peering, 99 active+clean, 26 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-26T20:07:40.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:39 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T20:07:40.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:39 smithi131 ceph-mon[27919]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T20:07:40.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:39 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/623160328' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T20:07:40.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:39 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T20:07:40.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:39 smithi131 ceph-mon[27919]: pgmap v333: 129 pgs: 4 creating+peering, 99 active+clean, 26 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-26T20:07:40.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.286+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e9c02dad0 con 0x7f7e9c108450 2024-04-26T20:07:40.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.288+0000 7f7e92a00640 1 -- 172.21.15.131:0/3655847365 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11824 (secure 0 0 0) 0x7f7e8c09c8c0 con 0x7f7e9c108450 2024-04-26T20:07:40.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:40.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:07:39.250942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24575,"qdb_cluster":[24575]},"id":1}]} 2024-04-26T20:07:40.289 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-26T20:07:40.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7e800a3bb0 msgr2=0x7f7e800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7e800a3bb0 0x7f7e800a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7e94031500 tx=0x7f7e94005a20 comp rx=0 tx=0).stop 2024-04-26T20:07:40.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c108450 msgr2=0x7f7e9c1016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c108450 0x7f7e9c1016e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d950 tx=0x7f7e8c00de10 comp rx=0 tx=0).stop 2024-04-26T20:07:40.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 shutdown_connections 2024-04-26T20:07:40.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e9c108450 0x7f7e9c1016e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7e800a3bb0 0x7f7e800a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9c105b20 0x7f7e9c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 --2- 172.21.15.131:0/3655847365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e9c1031f0 0x7f7e9c102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.290+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 >> 172.21.15.131:0/3655847365 conn(0x7f7e9c0fcbc0 msgr2=0x7f7e9c103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.291+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 shutdown_connections 2024-04-26T20:07:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.291+0000 7f7e9b200640 1 -- 172.21.15.131:0/3655847365 wait complete. 2024-04-26T20:07:40.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:07:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 -- 172.21.15.131:0/955202920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8108640 msgr2=0x7f05a810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 --2- 172.21.15.131:0/955202920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8108640 0x7f05a810aa90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009be0 tx=0x7f05a002a6d0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 -- 172.21.15.131:0/955202920 shutdown_connections 2024-04-26T20:07:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 --2- 172.21.15.131:0/955202920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8108640 0x7f05a810aa90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 --2- 172.21.15.131:0/955202920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a8105d10 0x7f05a8108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 --2- 172.21.15.131:0/955202920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 0x7f05a81057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 -- 172.21.15.131:0/955202920 >> 172.21.15.131:0/955202920 conn(0x7f05a80fcdf0 msgr2=0x7f05a80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 -- 172.21.15.131:0/955202920 shutdown_connections 2024-04-26T20:07:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.504+0000 7f05afc00640 1 -- 172.21.15.131:0/955202920 wait complete. 2024-04-26T20:07:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.505+0000 7f05afc00640 1 Processor -- start 2024-04-26T20:07:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.505+0000 7f05afc00640 1 -- start start 2024-04-26T20:07:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05afc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 0x7f05a819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05afc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8105d10 0x7f05a819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05afc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a8108640 0x7f05a81a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05afc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05a810e4b0 con 0x7f05a81033e0 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05afc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05a810e330 con 0x7f05a8108640 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05afc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05a810e630 con 0x7f05a8105d10 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 0x7f05a819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 0x7f05a819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36140/0 (socket says 172.21.15.131:36140) 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 -- 172.21.15.131:0/3845934502 learned_addr learned my addr 172.21.15.131:0/3845934502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05adc00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a8108640 0x7f05a81a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05a7e00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8105d10 0x7f05a819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 -- 172.21.15.131:0/3845934502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8105d10 msgr2=0x7f05a819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8105d10 0x7f05a819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 -- 172.21.15.131:0/3845934502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a8108640 msgr2=0x7f05a81a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a8108640 0x7f05a81a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.506+0000 7f05ad200640 1 -- 172.21.15.131:0/3845934502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a81a2980 con 0x7f05a81033e0 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.507+0000 7f05ad200640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 0x7f05a819ad60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9c0 tx=0x7f059800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.507+0000 7f05a5600640 1 -- 172.21.15.131:0/3845934502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059800ccc0 con 0x7f05a81033e0 2024-04-26T20:07:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.507+0000 7f05a5600640 1 -- 172.21.15.131:0/3845934502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05980054d0 con 0x7f05a81033e0 2024-04-26T20:07:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.507+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a81a2c70 con 0x7f05a81033e0 2024-04-26T20:07:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.507+0000 7f05a5600640 1 -- 172.21.15.131:0/3845934502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598010430 con 0x7f05a81033e0 2024-04-26T20:07:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.507+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81a31c0 con 0x7f05a81033e0 2024-04-26T20:07:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.508+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a810a130 con 0x7f05a81033e0 2024-04-26T20:07:40.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.510+0000 7f05a5600640 1 -- 172.21.15.131:0/3845934502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0598004f80 con 0x7f05a81033e0 2024-04-26T20:07:40.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.512+0000 7f05a5600640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f058c0a3bb0 0x7f058c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.512+0000 7f05a5600640 1 -- 172.21.15.131:0/3845934502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f05980d0e00 con 0x7f05a81033e0 2024-04-26T20:07:40.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.512+0000 7f05a7e00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f058c0a3bb0 0x7f058c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.513+0000 7f05a7e00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f058c0a3bb0 0x7f058c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05a819c270 tx=0x7f0590005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.516+0000 7f05a5600640 1 -- 172.21.15.131:0/3845934502 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f059809d540 con 0x7f05a81033e0 2024-04-26T20:07:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.719+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05a802dca0 con 0x7f05a81033e0 2024-04-26T20:07:40.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.722+0000 7f05a5600640 1 -- 172.21.15.131:0/3845934502 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24200 (secure 0 0 0) 0x7f059809ce40 con 0x7f05a81033e0 2024-04-26T20:07:40.723 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:40.724 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":91,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:07:39.250667+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:07:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.725+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f058c0a3bb0 msgr2=0x7f058c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.725+0000 7f05afc00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f058c0a3bb0 0x7f058c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05a819c270 tx=0x7f0590005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.725+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 msgr2=0x7f05a819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.725+0000 7f05afc00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 0x7f05a819ad60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9c0 tx=0x7f059800ee80 comp rx=0 tx=0).stop 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 shutdown_connections 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a8108640 0x7f05a81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f058c0a3bb0 0x7f058c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05a8105d10 0x7f05a819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 --2- 172.21.15.131:0/3845934502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05a81033e0 0x7f05a819ad60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 >> 172.21.15.131:0/3845934502 conn(0x7f05a80fcdf0 msgr2=0x7f05a8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 shutdown_connections 2024-04-26T20:07:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.726+0000 7f05afc00640 1 -- 172.21.15.131:0/3845934502 wait complete. 2024-04-26T20:07:40.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 -- 172.21.15.131:0/3033257395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 msgr2=0x7f6b281093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 --2- 172.21.15.131:0/3033257395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 0x7f6b281093b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009ca0 tx=0x7f6b2002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 -- 172.21.15.131:0/3033257395 shutdown_connections 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 --2- 172.21.15.131:0/3033257395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 0x7f6b281093b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 --2- 172.21.15.131:0/3033257395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 0x7f6b28102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 --2- 172.21.15.131:0/3033257395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b28068f30 0x7f6b28101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 -- 172.21.15.131:0/3033257395 >> 172.21.15.131:0/3033257395 conn(0x7f6b280fc120 msgr2=0x7f6b280fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 -- 172.21.15.131:0/3033257395 shutdown_connections 2024-04-26T20:07:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.939+0000 7f6b2d000640 1 -- 172.21.15.131:0/3033257395 wait complete. 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 Processor -- start 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 -- start start 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b28068f30 0x7f6b2819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 0x7f6b2819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 0x7f6b281a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2810e4d0 con 0x7f6b281022c0 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2810e350 con 0x7f6b28106fc0 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b2d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2810e650 con 0x7f6b28068f30 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.940+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 0x7f6b2819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 0x7f6b2819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36166/0 (socket says 172.21.15.131:36166) 2024-04-26T20:07:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 -- 172.21.15.131:0/2013875009 learned_addr learned my addr 172.21.15.131:0/2013875009 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b26800640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 0x7f6b281a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b25e00640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b28068f30 0x7f6b2819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 -- 172.21.15.131:0/2013875009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b28068f30 msgr2=0x7f6b2819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b28068f30 0x7f6b2819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 -- 172.21.15.131:0/2013875009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 msgr2=0x7f6b281a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 0x7f6b281a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 -- 172.21.15.131:0/2013875009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b281a2900 con 0x7f6b281022c0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1fe00640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 0x7f6b2819b200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000e9c0 tx=0x7f6b1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.941+0000 7f6b1ea00640 1 -- 172.21.15.131:0/2013875009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000cde0 con 0x7f6b281022c0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.942+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b281a2bf0 con 0x7f6b281022c0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.942+0000 7f6b1ea00640 1 -- 172.21.15.131:0/2013875009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b100108e0 con 0x7f6b281022c0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.942+0000 7f6b1ea00640 1 -- 172.21.15.131:0/2013875009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b100188d0 con 0x7f6b281022c0 2024-04-26T20:07:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.942+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a3190 con 0x7f6b281022c0 2024-04-26T20:07:40.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.943+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b281a3360 con 0x7f6b281022c0 2024-04-26T20:07:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.945+0000 7f6b1ea00640 1 -- 172.21.15.131:0/2013875009 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6b1001f020 con 0x7f6b281022c0 2024-04-26T20:07:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.946+0000 7f6b1ea00640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.947+0000 7f6b1ea00640 1 -- 172.21.15.131:0/2013875009 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f6b100d0810 con 0x7f6b281022c0 2024-04-26T20:07:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.947+0000 7f6b25e00640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.947+0000 7f6b25e00640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b2819bb90 tx=0x7f6b18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:40.951+0000 7f6b1ea00640 1 -- 172.21.15.131:0/2013875009 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6b1009cf50 con 0x7f6b281022c0 2024-04-26T20:07:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:40 smithi131 ceph-mon[27919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T20:07:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:40 smithi131 ceph-mon[27919]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T20:07:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:40 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T20:07:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:40 smithi131 ceph-mon[27919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:41.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:40 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3655847365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:41.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:40 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3845934502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:41.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.156+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b281a36f0 con 0x7f6b281022c0 2024-04-26T20:07:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.158+0000 7f6b1ea00640 1 -- 172.21.15.131:0/2013875009 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24187 (secure 0 0 0) 0x7f6b1009c850 con 0x7f6b281022c0 2024-04-26T20:07:41.159 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:41.159 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":92,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:07:40.812449+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6b0c0a3bb0 msgr2=0x7f6b0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b2819bb90 tx=0x7f6b18005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 msgr2=0x7f6b2819b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 0x7f6b2819b200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000e9c0 tx=0x7f6b1000ee80 comp rx=0 tx=0).stop 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 shutdown_connections 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28106fc0 0x7f6b281a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b281022c0 0x7f6b2819b200 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 --2- 172.21.15.131:0/2013875009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b28068f30 0x7f6b2819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 >> 172.21.15.131:0/2013875009 conn(0x7f6b280fc120 msgr2=0x7f6b280fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 shutdown_connections 2024-04-26T20:07:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.161+0000 7f6b2d000640 1 -- 172.21.15.131:0/2013875009 wait complete. 2024-04-26T20:07:41.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:07:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:40 smithi157 ceph-mon[32703]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T20:07:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:40 smithi157 ceph-mon[32703]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T20:07:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:40 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T20:07:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:40 smithi157 ceph-mon[32703]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:40 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3655847365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:41.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:40 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3845934502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:41.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:40 smithi148 ceph-mon[32281]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T20:07:41.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:40 smithi148 ceph-mon[32281]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T20:07:41.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:40 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T20:07:41.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:40 smithi148 ceph-mon[32281]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:07:41.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:40 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3655847365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:41.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:40 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3845934502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 -- 172.21.15.131:0/471165004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 msgr2=0x7f8ffc1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/471165004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc1091b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009900 tx=0x7f8fec02a410 comp rx=0 tx=0).stop 2024-04-26T20:07:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 -- 172.21.15.131:0/471165004 shutdown_connections 2024-04-26T20:07:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/471165004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ffc10a940 0x7f8ffc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/471165004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ffc109f80 0x7f8ffc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/471165004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc1091b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 -- 172.21.15.131:0/471165004 >> 172.21.15.131:0/471165004 conn(0x7f8ffc077d30 msgr2=0x7f8ffc07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 -- 172.21.15.131:0/471165004 shutdown_connections 2024-04-26T20:07:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.396+0000 7f8ffbe00640 1 -- 172.21.15.131:0/471165004 wait complete. 2024-04-26T20:07:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 Processor -- start 2024-04-26T20:07:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 -- start start 2024-04-26T20:07:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ffc109f80 0x7f8ffc19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ffc10a940 0x7f8ffc1a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc114900 con 0x7f8ffc108d90 2024-04-26T20:07:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc114780 con 0x7f8ffc109f80 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.397+0000 7f8ffbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc114a80 con 0x7f8ffc10a940 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ff2000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ffc109f80 0x7f8ffc19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36190/0 (socket says 172.21.15.131:36190) 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 -- 172.21.15.131:0/2310627150 learned_addr learned my addr 172.21.15.131:0/2310627150 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffb400640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ffc10a940 0x7f8ffc1a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 -- 172.21.15.131:0/2310627150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ffc10a940 msgr2=0x7f8ffc1a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ffc10a940 0x7f8ffc1a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 -- 172.21.15.131:0/2310627150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ffc109f80 msgr2=0x7f8ffc19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ffc109f80 0x7f8ffc19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.398+0000 7f8ffaa00640 1 -- 172.21.15.131:0/2310627150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc1a6c20 con 0x7f8ffc108d90 2024-04-26T20:07:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.399+0000 7f8ffaa00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc19f000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8fec031500 tx=0x7f8fec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.399+0000 7f8ff3e00640 1 -- 172.21.15.131:0/2310627150 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec03f070 con 0x7f8ffc108d90 2024-04-26T20:07:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.399+0000 7f8ff3e00640 1 -- 172.21.15.131:0/2310627150 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fec031c50 con 0x7f8ffc108d90 2024-04-26T20:07:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.399+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc1a6eb0 con 0x7f8ffc108d90 2024-04-26T20:07:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.399+0000 7f8ff3e00640 1 -- 172.21.15.131:0/2310627150 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec035030 con 0x7f8ffc108d90 2024-04-26T20:07:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.399+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1a7310 con 0x7f8ffc108d90 2024-04-26T20:07:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.400+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ffc110180 con 0x7f8ffc108d90 2024-04-26T20:07:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.402+0000 7f8ff3e00640 1 -- 172.21.15.131:0/2310627150 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8fec042070 con 0x7f8ffc108d90 2024-04-26T20:07:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.403+0000 7f8ff3e00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8fe00a3c30 0x7f8fe00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.404+0000 7f8ff3e00640 1 -- 172.21.15.131:0/2310627150 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f8fec0f14d0 con 0x7f8ffc108d90 2024-04-26T20:07:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.404+0000 7f8ff2000640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8fe00a3c30 0x7f8fe00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.404+0000 7f8ff2000640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8fe00a3c30 0x7f8fe00a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc1a0510 tx=0x7f8fe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.406+0000 7f8ff3e00640 1 -- 172.21.15.131:0/2310627150 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8fec0bdc10 con 0x7f8ffc108d90 2024-04-26T20:07:41.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.644+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ffc02dbf0 con 0x7f8ffc108d90 2024-04-26T20:07:41.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.646+0000 7f8ff3e00640 1 -- 172.21.15.131:0/2310627150 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11824 (secure 0 0 0) 0x7f8fec0bd510 con 0x7f8ffc108d90 2024-04-26T20:07:41.648 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:41.649 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:07:39.250942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24575,"qdb_cluster":[24575]},"id":1}]} 2024-04-26T20:07:41.649 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-26T20:07:41.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8fe00a3c30 msgr2=0x7f8fe00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8fe00a3c30 0x7f8fe00a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc1a0510 tx=0x7f8fe4005f90 comp rx=0 tx=0).stop 2024-04-26T20:07:41.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 msgr2=0x7f8ffc19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc19f000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8fec031500 tx=0x7f8fec003c10 comp rx=0 tx=0).stop 2024-04-26T20:07:41.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 shutdown_connections 2024-04-26T20:07:41.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8fe00a3c30 0x7f8fe00a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ffc10a940 0x7f8ffc1a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ffc109f80 0x7f8ffc19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 --2- 172.21.15.131:0/2310627150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ffc108d90 0x7f8ffc19f000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 >> 172.21.15.131:0/2310627150 conn(0x7f8ffc077d30 msgr2=0x7f8ffc10d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:41.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.648+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 shutdown_connections 2024-04-26T20:07:41.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.649+0000 7f8ffbe00640 1 -- 172.21.15.131:0/2310627150 wait complete. 2024-04-26T20:07:41.667 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 -- 172.21.15.131:0/1016327909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc1031f0 msgr2=0x7fc4bc1055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/1016327909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc1055e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009900 tx=0x7fc4b402a410 comp rx=0 tx=0).stop 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 -- 172.21.15.131:0/1016327909 shutdown_connections 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/1016327909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc108450 0x7fc4bc10a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/1016327909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4bc105b20 0x7fc4bc107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/1016327909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc1055e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 -- 172.21.15.131:0/1016327909 >> 172.21.15.131:0/1016327909 conn(0x7fc4bc0fcbc0 msgr2=0x7fc4bc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 -- 172.21.15.131:0/1016327909 shutdown_connections 2024-04-26T20:07:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.859+0000 7fc4c1c00640 1 -- 172.21.15.131:0/1016327909 wait complete. 2024-04-26T20:07:41.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 Processor -- start 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 -- start start 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc105b20 0x7fc4bc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4bc108450 0x7fc4bc19ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc10e2f0 con 0x7fc4bc105b20 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc10e170 con 0x7fc4bc1031f0 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4c1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc10e470 con 0x7fc4bc108450 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.860+0000 7fc4bbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36562/0 (socket says 172.21.15.131:36562) 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 -- 172.21.15.131:0/2915687523 learned_addr learned my addr 172.21.15.131:0/2915687523 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bb400640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc105b20 0x7fc4bc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4c1200640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4bc108450 0x7fc4bc19ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 -- 172.21.15.131:0/2915687523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4bc108450 msgr2=0x7fc4bc19ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4bc108450 0x7fc4bc19ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 -- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc105b20 msgr2=0x7fc4bc198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc105b20 0x7fc4bc198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 -- 172.21.15.131:0/2915687523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc1a05b0 con 0x7fc4bc1031f0 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4bbe00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc198990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4005ab0 tx=0x7fc4b4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.861+0000 7fc4b8c00640 1 -- 172.21.15.131:0/2915687523 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b403f070 con 0x7fc4bc1031f0 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.862+0000 7fc4b8c00640 1 -- 172.21.15.131:0/2915687523 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b4031c50 con 0x7fc4bc1031f0 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.862+0000 7fc4b8c00640 1 -- 172.21.15.131:0/2915687523 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4035030 con 0x7fc4bc1031f0 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.862+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4bc1a0840 con 0x7fc4bc1031f0 2024-04-26T20:07:41.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.862+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4bc1a0d80 con 0x7fc4bc1031f0 2024-04-26T20:07:41.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.863+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4bc0fe6b0 con 0x7fc4bc1031f0 2024-04-26T20:07:41.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.865+0000 7fc4b8c00640 1 -- 172.21.15.131:0/2915687523 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc4b4042070 con 0x7fc4bc1031f0 2024-04-26T20:07:41.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.866+0000 7fc4b8c00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:41.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.866+0000 7fc4b8c00640 1 -- 172.21.15.131:0/2915687523 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fc4b40f1520 con 0x7fc4bc1031f0 2024-04-26T20:07:41.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.866+0000 7fc4bb400640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:41.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.867+0000 7fc4bb400640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40099a0 tx=0x7fc4a4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:41.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:41.869+0000 7fc4b8c00640 1 -- 172.21.15.131:0/2915687523 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc4b40bdc60 con 0x7fc4bc1031f0 2024-04-26T20:07:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:41 smithi131 ceph-mon[27919]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T20:07:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:41 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2013875009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:41 smithi131 ceph-mon[27919]: pgmap v336: 129 pgs: 4 creating+peering, 107 active+clean, 18 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-26T20:07:42.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:41 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2310627150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.058+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4bc02dca0 con 0x7fc4bc1031f0 2024-04-26T20:07:42.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.060+0000 7fc4b8c00640 1 -- 172.21.15.131:0/2915687523 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24187 (secure 0 0 0) 0x7fc4b40bd560 con 0x7fc4bc1031f0 2024-04-26T20:07:42.062 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:42.062 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":92,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:07:40.812449+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4a00a3c30 msgr2=0x7fc4a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40099a0 tx=0x7fc4a4009500 comp rx=0 tx=0).stop 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc1031f0 msgr2=0x7fc4bc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc198990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4005ab0 tx=0x7fc4b4003b90 comp rx=0 tx=0).stop 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 shutdown_connections 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4bc108450 0x7fc4bc19ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4bc105b20 0x7fc4bc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 --2- 172.21.15.131:0/2915687523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc1031f0 0x7fc4bc198990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 >> 172.21.15.131:0/2915687523 conn(0x7fc4bc0fcbc0 msgr2=0x7fc4bc103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 shutdown_connections 2024-04-26T20:07:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.063+0000 7fc4c1c00640 1 -- 172.21.15.131:0/2915687523 wait complete. 2024-04-26T20:07:42.083 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:07:34.194239+0000', 'modified': '2024-04-26T20:07:39.250942+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6839/4047119723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 4047119723}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 4047119723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24575, 'qdb_cluster': [24575]} 2024-04-26T20:07:42.083 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:07:42.084 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T20:07:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:41 smithi157 ceph-mon[32703]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T20:07:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:41 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2013875009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:41 smithi157 ceph-mon[32703]: pgmap v336: 129 pgs: 4 creating+peering, 107 active+clean, 18 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-26T20:07:42.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:41 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2310627150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2293090552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68104310 msgr2=0x7f1d6810aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2293090552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68104310 0x7f1d6810aba0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009900 tx=0x7f1d6002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2293090552 shutdown_connections 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2293090552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68104310 0x7f1d6810aba0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2293090552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d68103950 0x7f1d68103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2293090552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 0x7f1d68102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2293090552 >> 172.21.15.131:0/2293090552 conn(0x7f1d680fdcb0 msgr2=0x7f1d68100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2293090552 shutdown_connections 2024-04-26T20:07:42.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.282+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2293090552 wait complete. 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 Processor -- start 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 -- start start 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 0x7f1d68198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68103950 0x7f1d68198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d68104310 0x7f1d6819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810e2e0 con 0x7f1d68103950 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810e160 con 0x7f1d68102760 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.283+0000 7f1d6ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810e460 con 0x7f1d68104310 2024-04-26T20:07:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68103950 0x7f1d68198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 0x7f1d68198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 0x7f1d68198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36586/0 (socket says 172.21.15.131:36586) 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68103950 0x7f1d68198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36234/0 (socket says 172.21.15.131:36234) 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 -- 172.21.15.131:0/2264925355 learned_addr learned my addr 172.21.15.131:0/2264925355 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6e000640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d68104310 0x7f1d6819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 -- 172.21.15.131:0/2264925355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d68104310 msgr2=0x7f1d6819fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d68104310 0x7f1d6819fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 -- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68103950 msgr2=0x7f1d68198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68103950 0x7f1d68198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 -- 172.21.15.131:0/2264925355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d681a0540 con 0x7f1d68102760 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.284+0000 7f1d6d600640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 0x7f1d68198920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9c0 tx=0x7f1d5800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.285+0000 7f1d66000640 1 -- 172.21.15.131:0/2264925355 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800cde0 con 0x7f1d68102760 2024-04-26T20:07:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.285+0000 7f1d66000640 1 -- 172.21.15.131:0/2264925355 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d580108e0 con 0x7f1d68102760 2024-04-26T20:07:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.285+0000 7f1d66000640 1 -- 172.21.15.131:0/2264925355 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58018960 con 0x7f1d68102760 2024-04-26T20:07:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.285+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d681a0830 con 0x7f1d68102760 2024-04-26T20:07:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.285+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d681a0d30 con 0x7f1d68102760 2024-04-26T20:07:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 Processor -- start 2024-04-26T20:07:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 -- start start 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d66000640 1 -- 172.21.15.131:0/2264925355 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1d5801f020 con 0x7f1d68102760 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce7e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55570cde2600 con 0x55570ce83e20 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55570cd7f100 con 0x55570ce7e2c0 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.288+0000 7f1d72de8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55570cdde5c0 con 0x55570ce7ecb0 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce862b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36248/0 (socket says 172.21.15.131:36248) 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3257178317 learned_addr learned my addr 172.21.15.131:0/3257178317 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce7e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6d600640 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3257178317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 msgr2=0x55570ce837b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce837b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3257178317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 msgr2=0x55570ce7e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce7e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3257178317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570caf42d0 con 0x55570ce83e20 2024-04-26T20:07:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6d600640 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:07:42.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.289+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce862b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d54011470 tx=0x7f1d54011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7158ce3c35360dda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:42.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d5e600640 1 -- 172.21.15.131:0/3257178317 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004d20 con 0x55570ce83e20 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d5e600640 1 -- 172.21.15.131:0/3257178317 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54004050 con 0x55570ce83e20 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d5e600640 1 -- 172.21.15.131:0/3257178317 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400f780 con 0x55570ce83e20 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d66000640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1d4c0a3c30 0x7f1d4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d66000640 1 -- 172.21.15.131:0/2264925355 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f1d580d0870 con 0x7f1d68102760 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d6e000640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1d4c0a3c30 0x7f1d4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d72de8740 1 -- 172.21.15.131:0/3257178317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 msgr2=0x55570ce862b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce862b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d54011470 tx=0x7f1d54011930 comp rx=0 tx=0).stop 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d72de8740 1 -- 172.21.15.131:0/3257178317 shutdown_connections 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce862b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3257178317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce7e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.290+0000 7f1d72de8740 1 -- 172.21.15.131:0/3257178317 >> 172.21.15.131:0/3257178317 conn(0x55570ce12d10 msgr2=0x55570ce79d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.291+0000 7f1d72de8740 1 -- 172.21.15.131:0/3257178317 shutdown_connections 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.291+0000 7f1d72de8740 1 -- 172.21.15.131:0/3257178317 wait complete. 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.291+0000 7f1d6e000640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1d4c0a3c30 0x7f1d4c0a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d60004770 tx=0x7f1d60005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.291+0000 7f1d72de8740 1 -- start start 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.291+0000 7f1d72de8740 1 Processor -- start 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce8cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce8d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce83e20 0x55570ce94040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d72de8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55570ce86dd0 con 0x55570ce7e2c0 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d72de8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55570cd69f70 con 0x55570ce83e20 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d72de8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55570ce86f50 con 0x55570ce7ecb0 2024-04-26T20:07:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce8cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d72de8740 10 client.0 _ll_register_callbacks cb 0x55570cdd8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce8cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36264/0 (socket says 172.21.15.131:36264) 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 -- 172.21.15.131:0/755490183 learned_addr learned my addr 172.21.15.131:0/755490183 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce8d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6e000640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce83e20 0x55570ce94040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 -- 172.21.15.131:0/755490183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 msgr2=0x55570ce8d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce8d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 -- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce83e20 msgr2=0x55570ce94040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce83e20 0x55570ce94040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6d600640 1 -- 172.21.15.131:0/755490183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570cbbe6b0 con 0x55570ce7e2c0 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.292+0000 7f1d6e000640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce83e20 0x55570ce94040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d6d600640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce8cac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d58098d10 tx=0x7f1d5809c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8855e46f5ce4f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d52a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d52a00640 1 -- 172.21.15.131:0/755490183 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5809c8b0 con 0x55570ce7e2c0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d52a00640 1 -- 172.21.15.131:0/755490183 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f1d58010a90 con 0x55570ce7e2c0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d54004050 con 0x55570ce7e2c0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d52a00640 1 -- 172.21.15.131:0/755490183 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1d580a0c60 con 0x55570ce7e2c0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d72de8740 10 client.14559 fetch_fsmap learned FSMap version 17 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55570ce97090 con 0x55570ce7e2c0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.293+0000 7f1d52a00640 1 -- 172.21.15.131:0/755490183 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54004050 con 0x55570ce7e2c0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d52a00640 1 -- 172.21.15.131:0/755490183 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f1d580a5120 con 0x55570ce7e2c0 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d72de8740 10 client.14559 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d72de8740 10 client.14559 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d72de8740 20 client.14559 populate_metadata read hostname 'smithi131' 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d72de8740 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570ce99730 0x55570ce9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d72de8740 4 client.14559 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:07:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570cdf0210 con 0x55570ce99730 2024-04-26T20:07:42.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.294+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570ce99730 0x55570ce9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.295+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570ce99730 0x55570ce9aba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14ad8d28c992765 server_cookie=a1f0448742879752 in_seq=0 out_seq=0 2024-04-26T20:07:42.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.295+0000 7f1d52a00640 10 client.14559 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d52a00640 1 -- 172.21.15.131:0/755490183 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55570cdf0210 con 0x55570ce99730 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d52a00640 10 client.14559 handle_command_reply: tid=0 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 client.14559 shutdown 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 msgr2=0x55570ce8cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce8cac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d58098d10 tx=0x7f1d5809c4b0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 shutdown_connections 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570ce99730 0x55570ce9aba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce83e20 0x55570ce94040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce8d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 --2- 172.21.15.131:0/755490183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce8cac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d52a00640 0 client.14559 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 >> 172.21.15.131:0/755490183 conn(0x55570ce12d10 msgr2=0x55570ce78f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:42.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 shutdown_connections 2024-04-26T20:07:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 1 -- 172.21.15.131:0/755490183 wait complete. 2024-04-26T20:07:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.296+0000 7f1d72de8740 20 client.14559 trim_cache size 0 max 16384 2024-04-26T20:07:42.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:41 smithi148 ceph-mon[32281]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T20:07:42.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:41 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2013875009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:42.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:41 smithi148 ceph-mon[32281]: pgmap v336: 129 pgs: 4 creating+peering, 107 active+clean, 18 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-26T20:07:42.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:41 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2310627150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 Processor -- start 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 -- start start 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce94000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55570ce98ee0 con 0x55570ce83e20 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55570ce995b0 con 0x55570ce7e2c0 2024-04-26T20:07:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.316+0000 7f1d72de8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55570ce99c80 con 0x55570ce7ecb0 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce94000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce94000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56418/0 (socket says 172.21.15.131:56418) 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3451581074 learned_addr learned my addr 172.21.15.131:0/3451581074 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3451581074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 msgr2=0x55570ce797c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3451581074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce797c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3451581074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 msgr2=0x55570ce79d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3451581074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce79d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 -- 172.21.15.131:0/3451581074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570ce83b60 con 0x55570ce7ecb0 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.317+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3451581074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce94000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d54011440 tx=0x7f1d54004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f8ee888a078fad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d5dc00640 1 -- 172.21.15.131:0/3451581074 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54009b30 con 0x55570ce7ecb0 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d5dc00640 1 -- 172.21.15.131:0/3451581074 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54003a30 con 0x55570ce7ecb0 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d5dc00640 1 -- 172.21.15.131:0/3451581074 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004af0 con 0x55570ce7ecb0 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 -- 172.21.15.131:0/3451581074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 msgr2=0x55570ce94000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3451581074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce94000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d54011440 tx=0x7f1d54004730 comp rx=0 tx=0).stop 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 -- 172.21.15.131:0/3451581074 shutdown_connections 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3451581074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce83e20 0x55570ce79d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3451581074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce7ecb0 0x55570ce94000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3451581074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7e2c0 0x55570ce797c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 -- 172.21.15.131:0/3451581074 >> 172.21.15.131:0/3451581074 conn(0x55570ce12d10 msgr2=0x55570ce87ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 -- 172.21.15.131:0/3451581074 shutdown_connections 2024-04-26T20:07:42.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.318+0000 7f1d72de8740 1 -- 172.21.15.131:0/3451581074 wait complete. 2024-04-26T20:07:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 -- start start 2024-04-26T20:07:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 Processor -- start 2024-04-26T20:07:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce93950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7ecb0 0x55570ce93e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce83e20 0x55570ce90460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55570ce9aa20 con 0x55570ce7e2c0 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55570ce9a350 con 0x55570ce7ecb0 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55570ce9b0f0 con 0x55570ce83e20 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d72de8740 10 client.0 _ll_register_callbacks cb 0x7f1d58098c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d6d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce93950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d6d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce93950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36274/0 (socket says 172.21.15.131:36274) 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.319+0000 7f1d6d600640 1 -- 172.21.15.131:0/3081177244 learned_addr learned my addr 172.21.15.131:0/3081177244 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6e000640 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce83e20 0x55570ce90460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7ecb0 0x55570ce93e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6d600640 1 -- 172.21.15.131:0/3081177244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce83e20 msgr2=0x55570ce90460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6d600640 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce83e20 0x55570ce90460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6d600640 1 -- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7ecb0 msgr2=0x55570ce93e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6d600640 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7ecb0 0x55570ce93e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6d600640 1 -- 172.21.15.131:0/3081177244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570ce90b70 con 0x55570ce7e2c0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d6d600640 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce93950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d580a5490 tx=0x7f1d580a53c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53f526c4b7f2690 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d52a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d52a00640 1 -- 172.21.15.131:0/3081177244 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5809c8b0 con 0x55570ce7e2c0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d52a00640 1 -- 172.21.15.131:0/3081177244 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f1d580870c0 con 0x55570ce7e2c0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d54003a30 con 0x55570ce7e2c0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d52a00640 1 -- 172.21.15.131:0/3081177244 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1d580872c0 con 0x55570ce7e2c0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d72de8740 10 client.14565 fetch_fsmap learned FSMap version 17 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.320+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55570cf003b0 con 0x55570ce7e2c0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d52a00640 1 -- 172.21.15.131:0/3081177244 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54003a30 con 0x55570ce7e2c0 2024-04-26T20:07:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d52a00640 1 -- 172.21.15.131:0/3081177244 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f1d580d0c80 con 0x55570ce7e2c0 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d72de8740 10 client.14565 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d72de8740 10 client.14565 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d72de8740 20 client.14565 populate_metadata read hostname 'smithi131' 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570cf054c0 0x55570cf01900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d72de8740 4 client.14565 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "status"}) -- 0x55570cdea080 con 0x55570cf054c0 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.321+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570cf054c0 0x55570cf01900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d6cc00640 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570cf054c0 0x55570cf01900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a8f113de365b89 server_cookie=5de3b621aa0772bc in_seq=0 out_seq=0 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d52a00640 10 client.14565 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d52a00640 1 -- 172.21.15.131:0/3081177244 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55570cdea080 con 0x55570cf054c0 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d52a00640 10 client.14565 handle_command_reply: tid=0 2024-04-26T20:07:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 client.14565 shutdown 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 msgr2=0x55570ce93950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce93950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d580a5490 tx=0x7f1d580a53c0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 shutdown_connections 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55570cf054c0 0x55570cf01900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55570ce83e20 0x55570ce90460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570ce7ecb0 0x55570ce93e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 --2- 172.21.15.131:0/3081177244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55570ce7e2c0 0x55570ce93950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.322+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 >> 172.21.15.131:0/3081177244 conn(0x55570ce12d10 msgr2=0x55570ce797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d52a00640 0 client.14565 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 shutdown_connections 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d72de8740 1 -- 172.21.15.131:0/3081177244 wait complete. 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d72de8740 20 client.14565 trim_cache size 0 max 16384 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24575, 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 7.5993758539999998, 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 17, 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 92, 2024-04-26T20:07:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 91, 2024-04-26T20:07:42.331 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 30.179038859999999 2024-04-26T20:07:42.331 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:07:42.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1d4c0a3c30 msgr2=0x7f1d4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1d4c0a3c30 0x7f1d4c0a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d60004770 tx=0x7f1d60005a20 comp rx=0 tx=0).stop 2024-04-26T20:07:42.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 msgr2=0x7f1d68198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:42.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 0x7f1d68198920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9c0 tx=0x7f1d5800ee80 comp rx=0 tx=0).stop 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 shutdown_connections 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1d4c0a3c30 0x7f1d4c0a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d68104310 0x7f1d6819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d68103950 0x7f1d68198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 --2- 172.21.15.131:0/2264925355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68102760 0x7f1d68198920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.323+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 >> 172.21.15.131:0/2264925355 conn(0x7f1d680fdcb0 msgr2=0x7f1d68106c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.324+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 shutdown_connections 2024-04-26T20:07:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:42.324+0000 7f1d6ea00640 1 -- 172.21.15.131:0/2264925355 wait complete. 2024-04-26T20:07:42.342 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:07:42.342 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:07:42.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T20:07:42.756 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:07:43.118 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:42 smithi131 ceph-mon[27919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T20:07:43.118 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:42 smithi131 ceph-mon[27919]: Cluster is now healthy 2024-04-26T20:07:43.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:42 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2915687523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:42 smithi157 ceph-mon[32703]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T20:07:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:42 smithi157 ceph-mon[32703]: Cluster is now healthy 2024-04-26T20:07:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:42 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2915687523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:43.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:42 smithi148 ceph-mon[32281]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T20:07:43.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:42 smithi148 ceph-mon[32281]: Cluster is now healthy 2024-04-26T20:07:43.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:42 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2915687523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:07:44.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:43 smithi157 ceph-mon[32703]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 529 B/s wr, 1 op/s 2024-04-26T20:07:44.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:43 smithi131 ceph-mon[27919]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 529 B/s wr, 1 op/s 2024-04-26T20:07:44.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:43 smithi148 ceph-mon[32281]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 529 B/s wr, 1 op/s 2024-04-26T20:07:45.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 -- 172.21.15.131:0/1988923507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e8103f50 msgr2=0x7fb7e8107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/1988923507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e8103f50 0x7fb7e8107fa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b0d0 tx=0x7fb7d000b3e0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 -- 172.21.15.131:0/1988923507 shutdown_connections 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/1988923507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e8108680 0x7fb7e8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/1988923507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e81035a0 0x7fb7e8103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/1988923507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e8103f50 0x7fb7e8107fa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 -- 172.21.15.131:0/1988923507 >> 172.21.15.131:0/1988923507 conn(0x7fb7e80fee30 msgr2=0x7fb7e8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 -- 172.21.15.131:0/1988923507 shutdown_connections 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.187+0000 7fb7ecc00700 1 -- 172.21.15.131:0/1988923507 wait complete. 2024-04-26T20:07:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 Processor -- start 2024-04-26T20:07:45.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 -- start start 2024-04-26T20:07:45.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e81035a0 0x7fb7e819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:45.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e8103f50 0x7fb7e819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e8108680 0x7fb7e81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8113540 con 0x7fb7e81035a0 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e81133c0 con 0x7fb7e8103f50 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.189+0000 7fb7ecc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e81136c0 con 0x7fb7e8108680 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e81035a0 0x7fb7e819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e81035a0 0x7fb7e819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60998/0 (socket says 172.21.15.131:60998) 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 -- 172.21.15.131:0/2182599052 learned_addr learned my addr 172.21.15.131:0/2182599052 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7dea00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e8103f50 0x7fb7e819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7e00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e8108680 0x7fb7e81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 -- 172.21.15.131:0/2182599052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e8108680 msgr2=0x7fb7e81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e8108680 0x7fb7e81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 -- 172.21.15.131:0/2182599052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e8103f50 msgr2=0x7fb7e819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e8103f50 0x7fb7e819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.190+0000 7fb7e7400700 1 -- 172.21.15.131:0/2182599052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e81a0d20 con 0x7fb7e81035a0 2024-04-26T20:07:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.191+0000 7fb7e7400700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e81035a0 0x7fb7e819bd10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800e9a0 tx=0x7fb7d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.191+0000 7fb7e4c00700 1 -- 172.21.15.131:0/2182599052 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d800c9c0 con 0x7fb7e81035a0 2024-04-26T20:07:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.191+0000 7fb7e4c00700 1 -- 172.21.15.131:0/2182599052 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d800cb60 con 0x7fb7e81035a0 2024-04-26T20:07:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.191+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e81a1010 con 0x7fb7e81035a0 2024-04-26T20:07:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.191+0000 7fb7e4c00700 1 -- 172.21.15.131:0/2182599052 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8018680 con 0x7fb7e81035a0 2024-04-26T20:07:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.191+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e81a8890 con 0x7fb7e81035a0 2024-04-26T20:07:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.192+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e806a110 con 0x7fb7e81035a0 2024-04-26T20:07:45.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.194+0000 7fb7e4c00700 1 -- 172.21.15.131:0/2182599052 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb7d8010430 con 0x7fb7e81035a0 2024-04-26T20:07:45.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.196+0000 7fb7e4c00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7cc0a3c40 0x7fb7cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:45.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.196+0000 7fb7e4c00700 1 -- 172.21.15.131:0/2182599052 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb7d80d0950 con 0x7fb7e81035a0 2024-04-26T20:07:45.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.196+0000 7fb7dea00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7cc0a3c40 0x7fb7cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:07:45.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.196+0000 7fb7dea00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7cc0a3c40 0x7fb7cc0a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7e81027f0 tx=0x7fb7d000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:07:45.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.200+0000 7fb7e4c00700 1 -- 172.21.15.131:0/2182599052 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb7d809d090 con 0x7fb7e81035a0 2024-04-26T20:07:45.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.479+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb7e81a8a60 con 0x7fb7e81035a0 2024-04-26T20:07:45.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.479+0000 7fb7e4c00700 1 -- 172.21.15.131:0/2182599052 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fb7d809c990 con 0x7fb7e81035a0 2024-04-26T20:07:45.481 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.482+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7cc0a3c40 msgr2=0x7fb7cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.482+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7cc0a3c40 0x7fb7cc0a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7e81027f0 tx=0x7fb7d000b040 comp rx=0 tx=0).stop 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e81035a0 msgr2=0x7fb7e819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e81035a0 0x7fb7e819bd10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800e9a0 tx=0x7fb7d800ecb0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 shutdown_connections 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e8108680 0x7fb7e81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e81035a0 0x7fb7e819bd10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7cc0a3c40 0x7fb7cc0a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 --2- 172.21.15.131:0/2182599052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e8103f50 0x7fb7e819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 >> 172.21.15.131:0/2182599052 conn(0x7fb7e80fee30 msgr2=0x7fb7e8101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 shutdown_connections 2024-04-26T20:07:45.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:45.483+0000 7fb7ecc00700 1 -- 172.21.15.131:0/2182599052 wait complete. 2024-04-26T20:07:45.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:45 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2182599052' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T20:07:45.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:45 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2182599052' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T20:07:45.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:45 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2182599052' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T20:07:46.399 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:07:46.399 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:07:46.399 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-04-26T20:07:46.400 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T20:07:46.400 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:07:46.400 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:07:46.400 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:07:46.400 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:07:46.400 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T20:07:46.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.400 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:07:46.453 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42098sec preferred_lft 42098sec 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-04-26T20:07:46.454 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:07:46.455 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:07:46.456 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:07:46.456 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:07:46.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:07:46.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-04-26T20:07:46.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-04-26T20:07:46.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:07:46.456 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:07:46.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:07:46.593 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:46 smithi131 ceph-mon[27919]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:46.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:46 smithi148 ceph-mon[32281]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:46.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:07:46.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:46 smithi157 ceph-mon[32703]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:46.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.991 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:07:47.058 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-04-26T20:07:47.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:47.062 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-04-26T20:07:47.120 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-04-26T20:07:47.120 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:07:47.120 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:07:47.120 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:07:47.120 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:07:47.121 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:07:47.121 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:07:47.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:07:47.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:07:47.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:07:47.121 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:07:47.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:07:47.939 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:47 smithi131 ceph-mon[27919]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:47.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:07:48.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:48.047 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-26T20:07:48.079 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:47 smithi148 ceph-mon[32281]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:48.112 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:47 smithi157 ceph-mon[32703]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:48.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:48.229 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-04-26T20:07:48.322 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:07:48.354 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:07:48.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:07:48.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T20:07:48.504 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:07:48.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:07:48.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:07:48.747 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T20:07:48.779 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:07:48.821 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:07:48.854 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T20:07:48.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:07:48.937 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T20:07:48.971 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T20:07:49.012 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T20:07:49.079 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:07:49.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:07:49.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:07:49.464 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:07:49.529 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:07:49.563 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-04-26T20:07:49.596 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:07:49.629 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:07:49.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:07:49.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:07:49.898 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T20:07:49.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:07:49.963 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:07:50.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:49 smithi157 ceph-mon[32703]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:50.063 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:49 smithi131 ceph-mon[27919]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:50.097 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:49 smithi148 ceph-mon[32281]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T20:07:50.153 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T20:07:50.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:07:50.213 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:07:50.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-26T20:07:50.517 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:07:50.605 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-26T20:07:50.671 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-26T20:07:50.705 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-26T20:07:50.738 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-26T20:07:50.771 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-26T20:07:50.805 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-26T20:07:50.838 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-26T20:07:50.884 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-26T20:07:50.954 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-26T20:07:50.993 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-26T20:07:51.030 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-26T20:07:51.063 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-26T20:07:51.097 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:07:51.130 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-26T20:07:51.163 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-26T20:07:51.197 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-26T20:07:51.255 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-26T20:07:51.288 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-26T20:07:51.322 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-26T20:07:51.355 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:07:51.389 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:07:51.422 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:07:51.473 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:07:51.505 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-26T20:07:51.539 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-26T20:07:51.596 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay 2024-04-26T20:07:51.642 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/7a08768083423340753b4bbb37a7f56ad478180c439e9b6225804f60434551dd/merged 2024-04-26T20:07:51.680 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/e78b17a0d1e152ceacf29dda04c6f9ac26350fdb6fe1681133fb5c26974a80d6/merged 2024-04-26T20:07:51.715 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/0 2024-04-26T20:07:51.757 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:07:51.789 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/bb128e8252f64e1791cb98335bf223feee6ce870586ae802b7e1c1da220dc763/merged 2024-04-26T20:07:51.822 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/825d8bab6198a8b6531570cece4820a291e5a0b2d697c8ecc652d839790f076e/merged 2024-04-26T20:07:51.872 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/9627f1c80f712aa2efb6e41096612c6cc168c2661d65429bba78ebe6a84cfefd/merged 2024-04-26T20:07:51.906 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/66a9f2aa83f1f7ece4926b2eeab658fc329bf629eac4bad07804872e5337b981/merged 2024-04-26T20:07:51.939 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/d3b09a67a1b43a0e88cdb68a4d66affd23644d55e09cccd3e7d3198ff2c3d6d7/merged 2024-04-26T20:07:52.008 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/00f30a7ea2827a966c20fe0ebe0aafa34cbd39a15099aca43e7295741b38a91c/merged 2024-04-26T20:07:52.047 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/bd565d09141c116f60bc174c6128a2848c46afca56e342f0a70217cde7212df0/merged 2024-04-26T20:07:52.081 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/85a6303defeede33e7af379cf2451d02004ad61939980448be62021501a64808/merged 2024-04-26T20:07:52.114 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-26T20:07:52.147 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:07:52.181 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:07:52.215 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:51 smithi148 ceph-mon[32281]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:52.249 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:51 smithi157 ceph-mon[32703]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:52.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:52.331 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-26T20:07:52.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:51 smithi131 ceph-mon[27919]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:07:52.456 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:07:52.481 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T20:07:52.539 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-26T20:07:52.639 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:07:52.714 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-26T20:07:52.739 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-26T20:07:52.773 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-26T20:07:52.806 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-26T20:07:52.840 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-26T20:07:52.889 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-26T20:07:52.932 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-26T20:07:52.965 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-26T20:07:53.015 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-26T20:07:53.091 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-26T20:07:53.142 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-26T20:07:53.182 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-26T20:07:53.215 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:07:53.248 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-26T20:07:53.315 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-26T20:07:53.349 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-26T20:07:53.419 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-26T20:07:53.451 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-26T20:07:53.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-26T20:07:53.524 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:07:53.557 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:07:53.591 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:07:53.630 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:07:53.666 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-26T20:07:53.699 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-26T20:07:53.732 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay 2024-04-26T20:07:53.788 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/7a08768083423340753b4bbb37a7f56ad478180c439e9b6225804f60434551dd/merged 2024-04-26T20:07:53.851 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/e78b17a0d1e152ceacf29dda04c6f9ac26350fdb6fe1681133fb5c26974a80d6/merged 2024-04-26T20:07:53.891 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/0 2024-04-26T20:07:53.924 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:07:53.982 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/bb128e8252f64e1791cb98335bf223feee6ce870586ae802b7e1c1da220dc763/merged 2024-04-26T20:07:54.016 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/825d8bab6198a8b6531570cece4820a291e5a0b2d697c8ecc652d839790f076e/merged 2024-04-26T20:07:54.066 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/9627f1c80f712aa2efb6e41096612c6cc168c2661d65429bba78ebe6a84cfefd/merged 2024-04-26T20:07:54.099 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/66a9f2aa83f1f7ece4926b2eeab658fc329bf629eac4bad07804872e5337b981/merged 2024-04-26T20:07:54.133 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/d3b09a67a1b43a0e88cdb68a4d66affd23644d55e09cccd3e7d3198ff2c3d6d7/merged 2024-04-26T20:07:54.166 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/00f30a7ea2827a966c20fe0ebe0aafa34cbd39a15099aca43e7295741b38a91c/merged 2024-04-26T20:07:54.199 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/bd565d09141c116f60bc174c6128a2848c46afca56e342f0a70217cde7212df0/merged 2024-04-26T20:07:54.232 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/containers/storage/overlay/85a6303defeede33e7af379cf2451d02004ad61939980448be62021501a64808/merged 2024-04-26T20:07:54.266 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-26T20:07:54.326 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:07:54.358 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:07:54.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:53 smithi148 ceph-mon[32281]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:54.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:53 smithi157 ceph-mon[32703]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:54.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:54.526 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-26T20:07:54.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:2024-04-26T20:07:52.796+0000 7f9852522480 -1 init, newargv = 0x56279a1bd850 newargc=15 2024-04-26T20:07:54.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[71348]: starting ceph client 2024-04-26T20:07:54.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[71348]: starting fuse 2024-04-26T20:07:54.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:53 smithi131 ceph-mon[27919]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:54.774 INFO:teuthology.orchestra.run.smithi131.stdout:114 2024-04-26T20:07:54.800 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-26T20:07:54.833 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:07:54.882 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-04-26T20:07:54.916 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-04-26T20:07:54.950 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-04-26T20:07:54.983 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-04-26T20:07:55.042 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-26T20:07:55.101 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-04-26T20:07:55.133 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T20:07:55.167 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-04-26T20:07:55.201 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T20:07:55.251 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-26T20:07:55.317 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-04-26T20:07:55.363 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-04-26T20:07:55.400 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-04-26T20:07:55.460 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-04-26T20:07:55.492 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-26T20:07:55.525 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-04-26T20:07:55.558 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:07:55.592 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:07:55.625 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:07:55.659 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-04-26T20:07:55.692 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-04-26T20:07:55.748 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-04-26T20:07:55.784 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:07:55.835 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-26T20:07:55.867 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.admin")) 2024-04-26T20:07:55.935 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-04-26T20:07:55.994 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:55 smithi157 ceph-mon[32703]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:56.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:55 smithi131 ceph-mon[27919]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:56.069 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:55 smithi148 ceph-mon[32281]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:56.168 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.admin.71348.asok 2024-04-26T20:07:56.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:07:56.226 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71348.asok 2024-04-26T20:07:56.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:56.292 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71348.asok status 2024-04-26T20:07:56.516 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:07:56.584 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-04-26T20:07:56.617 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:07:56.651 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:07:56.684 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "admin", 2024-04-26T20:07:56.717 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-04-26T20:07:56.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T20:07:56.784 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "71348", 2024-04-26T20:07:56.853 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-04-26T20:07:56.885 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:07:56.918 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-04-26T20:07:56.951 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-04-26T20:07:56.984 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24698, 2024-04-26T20:07:57.018 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-04-26T20:07:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-04-26T20:07:57.093 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-04-26T20:07:57.126 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 24698 2024-04-26T20:07:57.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:07:57.203 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-26T20:07:57.234 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-26T20:07:57.268 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-04-26T20:07:57.301 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1191296517 2024-04-26T20:07:57.370 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:07:57.409 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:07:57.460 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-26T20:07:57.516 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-26T20:07:57.551 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-04-26T20:07:57.585 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1191296517 2024-04-26T20:07:57.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:07:57.651 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.24698 172.21.15.131:0/1191296517", 2024-04-26T20:07:57.685 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "172.21.15.131:0/1191296517", 2024-04-26T20:07:57.718 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-04-26T20:07:57.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 17, 2024-04-26T20:07:57.785 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 92, 2024-04-26T20:07:57.818 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:07:57.852 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-04-26T20:07:57.885 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-04-26T20:07:57.918 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:07:57.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:58.019 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-04-26T20:07:58.044 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:07:58.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:57 smithi148 ceph-mon[32281]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:58.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:57 smithi157 ceph-mon[32703]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:58.186 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:57 smithi131 ceph-mon[27919]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:07:58.326 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-04-26T20:07:58.361 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:07:58.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:58.460 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:07:58.579 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T20:07:58.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:07:58.765 INFO:teuthology.orchestra.run.smithi131.stdout:# file: . 2024-04-26T20:07:58.812 INFO:teuthology.orchestra.run.smithi131.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T20:07:58.872 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:07:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:07:59.024 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:07:59.077 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:59.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:07:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 -- 172.21.15.131:0/3291766523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 msgr2=0x7f09401091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:07:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 --2- 172.21.15.131:0/3291766523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f09401091b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0930009860 tx=0x7f093002a340 comp rx=0 tx=0).stop 2024-04-26T20:07:59.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 -- 172.21.15.131:0/3291766523 shutdown_connections 2024-04-26T20:07:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 --2- 172.21.15.131:0/3291766523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094010a940 0x7f09401111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 --2- 172.21.15.131:0/3291766523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0940109f80 0x7f094010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:59.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 --2- 172.21.15.131:0/3291766523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f09401091b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:07:59.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 -- 172.21.15.131:0/3291766523 >> 172.21.15.131:0/3291766523 conn(0x7f0940077d30 msgr2=0x7f094007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:07:59.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 -- 172.21.15.131:0/3291766523 shutdown_connections 2024-04-26T20:07:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.469+0000 7f0945c00640 1 -- 172.21.15.131:0/3291766523 wait complete. 2024-04-26T20:07:59.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 Processor -- start 2024-04-26T20:07:59.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 -- start start 2024-04-26T20:07:59.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f094019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:07:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0940109f80 0x7f094019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094010a940 0x7f09401a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:00.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0940114890 con 0x7f0940109f80 2024-04-26T20:08:00.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0940114710 con 0x7f0940108d90 2024-04-26T20:08:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.470+0000 7f0945c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0940114a10 con 0x7f094010a940 2024-04-26T20:08:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f094019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:00.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f094019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:47318/0 (socket says 172.21.15.131:47318) 2024-04-26T20:08:00.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 -- 172.21.15.131:0/241708092 learned_addr learned my addr 172.21.15.131:0/241708092 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f0945200640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094010a940 0x7f09401a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f0937e00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0940109f80 0x7f094019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:00.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 -- 172.21.15.131:0/241708092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094010a940 msgr2=0x7f09401a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094010a940 0x7f09401a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 -- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0940109f80 msgr2=0x7f094019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:00.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0940109f80 0x7f094019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:00.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.471+0000 7f093fe00640 1 -- 172.21.15.131:0/241708092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09401a6bd0 con 0x7f0940108d90 2024-04-26T20:08:00.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.472+0000 7f093fe00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f094019efb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0930005940 tx=0x7f0930032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:00.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.472+0000 7f093d600640 1 -- 172.21.15.131:0/241708092 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930040020 con 0x7f0940108d90 2024-04-26T20:08:00.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.472+0000 7f093d600640 1 -- 172.21.15.131:0/241708092 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0930004040 con 0x7f0940108d90 2024-04-26T20:08:00.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.472+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09401a6e60 con 0x7f0940108d90 2024-04-26T20:08:00.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.472+0000 7f093d600640 1 -- 172.21.15.131:0/241708092 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930037040 con 0x7f0940108d90 2024-04-26T20:08:00.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.472+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09401a7270 con 0x7f0940108d90 2024-04-26T20:08:00.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.473+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09401101d0 con 0x7f0940108d90 2024-04-26T20:08:00.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.475+0000 7f093d600640 1 -- 172.21.15.131:0/241708092 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0930043020 con 0x7f0940108d90 2024-04-26T20:08:01.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.476+0000 7f093d600640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f09240a3bb0 0x7f09240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:01.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.477+0000 7f093d600640 1 -- 172.21.15.131:0/241708092 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f09300f0c10 con 0x7f0940108d90 2024-04-26T20:08:01.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.477+0000 7f0937e00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f09240a3bb0 0x7f09240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.477+0000 7f0937e00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f09240a3bb0 0x7f09240a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09401a04c0 tx=0x7f0928005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:01.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.479+0000 7f093d600640 1 -- 172.21.15.131:0/241708092 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f093004a020 con 0x7f0940108d90 2024-04-26T20:08:01.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.684+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f094002dca0 con 0x7f0940108d90 2024-04-26T20:08:01.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.686+0000 7f093d600640 1 -- 172.21.15.131:0/241708092 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24187 (secure 0 0 0) 0x7f09300bcd00 con 0x7f0940108d90 2024-04-26T20:08:01.239 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:08:01.305 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":92,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:07:40.812449+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f09240a3bb0 msgr2=0x7f09240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f09240a3bb0 0x7f09240a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09401a04c0 tx=0x7f0928005f90 comp rx=0 tx=0).stop 2024-04-26T20:08:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 msgr2=0x7f094019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f094019efb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0930005940 tx=0x7f0930032dc0 comp rx=0 tx=0).stop 2024-04-26T20:08:01.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 shutdown_connections 2024-04-26T20:08:01.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f09240a3bb0 0x7f09240a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:01.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094010a940 0x7f09401a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:01.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0940109f80 0x7f094019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 --2- 172.21.15.131:0/241708092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0940108d90 0x7f094019efb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:01.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.689+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 >> 172.21.15.131:0/241708092 conn(0x7f0940077d30 msgr2=0x7f094010d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:01.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.690+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 shutdown_connections 2024-04-26T20:08:01.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:07:59.690+0000 7f0945c00640 1 -- 172.21.15.131:0/241708092 wait complete. 2024-04-26T20:08:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:08:01.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:08:01.889 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:07:59 smithi131 ceph-mon[27919]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:01.922 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:00 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/241708092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:01.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:07:59 smithi148 ceph-mon[32281]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:01.988 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:00 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/241708092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:02.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:07:59 smithi157 ceph-mon[32703]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:02.114 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:00 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/241708092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:02.148 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:01 smithi148 ceph-mon[32281]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:02.224 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:01 smithi157 ceph-mon[32703]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:02.264 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:01 smithi131 ceph-mon[27919]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.173+0000 7f8d11200640 1 -- 172.21.15.131:0/4281823465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c1043f0 msgr2=0x7f8d0c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.173+0000 7f8d11200640 1 --2- 172.21.15.131:0/4281823465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c1043f0 0x7f8d0c10ac80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009ca0 tx=0x7f8d0402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:08:02.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.174+0000 7f8d11200640 1 -- 172.21.15.131:0/4281823465 shutdown_connections 2024-04-26T20:08:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.174+0000 7f8d11200640 1 --2- 172.21.15.131:0/4281823465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c1043f0 0x7f8d0c10ac80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:02.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.174+0000 7f8d11200640 1 --2- 172.21.15.131:0/4281823465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d0c103a30 0x7f8d0c103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.174+0000 7f8d11200640 1 --2- 172.21.15.131:0/4281823465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d0c102840 0x7f8d0c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.174+0000 7f8d11200640 1 -- 172.21.15.131:0/4281823465 >> 172.21.15.131:0/4281823465 conn(0x7f8d0c0fddd0 msgr2=0x7f8d0c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:02.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.174+0000 7f8d11200640 1 -- 172.21.15.131:0/4281823465 shutdown_connections 2024-04-26T20:08:02.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.174+0000 7f8d11200640 1 -- 172.21.15.131:0/4281823465 wait complete. 2024-04-26T20:08:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 Processor -- start 2024-04-26T20:08:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 -- start start 2024-04-26T20:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c102840 0x7f8d0c108630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:02.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d0c103a30 0x7f8d0c108b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:02.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d0c1043f0 0x7f8d0c1051a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10e3c0 con 0x7f8d0c103a30 2024-04-26T20:08:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10e240 con 0x7f8d0c1043f0 2024-04-26T20:08:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d11200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10e540 con 0x7f8d0c102840 2024-04-26T20:08:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d0b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c102840 0x7f8d0c108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d02a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d0c103a30 0x7f8d0c108b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d0b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c102840 0x7f8d0c108630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47674/0 (socket says 172.21.15.131:47674) 2024-04-26T20:08:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d0b400640 1 -- 172.21.15.131:0/735229542 learned_addr learned my addr 172.21.15.131:0/735229542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.175+0000 7f8d0be00640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d0c1043f0 0x7f8d0c1051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:03.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d0b400640 1 -- 172.21.15.131:0/735229542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d0c1043f0 msgr2=0x7f8d0c1051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:03.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d0b400640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d0c1043f0 0x7f8d0c1051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d0b400640 1 -- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d0c103a30 msgr2=0x7f8d0c108b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:03.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d0b400640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d0c103a30 0x7f8d0c108b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d0b400640 1 -- 172.21.15.131:0/735229542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c105a60 con 0x7f8d0c102840 2024-04-26T20:08:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d0be00640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d0c1043f0 0x7f8d0c1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:08:03.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d02a00640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d0c103a30 0x7f8d0c108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:08:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d0b400640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c102840 0x7f8d0c108630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00e9c0 tx=0x7f8cfc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d08c00640 1 -- 172.21.15.131:0/735229542 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc00cd80 con 0x7f8d0c102840 2024-04-26T20:08:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d08c00640 1 -- 172.21.15.131:0/735229542 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cfc0035f0 con 0x7f8d0c102840 2024-04-26T20:08:03.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d08c00640 1 -- 172.21.15.131:0/735229542 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc010690 con 0x7f8d0c102840 2024-04-26T20:08:03.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.176+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c1a7010 con 0x7f8d0c102840 2024-04-26T20:08:03.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.177+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c1a7420 con 0x7f8d0c102840 2024-04-26T20:08:03.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.178+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c1a76b0 con 0x7f8d0c102840 2024-04-26T20:08:03.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.180+0000 7f8d08c00640 1 -- 172.21.15.131:0/735229542 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8cfc004000 con 0x7f8d0c102840 2024-04-26T20:08:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.181+0000 7f8d08c00640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cf00a3bb0 0x7f8cf00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:04.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.181+0000 7f8d08c00640 1 -- 172.21.15.131:0/735229542 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f8cfc0d0d60 con 0x7f8d0c102840 2024-04-26T20:08:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.181+0000 7f8d02a00640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cf00a3bb0 0x7f8cf00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.182+0000 7f8d02a00640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cf00a3bb0 0x7f8cf00a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1065b0 tx=0x7f8cf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.184+0000 7f8d08c00640 1 -- 172.21.15.131:0/735229542 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8cfc09d4a0 con 0x7f8d0c102840 2024-04-26T20:08:04.199 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:03 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/735229542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:08:04.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:03 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/735229542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:08:04.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.397+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d0c1a7880 con 0x7f8d0c102840 2024-04-26T20:08:04.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.398+0000 7f8d08c00640 1 -- 172.21.15.131:0/735229542 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f8cfc09cda0 con 0x7f8d0c102840 2024-04-26T20:08:04.400 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:08:04.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cf00a3bb0 msgr2=0x7f8cf00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cf00a3bb0 0x7f8cf00a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1065b0 tx=0x7f8cf4005f90 comp rx=0 tx=0).stop 2024-04-26T20:08:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c102840 msgr2=0x7f8d0c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c102840 0x7f8d0c108630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00e9c0 tx=0x7f8cfc00ee80 comp rx=0 tx=0).stop 2024-04-26T20:08:04.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 shutdown_connections 2024-04-26T20:08:04.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d0c1043f0 0x7f8d0c1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:04.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d0c103a30 0x7f8d0c108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:04.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cf00a3bb0 0x7f8cf00a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:04.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 --2- 172.21.15.131:0/735229542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0c102840 0x7f8d0c108630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:04.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 >> 172.21.15.131:0/735229542 conn(0x7f8d0c0fddd0 msgr2=0x7f8d0c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:04.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 shutdown_connections 2024-04-26T20:08:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:02.401+0000 7f8d11200640 1 -- 172.21.15.131:0/735229542 wait complete. 2024-04-26T20:08:04.874 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:08:04.908 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:03 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/735229542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:08:04.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:04 smithi131 ceph-mon[27919]: pgmap v347: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T20:08:04.975 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:04 smithi148 ceph-mon[32281]: pgmap v347: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T20:08:05.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:04 smithi157 ceph-mon[32703]: pgmap v347: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T20:08:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 -- 172.21.15.131:0/37207968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7410a6f0 msgr2=0x7f6f7410cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 --2- 172.21.15.131:0/37207968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7410a6f0 0x7f6f7410cb80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009ca0 tx=0x7f6f6c02a950 comp rx=0 tx=0).stop 2024-04-26T20:08:05.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 -- 172.21.15.131:0/37207968 shutdown_connections 2024-04-26T20:08:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 --2- 172.21.15.131:0/37207968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7410a6f0 0x7f6f7410cb80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 --2- 172.21.15.131:0/37207968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f74075310 0x7f6f74075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 --2- 172.21.15.131:0/37207968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f740768f0 0x7f6f74074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:05.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 -- 172.21.15.131:0/37207968 >> 172.21.15.131:0/37207968 conn(0x7f6f740fddd0 msgr2=0x7f6f741001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:05.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 -- 172.21.15.131:0/37207968 shutdown_connections 2024-04-26T20:08:05.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.190+0000 7f6f73200640 1 -- 172.21.15.131:0/37207968 wait complete. 2024-04-26T20:08:05.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.191+0000 7f6f73200640 1 Processor -- start 2024-04-26T20:08:05.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.191+0000 7f6f73200640 1 -- start start 2024-04-26T20:08:05.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f73200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f74075310 0x7f6f7419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:05.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f73200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f740768f0 0x7f6f7419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:05.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f73200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7410a6f0 0x7f6f741a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:05.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f73200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74110580 con 0x7f6f74075310 2024-04-26T20:08:05.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f73200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74110400 con 0x7f6f7410a6f0 2024-04-26T20:08:06.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f73200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74110700 con 0x7f6f740768f0 2024-04-26T20:08:06.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f71e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f74075310 0x7f6f7419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:06.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f740768f0 0x7f6f7419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:06.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f740768f0 0x7f6f7419b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56120/0 (socket says 172.21.15.131:56120) 2024-04-26T20:08:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 -- 172.21.15.131:0/1851859064 learned_addr learned my addr 172.21.15.131:0/1851859064 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:06.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f72800640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7410a6f0 0x7f6f741a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 -- 172.21.15.131:0/1851859064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7410a6f0 msgr2=0x7f6f741a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:06.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7410a6f0 0x7f6f741a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 -- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f74075310 msgr2=0x7f6f7419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:06.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f74075310 0x7f6f7419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:06.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f6be00640 1 -- 172.21.15.131:0/1851859064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f741a2870 con 0x7f6f740768f0 2024-04-26T20:08:06.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f72800640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7410a6f0 0x7f6f741a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:08:06.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.192+0000 7f6f71e00640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f74075310 0x7f6f7419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:08:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.193+0000 7f6f6be00640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f740768f0 0x7f6f7419b1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00e9f0 tx=0x7f6f5c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.193+0000 7f6f6aa00640 1 -- 172.21.15.131:0/1851859064 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00cd80 con 0x7f6f740768f0 2024-04-26T20:08:06.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.193+0000 7f6f6aa00640 1 -- 172.21.15.131:0/1851859064 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5c0054d0 con 0x7f6f740768f0 2024-04-26T20:08:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.193+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741a2b60 con 0x7f6f740768f0 2024-04-26T20:08:06.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.193+0000 7f6f6aa00640 1 -- 172.21.15.131:0/1851859064 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c010690 con 0x7f6f740768f0 2024-04-26T20:08:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.193+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741a30b0 con 0x7f6f740768f0 2024-04-26T20:08:06.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.194+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f741094b0 con 0x7f6f740768f0 2024-04-26T20:08:06.843 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:05 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1851859064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:08:06.875 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:06 smithi148 ceph-mon[32281]: pgmap v348: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:06.909 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:05 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1851859064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:08:06.975 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:06 smithi157 ceph-mon[32703]: pgmap v348: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:07.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.196+0000 7f6f6aa00640 1 -- 172.21.15.131:0/1851859064 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6f5c004e30 con 0x7f6f740768f0 2024-04-26T20:08:07.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.198+0000 7f6f6aa00640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f580a3c30 0x7f6f580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:07.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.198+0000 7f6f6aa00640 1 -- 172.21.15.131:0/1851859064 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f6f5c0d0d20 con 0x7f6f740768f0 2024-04-26T20:08:07.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.198+0000 7f6f71e00640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f580a3c30 0x7f6f580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:07.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.199+0000 7f6f71e00640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f580a3c30 0x7f6f580a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6f7419c130 tx=0x7f6f6400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:07.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.200+0000 7f6f6aa00640 1 -- 172.21.15.131:0/1851859064 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6f5c054060 con 0x7f6f740768f0 2024-04-26T20:08:07.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.405+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6f7402dc60 con 0x7f6f740768f0 2024-04-26T20:08:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.406+0000 7f6f6aa00640 1 -- 172.21.15.131:0/1851859064 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f6f5c09d460 con 0x7f6f740768f0 2024-04-26T20:08:07.334 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:08:07.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f580a3c30 msgr2=0x7f6f580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:07.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f580a3c30 0x7f6f580a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6f7419c130 tx=0x7f6f6400b410 comp rx=0 tx=0).stop 2024-04-26T20:08:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f740768f0 msgr2=0x7f6f7419b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f740768f0 0x7f6f7419b1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00e9f0 tx=0x7f6f5c00edf0 comp rx=0 tx=0).stop 2024-04-26T20:08:07.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 shutdown_connections 2024-04-26T20:08:07.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7410a6f0 0x7f6f741a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:07.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f580a3c30 0x7f6f580a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f740768f0 0x7f6f7419b1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 --2- 172.21.15.131:0/1851859064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f74075310 0x7f6f7419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.409+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 >> 172.21.15.131:0/1851859064 conn(0x7f6f740fddd0 msgr2=0x7f6f74106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:07.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.410+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 shutdown_connections 2024-04-26T20:08:07.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:05.410+0000 7f6f73200640 1 -- 172.21.15.131:0/1851859064 wait complete. 2024-04-26T20:08:07.943 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:08:08.008 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:05 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1851859064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:08:08.043 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:06 smithi131 ceph-mon[27919]: pgmap v348: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:08.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:07 smithi131 ceph-mon[27919]: pgmap v349: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:08.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:07 smithi148 ceph-mon[32281]: pgmap v349: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:08.158 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:07 smithi157 ceph-mon[32703]: pgmap v349: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:08.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.308+0000 7fdee9c00640 1 -- 172.21.15.131:0/1236340745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 msgr2=0x7fdee4102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:08.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.308+0000 7fdee9c00640 1 --2- 172.21.15.131:0/1236340745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 0x7fdee4102b90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fded4009ca0 tx=0x7fded402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:08:08.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.309+0000 7fdee9c00640 1 -- 172.21.15.131:0/1236340745 shutdown_connections 2024-04-26T20:08:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.309+0000 7fdee9c00640 1 --2- 172.21.15.131:0/1236340745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee4104320 0x7fdee410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:08.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.309+0000 7fdee9c00640 1 --2- 172.21.15.131:0/1236340745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4103960 0x7fdee4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.309+0000 7fdee9c00640 1 --2- 172.21.15.131:0/1236340745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 0x7fdee4102b90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.309+0000 7fdee9c00640 1 -- 172.21.15.131:0/1236340745 >> 172.21.15.131:0/1236340745 conn(0x7fdee40fdce0 msgr2=0x7fdee4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:08.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.309+0000 7fdee9c00640 1 -- 172.21.15.131:0/1236340745 shutdown_connections 2024-04-26T20:08:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.309+0000 7fdee9c00640 1 -- 172.21.15.131:0/1236340745 wait complete. 2024-04-26T20:08:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 Processor -- start 2024-04-26T20:08:08.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 -- start start 2024-04-26T20:08:08.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 0x7fdee419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:08.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee4103960 0x7fdee419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:08.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4104320 0x7fdee41a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdee410e2f0 con 0x7fdee4104320 2024-04-26T20:08:08.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdee410e170 con 0x7fdee4103960 2024-04-26T20:08:09.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdee410e470 con 0x7fdee4102770 2024-04-26T20:08:09.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4104320 0x7fdee41a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:09.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4104320 0x7fdee41a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42124/0 (socket says 172.21.15.131:42124) 2024-04-26T20:08:09.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee9200640 1 -- 172.21.15.131:0/2985212533 learned_addr learned my addr 172.21.15.131:0/2985212533 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdee3e00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 0x7fdee419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:09.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.310+0000 7fdedb400640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee4103960 0x7fdee419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee9200640 1 -- 172.21.15.131:0/2985212533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 msgr2=0x7fdee419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee9200640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 0x7fdee419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee9200640 1 -- 172.21.15.131:0/2985212533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee4103960 msgr2=0x7fdee419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:09.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee9200640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee4103960 0x7fdee419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:09.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee9200640 1 -- 172.21.15.131:0/2985212533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee41a27a0 con 0x7fdee4104320 2024-04-26T20:08:09.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee9200640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4104320 0x7fdee41a2130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00e970 tx=0x7fdedc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee1600640 1 -- 172.21.15.131:0/2985212533 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc00ccc0 con 0x7fdee4104320 2024-04-26T20:08:09.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee1600640 1 -- 172.21.15.131:0/2985212533 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdedc00ce60 con 0x7fdee4104320 2024-04-26T20:08:09.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.311+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee41a2a90 con 0x7fdee4104320 2024-04-26T20:08:09.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.312+0000 7fdee1600640 1 -- 172.21.15.131:0/2985212533 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc010430 con 0x7fdee4104320 2024-04-26T20:08:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.312+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee41a3030 con 0x7fdee4104320 2024-04-26T20:08:09.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.313+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee41a3260 con 0x7fdee4104320 2024-04-26T20:08:09.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.315+0000 7fdee1600640 1 -- 172.21.15.131:0/2985212533 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdedc004f90 con 0x7fdee4104320 2024-04-26T20:08:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.316+0000 7fdee1600640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdec80a3c30 0x7fdec80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.316+0000 7fdee1600640 1 -- 172.21.15.131:0/2985212533 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fdedc0d09a0 con 0x7fdee4104320 2024-04-26T20:08:09.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.317+0000 7fdee3e00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdec80a3c30 0x7fdec80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:09.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.317+0000 7fdee3e00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdec80a3c30 0x7fdec80a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fded4000f80 tx=0x7fded4005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:10.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.321+0000 7fdee1600640 1 -- 172.21.15.131:0/2985212533 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdedc09d0e0 con 0x7fdee4104320 2024-04-26T20:08:10.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.528+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdee41a35c0 con 0x7fdee4104320 2024-04-26T20:08:10.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.529+0000 7fdee1600640 1 -- 172.21.15.131:0/2985212533 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fdedc09c9e0 con 0x7fdee4104320 2024-04-26T20:08:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:08:10.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:08 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2985212533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:08:10.220 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:09 smithi148 ceph-mon[32281]: pgmap v350: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:10.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:08 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2985212533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:08:10.286 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:09 smithi157 ceph-mon[32703]: pgmap v350: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:10.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.531+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdec80a3c30 msgr2=0x7fdec80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:10.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.531+0000 7fdee9c00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdec80a3c30 0x7fdec80a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fded4000f80 tx=0x7fded4005f30 comp rx=0 tx=0).stop 2024-04-26T20:08:10.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.531+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4104320 msgr2=0x7fdee41a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:10.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.531+0000 7fdee9c00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4104320 0x7fdee41a2130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00e970 tx=0x7fdedc00ed70 comp rx=0 tx=0).stop 2024-04-26T20:08:10.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 shutdown_connections 2024-04-26T20:08:10.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee4104320 0x7fdee41a2130 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:10.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee4103960 0x7fdee419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:10.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdec80a3c30 0x7fdec80a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:10.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 --2- 172.21.15.131:0/2985212533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee4102770 0x7fdee419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:10.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 >> 172.21.15.131:0/2985212533 conn(0x7fdee40fdce0 msgr2=0x7fdee4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:10.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 shutdown_connections 2024-04-26T20:08:10.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:08.532+0000 7fdee9c00640 1 -- 172.21.15.131:0/2985212533 wait complete. 2024-04-26T20:08:10.937 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:08:10.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:08 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2985212533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:08:11.003 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:09 smithi131 ceph-mon[27919]: pgmap v350: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:11.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 -- 172.21.15.131:0/28578701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8102840 msgr2=0x7f2ea8102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 --2- 172.21.15.131:0/28578701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8102840 0x7f2ea8102c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009900 tx=0x7f2e9802a410 comp rx=0 tx=0).stop 2024-04-26T20:08:11.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 -- 172.21.15.131:0/28578701 shutdown_connections 2024-04-26T20:08:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 --2- 172.21.15.131:0/28578701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea81043f0 0x7f2ea810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:11.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 --2- 172.21.15.131:0/28578701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8103a30 0x7f2ea8103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:11.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 --2- 172.21.15.131:0/28578701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8102840 0x7f2ea8102c60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:11.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 -- 172.21.15.131:0/28578701 >> 172.21.15.131:0/28578701 conn(0x7f2ea80fddd0 msgr2=0x7f2ea81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:11.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 -- 172.21.15.131:0/28578701 shutdown_connections 2024-04-26T20:08:11.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.258+0000 7f2eaca00640 1 -- 172.21.15.131:0/28578701 wait complete. 2024-04-26T20:08:11.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.259+0000 7f2eaca00640 1 Processor -- start 2024-04-26T20:08:11.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2eaca00640 1 -- start start 2024-04-26T20:08:11.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2eaca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8102840 0x7f2ea819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:11.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2eaca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8103a30 0x7f2ea819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:11.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2eaca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea81043f0 0x7f2ea81a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:12.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2eaca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea810e340 con 0x7f2ea8103a30 2024-04-26T20:08:12.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2eaca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea810e1c0 con 0x7f2ea81043f0 2024-04-26T20:08:12.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2eaca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea810e4c0 con 0x7f2ea8102840 2024-04-26T20:08:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2e9de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8103a30 0x7f2ea819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2ea6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8102840 0x7f2ea819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2e9de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8103a30 0x7f2ea819b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42144/0 (socket says 172.21.15.131:42144) 2024-04-26T20:08:12.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2e9de00640 1 -- 172.21.15.131:0/3815160648 learned_addr learned my addr 172.21.15.131:0/3815160648 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:12.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.260+0000 7f2ea7200640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea81043f0 0x7f2ea81a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:12.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9de00640 1 -- 172.21.15.131:0/3815160648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8102840 msgr2=0x7f2ea819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:12.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9de00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8102840 0x7f2ea819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9de00640 1 -- 172.21.15.131:0/3815160648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea81043f0 msgr2=0x7f2ea81a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:12.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9de00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea81043f0 0x7f2ea81a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:12.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9de00640 1 -- 172.21.15.131:0/3815160648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea81a2910 con 0x7f2ea8103a30 2024-04-26T20:08:12.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2ea7200640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea81043f0 0x7f2ea81a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:08:12.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9de00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8103a30 0x7f2ea819b240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e9f0 tx=0x7f2e9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:12.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9fe00640 1 -- 172.21.15.131:0/3815160648 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000cd80 con 0x7f2ea8103a30 2024-04-26T20:08:12.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9fe00640 1 -- 172.21.15.131:0/3815160648 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e900108e0 con 0x7f2ea8103a30 2024-04-26T20:08:12.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2e9fe00640 1 -- 172.21.15.131:0/3815160648 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90018980 con 0x7f2ea8103a30 2024-04-26T20:08:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.261+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea81a2c00 con 0x7f2ea8103a30 2024-04-26T20:08:12.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.262+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea81a3100 con 0x7f2ea8103a30 2024-04-26T20:08:12.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.263+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea81a32d0 con 0x7f2ea8103a30 2024-04-26T20:08:12.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.264+0000 7f2e9fe00640 1 -- 172.21.15.131:0/3815160648 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2e90010430 con 0x7f2ea8103a30 2024-04-26T20:08:12.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.266+0000 7f2e9fe00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e8c0a3c30 0x7f2e8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:12.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.266+0000 7f2e9fe00640 1 -- 172.21.15.131:0/3815160648 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f2e900d0aa0 con 0x7f2ea8103a30 2024-04-26T20:08:12.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.266+0000 7f2ea6800640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e8c0a3c30 0x7f2e8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:12.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.267+0000 7f2ea6800640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e8c0a3c30 0x7f2e8c0a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ea819c1d0 tx=0x7f2e980314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:13.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.271+0000 7f2e9fe00640 1 -- 172.21.15.131:0/3815160648 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2e9009d1e0 con 0x7f2ea8103a30 2024-04-26T20:08:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.474+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2ea81a3660 con 0x7f2ea8103a30 2024-04-26T20:08:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.475+0000 7f2e9fe00640 1 -- 172.21.15.131:0/3815160648 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f2e9009cae0 con 0x7f2ea8103a30 2024-04-26T20:08:13.171 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:08:13.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:11 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3815160648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:08:13.267 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:12 smithi157 ceph-mon[32703]: pgmap v351: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:13.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:11 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3815160648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:08:13.347 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:12 smithi148 ceph-mon[32281]: pgmap v351: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:13.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e8c0a3c30 msgr2=0x7f2e8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:13.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e8c0a3c30 0x7f2e8c0a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ea819c1d0 tx=0x7f2e980314a0 comp rx=0 tx=0).stop 2024-04-26T20:08:13.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8103a30 msgr2=0x7f2ea819b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:13.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8103a30 0x7f2ea819b240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e9f0 tx=0x7f2e9000eeb0 comp rx=0 tx=0).stop 2024-04-26T20:08:13.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 shutdown_connections 2024-04-26T20:08:13.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea81043f0 0x7f2ea81a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:13.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea8103a30 0x7f2ea819b240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:13.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e8c0a3c30 0x7f2e8c0a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:13.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.477+0000 7f2eaca00640 1 --2- 172.21.15.131:0/3815160648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8102840 0x7f2ea819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:13.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.478+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 >> 172.21.15.131:0/3815160648 conn(0x7f2ea80fddd0 msgr2=0x7f2ea81091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:13.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.478+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 shutdown_connections 2024-04-26T20:08:13.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:11.478+0000 7f2eaca00640 1 -- 172.21.15.131:0/3815160648 wait complete. 2024-04-26T20:08:13.864 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:08:13.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:11 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3815160648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:08:13.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:12 smithi131 ceph-mon[27919]: pgmap v351: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:14.012 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-26T20:08:14.047 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-26T20:08:14.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-26T20:08:14.113 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-04-26T20:08:14.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-26T20:08:14.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-26T20:08:14.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-26T20:08:14.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-26T20:08:14.322 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-04-26T20:08:14.356 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-26T20:08:14.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-26T20:08:14.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-26T20:08:14.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-26T20:08:14.589 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-04-26T20:08:14.622 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-26T20:08:14.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.229+0000 7fed05000640 1 -- 172.21.15.131:0/901474507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 msgr2=0x7fed0010cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.229+0000 7fed05000640 1 --2- 172.21.15.131:0/901474507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 0x7fed0010cff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009ca0 tx=0x7fecf802a8c0 comp rx=0 tx=0).stop 2024-04-26T20:08:14.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.229+0000 7fed05000640 1 -- 172.21.15.131:0/901474507 shutdown_connections 2024-04-26T20:08:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.229+0000 7fed05000640 1 --2- 172.21.15.131:0/901474507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 0x7fed0010cff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:14.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.229+0000 7fed05000640 1 --2- 172.21.15.131:0/901474507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed00105da0 0x7fed00106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:14.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.229+0000 7fed05000640 1 --2- 172.21.15.131:0/901474507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed00104bb0 0x7fed00104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:14.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.229+0000 7fed05000640 1 -- 172.21.15.131:0/901474507 >> 172.21.15.131:0/901474507 conn(0x7fed00100120 msgr2=0x7fed00102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:14.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.230+0000 7fed05000640 1 -- 172.21.15.131:0/901474507 shutdown_connections 2024-04-26T20:08:15.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.230+0000 7fed05000640 1 -- 172.21.15.131:0/901474507 wait complete. 2024-04-26T20:08:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 Processor -- start 2024-04-26T20:08:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 -- start start 2024-04-26T20:08:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed00104bb0 0x7fed0019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:15.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed00105da0 0x7fed0019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:15.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 0x7fed001a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:15.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed001106e0 con 0x7fed00104bb0 2024-04-26T20:08:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed00110560 con 0x7fed00106760 2024-04-26T20:08:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.231+0000 7fed05000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed00110860 con 0x7fed00105da0 2024-04-26T20:08:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed00104bb0 0x7fed0019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:15.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed00104bb0 0x7fed0019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41682/0 (socket says 172.21.15.131:41682) 2024-04-26T20:08:15.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfe800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 0x7fed001a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:15.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfde00640 1 -- 172.21.15.131:0/1624121527 learned_addr learned my addr 172.21.15.131:0/1624121527 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:15.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecf7e00640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed00105da0 0x7fed0019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfe800640 1 -- 172.21.15.131:0/1624121527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed00105da0 msgr2=0x7fed0019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:15.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfe800640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed00105da0 0x7fed0019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfe800640 1 -- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed00104bb0 msgr2=0x7fed0019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:15.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfe800640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed00104bb0 0x7fed0019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfe800640 1 -- 172.21.15.131:0/1624121527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed001a2980 con 0x7fed00106760 2024-04-26T20:08:15.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecf7e00640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed00105da0 0x7fed0019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:08:15.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.232+0000 7fecfe800640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 0x7fed001a2310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fecf8031a20 tx=0x7fecf80317a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:15.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.233+0000 7fecf6a00640 1 -- 172.21.15.131:0/1624121527 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8003c30 con 0x7fed00106760 2024-04-26T20:08:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.233+0000 7fecf6a00640 1 -- 172.21.15.131:0/1624121527 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf8003dd0 con 0x7fed00106760 2024-04-26T20:08:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.233+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed001a2c10 con 0x7fed00106760 2024-04-26T20:08:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.233+0000 7fecf6a00640 1 -- 172.21.15.131:0/1624121527 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf802fb60 con 0x7fed00106760 2024-04-26T20:08:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.233+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed001a3160 con 0x7fed00106760 2024-04-26T20:08:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.234+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0010bfa0 con 0x7fed00106760 2024-04-26T20:08:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.236+0000 7fecf6a00640 1 -- 172.21.15.131:0/1624121527 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fecf802fd00 con 0x7fed00106760 2024-04-26T20:08:16.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.237+0000 7fecf6a00640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fece40a3bb0 0x7fece40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:16.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.238+0000 7fecf6a00640 1 -- 172.21.15.131:0/1624121527 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8947+0+0 (secure 0 0 0) 0x7fecf80f1df0 con 0x7fed00106760 2024-04-26T20:08:16.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.238+0000 7fecfde00640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fece40a3bb0 0x7fece40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:16.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.238+0000 7fecfde00640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fece40a3bb0 0x7fece40a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fecf00095f0 tx=0x7fecf0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.240+0000 7fecf6a00640 1 -- 172.21.15.131:0/1624121527 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fecf80be530 con 0x7fed00106760 2024-04-26T20:08:16.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.459+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed0002dad0 con 0x7fed00106760 2024-04-26T20:08:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.460+0000 7fecf6a00640 1 -- 172.21.15.131:0/1624121527 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+601 (secure 0 0 0) 0x7fecf80bde30 con 0x7fed00106760 2024-04-26T20:08:16.450 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3514900702 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:16.499 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/3789992830 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:16.532 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/3789992830 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:16.615 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/2560715060 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:16.641 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/2579641682 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:16.674 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/2560715060 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:16.707 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3589087841 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:16.741 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3946098705 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:16.774 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1232073682 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:16.807 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/537078740 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:16.871 INFO:teuthology.orchestra.run.smithi131.stderr:listed 10 entries 2024-04-26T20:08:16.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fece40a3bb0 msgr2=0x7fece40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fece40a3bb0 0x7fece40a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fecf00095f0 tx=0x7fecf0005f90 comp rx=0 tx=0).stop 2024-04-26T20:08:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 msgr2=0x7fed001a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:17.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 0x7fed001a2310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fecf8031a20 tx=0x7fecf80317a0 comp rx=0 tx=0).stop 2024-04-26T20:08:17.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 shutdown_connections 2024-04-26T20:08:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed00106760 0x7fed001a2310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fece40a3bb0 0x7fece40a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:17.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed00105da0 0x7fed0019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:17.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 --2- 172.21.15.131:0/1624121527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed00104bb0 0x7fed0019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 >> 172.21.15.131:0/1624121527 conn(0x7fed00100120 msgr2=0x7fed00109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:17.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 shutdown_connections 2024-04-26T20:08:17.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:14.463+0000 7fed05000640 1 -- 172.21.15.131:0/1624121527 wait complete. 2024-04-26T20:08:17.408 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:08:17.441 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:17.475 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-26T20:08:17.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-04-26T20:08:17.569 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]: dispatch 2024-04-26T20:08:17.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-26T20:08:17.665 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-26T20:08:17.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-26T20:08:17.750 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-26T20:08:17.783 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-26T20:08:17.833 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-04-26T20:08:17.866 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-26T20:08:17.900 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T20:08:17.933 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: pgmap v353: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:17.967 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:14 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1624121527' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:08:18.044 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-26T20:08:18.075 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-04-26T20:08:18.108 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2024-04-26T20:08:18.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-26T20:08:18.175 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-26T20:08:18.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:15 smithi131 ceph-mon[27919]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T20:08:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:16 smithi131 ceph-mon[27919]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T20:08:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:16 smithi131 ceph-mon[27919]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:18.369 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-26T20:08:18.400 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-04-26T20:08:18.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]: dispatch 2024-04-26T20:08:18.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-26T20:08:18.525 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-26T20:08:18.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-26T20:08:18.592 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-26T20:08:18.625 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-26T20:08:18.659 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-04-26T20:08:18.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-26T20:08:18.750 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T20:08:18.784 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: pgmap v353: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:18.825 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:14 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1624121527' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:08:18.875 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-26T20:08:18.909 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-04-26T20:08:18.969 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2024-04-26T20:08:19.001 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-26T20:08:19.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-26T20:08:19.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:15 smithi157 ceph-mon[32703]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T20:08:19.217 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:16 smithi157 ceph-mon[32703]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T20:08:19.251 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:16 smithi157 ceph-mon[32703]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:19.284 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:17 smithi157 ceph-mon[32703]: pgmap v357: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:19.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-26T20:08:19.351 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-04-26T20:08:19.384 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]: dispatch 2024-04-26T20:08:19.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-26T20:08:19.509 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-26T20:08:19.548 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-26T20:08:19.584 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-26T20:08:19.627 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-26T20:08:19.701 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-04-26T20:08:19.734 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-26T20:08:19.776 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T20:08:19.802 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: pgmap v353: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:19.835 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:14 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1624121527' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:08:19.935 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-26T20:08:19.968 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-04-26T20:08:20.002 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2024-04-26T20:08:20.035 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-26T20:08:20.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-26T20:08:20.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:15 smithi148 ceph-mon[32281]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T20:08:20.162 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:16 smithi148 ceph-mon[32281]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T20:08:20.194 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:16 smithi148 ceph-mon[32281]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:20.227 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:17 smithi148 ceph-mon[32281]: pgmap v357: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:20.262 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:08:20.326 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:19 smithi157 ceph-mon[32703]: pgmap v358: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:20.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:08:20.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:19 smithi148 ceph-mon[32281]: pgmap v358: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:20.427 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:17 smithi131 ceph-mon[27919]: pgmap v357: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:20.476 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:08:20.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:19 smithi131 ceph-mon[27919]: pgmap v358: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:20.589 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:08:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 -- 172.21.15.131:0/3343089644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620106740 msgr2=0x7f262010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 --2- 172.21.15.131:0/3343089644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620106740 0x7f262010cfd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2618009860 tx=0x7f261802a340 comp rx=0 tx=0).stop 2024-04-26T20:08:20.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 -- 172.21.15.131:0/3343089644 shutdown_connections 2024-04-26T20:08:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 --2- 172.21.15.131:0/3343089644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620106740 0x7f262010cfd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:21.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 --2- 172.21.15.131:0/3343089644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2620105d80 0x7f2620106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:21.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 --2- 172.21.15.131:0/3343089644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2620104b90 0x7f2620104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:21.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 -- 172.21.15.131:0/3343089644 >> 172.21.15.131:0/3343089644 conn(0x7f2620100120 msgr2=0x7f2620102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 -- 172.21.15.131:0/3343089644 shutdown_connections 2024-04-26T20:08:21.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.837+0000 7f2628600640 1 -- 172.21.15.131:0/3343089644 wait complete. 2024-04-26T20:08:21.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.838+0000 7f2628600640 1 Processor -- start 2024-04-26T20:08:21.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.838+0000 7f2628600640 1 -- start start 2024-04-26T20:08:21.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2628600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620104b90 0x7f262019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2628600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2620105d80 0x7f262019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2628600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2620106740 0x7f26201a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:21.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2628600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26201106a0 con 0x7f2620105d80 2024-04-26T20:08:21.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2628600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2620110520 con 0x7f2620106740 2024-04-26T20:08:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2628600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2620110820 con 0x7f2620104b90 2024-04-26T20:08:21.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620104b90 0x7f262019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:21.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620104b90 0x7f262019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43622/0 (socket says 172.21.15.131:43622) 2024-04-26T20:08:21.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 -- 172.21.15.131:0/282327964 learned_addr learned my addr 172.21.15.131:0/282327964 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:21.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2626600640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2620106740 0x7f26201a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:21.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625200640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2620105d80 0x7f262019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:21.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 -- 172.21.15.131:0/282327964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2620106740 msgr2=0x7f26201a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:21.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2620106740 0x7f26201a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:21.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 -- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2620105d80 msgr2=0x7f262019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:21.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2620105d80 0x7f262019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.839+0000 7f2625c00640 1 -- 172.21.15.131:0/282327964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26201a2940 con 0x7f2620104b90 2024-04-26T20:08:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.840+0000 7f2625c00640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620104b90 0x7f262019ad30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f261400ebc0 tx=0x7f261400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:22.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.840+0000 7f261e000640 1 -- 172.21.15.131:0/282327964 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614003b70 con 0x7f2620104b90 2024-04-26T20:08:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.840+0000 7f261e000640 1 -- 172.21.15.131:0/282327964 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2614003d10 con 0x7f2620104b90 2024-04-26T20:08:22.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.840+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26201a2c30 con 0x7f2620104b90 2024-04-26T20:08:22.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.840+0000 7f261e000640 1 -- 172.21.15.131:0/282327964 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614019800 con 0x7f2620104b90 2024-04-26T20:08:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.840+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201a3180 con 0x7f2620104b90 2024-04-26T20:08:22.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.841+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262010bf00 con 0x7f2620104b90 2024-04-26T20:08:22.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.843+0000 7f261e000640 1 -- 172.21.15.131:0/282327964 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2614004b60 con 0x7f2620104b90 2024-04-26T20:08:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.845+0000 7f261e000640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26080a3bb0 0x7f26080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:22.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.845+0000 7f261e000640 1 -- 172.21.15.131:0/282327964 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f26140d12f0 con 0x7f2620104b90 2024-04-26T20:08:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.845+0000 7f2625200640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26080a3bb0 0x7f26080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:22.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.846+0000 7f2625200640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26080a3bb0 0x7f26080a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f260c009530 tx=0x7f260c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:22.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:20.851+0000 7f261e000640 1 -- 172.21.15.131:0/282327964 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f261409d910 con 0x7f2620104b90 2024-04-26T20:08:22.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.063+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f262002db70 con 0x7f2620104b90 2024-04-26T20:08:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.066+0000 7f261e000640 1 -- 172.21.15.131:0/282327964 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24715 (secure 0 0 0) 0x7f261409d210 con 0x7f2620104b90 2024-04-26T20:08:22.758 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:22.791 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:22.824 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:22.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:21 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:22.891 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:21 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/282327964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:22.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:21 smithi148 ceph-mon[32281]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:23.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:23.091 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:23.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:23.157 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:21 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:23.211 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:21 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/282327964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:23.249 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:21 smithi157 ceph-mon[32703]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:23.283 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:08:23.316 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:08:15.562795+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:08:23.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26080a3bb0 msgr2=0x7f26080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:23.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26080a3bb0 0x7f26080a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f260c009530 tx=0x7f260c009260 comp rx=0 tx=0).stop 2024-04-26T20:08:23.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620104b90 msgr2=0x7f262019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:23.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620104b90 0x7f262019ad30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f261400ebc0 tx=0x7f261400c6a0 comp rx=0 tx=0).stop 2024-04-26T20:08:23.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 shutdown_connections 2024-04-26T20:08:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2620106740 0x7f26201a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:23.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2620105d80 0x7f262019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26080a3bb0 0x7f26080a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:23.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 --2- 172.21.15.131:0/282327964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2620104b90 0x7f262019ad30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.068+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 >> 172.21.15.131:0/282327964 conn(0x7f2620100120 msgr2=0x7f26201090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:23.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.069+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 shutdown_connections 2024-04-26T20:08:23.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:21.069+0000 7f2628600640 1 -- 172.21.15.131:0/282327964 wait complete. 2024-04-26T20:08:23.876 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:08:23.908 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:08:23.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:23.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.042 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:21 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.111 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:21 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/282327964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:24.162 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:21 smithi131 ceph-mon[27919]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:24.200 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.234 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:22 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:08:24.317 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:08:24.357 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:23 smithi131 ceph-mon[27919]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:24.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.492 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:22 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:08:24.559 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:08:24.592 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.626 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:23 smithi148 ceph-mon[32281]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:24.659 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.692 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:22 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:08:24.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:08:24.834 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:08:24.869 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:23 smithi157 ceph-mon[32703]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-04-26T20:08:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 -- 172.21.15.131:0/2010380152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 msgr2=0x7f4af8106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:24.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 --2- 172.21.15.131:0/2010380152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af8106220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009860 tx=0x7f4ae002a340 comp rx=0 tx=0).stop 2024-04-26T20:08:25.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 -- 172.21.15.131:0/2010380152 shutdown_connections 2024-04-26T20:08:25.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 --2- 172.21.15.131:0/2010380152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4af8106760 0x7f4af810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:25.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 --2- 172.21.15.131:0/2010380152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af8106220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 --2- 172.21.15.131:0/2010380152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4af8104bb0 0x7f4af8104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:25.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 -- 172.21.15.131:0/2010380152 >> 172.21.15.131:0/2010380152 conn(0x7f4af8100120 msgr2=0x7f4af8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:25.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 -- 172.21.15.131:0/2010380152 shutdown_connections 2024-04-26T20:08:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.485+0000 7f4aff800640 1 -- 172.21.15.131:0/2010380152 wait complete. 2024-04-26T20:08:25.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 Processor -- start 2024-04-26T20:08:25.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 -- start start 2024-04-26T20:08:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4af8104bb0 0x7f4af819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:25.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4af8106760 0x7f4af81a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:25.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81106c0 con 0x7f4af8106760 2024-04-26T20:08:25.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8110540 con 0x7f4af8105da0 2024-04-26T20:08:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.486+0000 7f4aff800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8110840 con 0x7f4af8104bb0 2024-04-26T20:08:25.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:25.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4afd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4af8106760 0x7f4af81a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:25.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:35220/0 (socket says 172.21.15.131:35220) 2024-04-26T20:08:25.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 -- 172.21.15.131:0/4241815421 learned_addr learned my addr 172.21.15.131:0/4241815421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:25.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4afd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4af8106760 0x7f4af81a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38076/0 (socket says 172.21.15.131:38076) 2024-04-26T20:08:25.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4afce00640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4af8104bb0 0x7f4af819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 -- 172.21.15.131:0/4241815421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4af8104bb0 msgr2=0x7f4af819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:25.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4af8104bb0 0x7f4af819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 -- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4af8106760 msgr2=0x7f4af81a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4af8106760 0x7f4af81a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 -- 172.21.15.131:0/4241815421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af81a29e0 con 0x7f4af8105da0 2024-04-26T20:08:26.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.487+0000 7f4af7e00640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af819b300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00057c0 tx=0x7f4ae0032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:26.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.488+0000 7f4af5600640 1 -- 172.21.15.131:0/4241815421 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0037040 con 0x7f4af8105da0 2024-04-26T20:08:26.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.488+0000 7f4af5600640 1 -- 172.21.15.131:0/4241815421 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae0030bd0 con 0x7f4af8105da0 2024-04-26T20:08:26.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.488+0000 7f4af5600640 1 -- 172.21.15.131:0/4241815421 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0005280 con 0x7f4af8105da0 2024-04-26T20:08:26.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.488+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af81a2c70 con 0x7f4af8105da0 2024-04-26T20:08:26.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.488+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af81a3150 con 0x7f4af8105da0 2024-04-26T20:08:26.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.489+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af810bf50 con 0x7f4af8105da0 2024-04-26T20:08:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.491+0000 7f4af5600640 1 -- 172.21.15.131:0/4241815421 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4ae0043020 con 0x7f4af8105da0 2024-04-26T20:08:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.492+0000 7f4af5600640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4adc0a3c30 0x7f4adc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:26.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.492+0000 7f4af5600640 1 -- 172.21.15.131:0/4241815421 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f4ae00efd60 con 0x7f4af8105da0 2024-04-26T20:08:26.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.492+0000 7f4afce00640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4adc0a3c30 0x7f4adc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:26.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.493+0000 7f4afce00640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4adc0a3c30 0x7f4adc0a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098d0 tx=0x7f4ae8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:26.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.497+0000 7f4af5600640 1 -- 172.21.15.131:0/4241815421 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4ae00bc380 con 0x7f4af8105da0 2024-04-26T20:08:26.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.699+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4af80635d0 con 0x7f4af8105da0 2024-04-26T20:08:26.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.700+0000 7f4af5600640 1 -- 172.21.15.131:0/4241815421 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f4ae00bbc80 con 0x7f4af8105da0 2024-04-26T20:08:26.578 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:08:26.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4adc0a3c30 msgr2=0x7f4adc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:26.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4adc0a3c30 0x7f4adc0a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098d0 tx=0x7f4ae8010040 comp rx=0 tx=0).stop 2024-04-26T20:08:26.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 msgr2=0x7f4af819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af819b300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00057c0 tx=0x7f4ae0032da0 comp rx=0 tx=0).stop 2024-04-26T20:08:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 shutdown_connections 2024-04-26T20:08:26.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4af8106760 0x7f4af81a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:26.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af8105da0 0x7f4af819b300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:26.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4adc0a3c30 0x7f4adc0a60f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:26.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 --2- 172.21.15.131:0/4241815421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4af8104bb0 0x7f4af819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:26.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 >> 172.21.15.131:0/4241815421 conn(0x7f4af8100120 msgr2=0x7f4af8109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:27.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 shutdown_connections 2024-04-26T20:08:27.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:24.703+0000 7f4aff800640 1 -- 172.21.15.131:0/4241815421 wait complete. 2024-04-26T20:08:27.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:08:27.262 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:25 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4241815421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:08:27.295 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:26 smithi131 ceph-mon[27919]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:27.329 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:25 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4241815421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:08:27.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:26 smithi157 ceph-mon[32703]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:27.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:25 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4241815421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:08:27.445 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:26 smithi148 ceph-mon[32281]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:27.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 -- 172.21.15.131:0/3351417694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10109fc0 msgr2=0x7f1f1010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:27.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3351417694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10109fc0 0x7f1f1010a440 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009900 tx=0x7f1efc02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:08:27.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 -- 172.21.15.131:0/3351417694 shutdown_connections 2024-04-26T20:08:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3351417694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f1010a980 0x7f1f101112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:27.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3351417694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10109fc0 0x7f1f1010a440 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3351417694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f10108e70 0x7f1f10109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 -- 172.21.15.131:0/3351417694 >> 172.21.15.131:0/3351417694 conn(0x7f1f10077e00 msgr2=0x7f1f1007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:27.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 -- 172.21.15.131:0/3351417694 shutdown_connections 2024-04-26T20:08:27.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.544+0000 7f1f0f200640 1 -- 172.21.15.131:0/3351417694 wait complete. 2024-04-26T20:08:27.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.545+0000 7f1f0f200640 1 Processor -- start 2024-04-26T20:08:27.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.545+0000 7f1f0f200640 1 -- start start 2024-04-26T20:08:27.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.545+0000 7f1f0f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10108e70 0x7f1f1019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:28.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.545+0000 7f1f0f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f10109fc0 0x7f1f1019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:28.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.545+0000 7f1f0f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1010a980 0x7f1f101a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101149d0 con 0x7f1f10108e70 2024-04-26T20:08:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10114850 con 0x7f1f1010a980 2024-04-26T20:08:28.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10114b50 con 0x7f1f10109fc0 2024-04-26T20:08:28.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1010a980 0x7f1f101a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1010a980 0x7f1f101a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:35234/0 (socket says 172.21.15.131:35234) 2024-04-26T20:08:28.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 -- 172.21.15.131:0/3891908611 learned_addr learned my addr 172.21.15.131:0/3891908611 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:28.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0de00640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10108e70 0x7f1f1019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:28.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0d400640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f10109fc0 0x7f1f1019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:28.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 -- 172.21.15.131:0/3891908611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f10109fc0 msgr2=0x7f1f1019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:28.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f10109fc0 0x7f1f1019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 -- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10108e70 msgr2=0x7f1f1019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:28.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10108e70 0x7f1f1019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.546+0000 7f1f0e800640 1 -- 172.21.15.131:0/3891908611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f101a6cc0 con 0x7f1f1010a980 2024-04-26T20:08:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.547+0000 7f1f0e800640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1010a980 0x7f1f101a6650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800e970 tx=0x7f1f0800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.547+0000 7f1f06a00640 1 -- 172.21.15.131:0/3891908611 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800ccc0 con 0x7f1f1010a980 2024-04-26T20:08:28.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.547+0000 7f1f06a00640 1 -- 172.21.15.131:0/3891908611 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0800ce60 con 0x7f1f1010a980 2024-04-26T20:08:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.547+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f101a6fb0 con 0x7f1f1010a980 2024-04-26T20:08:28.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.547+0000 7f1f06a00640 1 -- 172.21.15.131:0/3891908611 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08010430 con 0x7f1f1010a980 2024-04-26T20:08:28.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.547+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f101a7470 con 0x7f1f1010a980 2024-04-26T20:08:28.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.548+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f10110220 con 0x7f1f1010a980 2024-04-26T20:08:28.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.550+0000 7f1f06a00640 1 -- 172.21.15.131:0/3891908611 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1f08004f70 con 0x7f1f1010a980 2024-04-26T20:08:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.552+0000 7f1f06a00640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ef40a3bb0 0x7f1ef40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:29.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.552+0000 7f1f06a00640 1 -- 172.21.15.131:0/3891908611 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f1f080d0d60 con 0x7f1f1010a980 2024-04-26T20:08:29.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.552+0000 7f1f0de00640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ef40a3bb0 0x7f1ef40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.553+0000 7f1f0de00640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ef40a3bb0 0x7f1ef40a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f00005fd0 tx=0x7f1f00009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:29.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.554+0000 7f1f06a00640 1 -- 172.21.15.131:0/3891908611 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1f08054060 con 0x7f1f1010a980 2024-04-26T20:08:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.767+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f1002dc60 con 0x7f1f1010a980 2024-04-26T20:08:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.768+0000 7f1f06a00640 1 -- 172.21.15.131:0/3891908611 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f1f0809d380 con 0x7f1f1010a980 2024-04-26T20:08:29.290 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:08:29.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ef40a3bb0 msgr2=0x7f1ef40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:29.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ef40a3bb0 0x7f1ef40a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f00005fd0 tx=0x7f1f00009500 comp rx=0 tx=0).stop 2024-04-26T20:08:29.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1010a980 msgr2=0x7f1f101a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1010a980 0x7f1f101a6650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800e970 tx=0x7f1f0800ed70 comp rx=0 tx=0).stop 2024-04-26T20:08:29.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 shutdown_connections 2024-04-26T20:08:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1010a980 0x7f1f101a6650 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1ef40a3bb0 0x7f1ef40a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f10109fc0 0x7f1f1019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:29.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 --2- 172.21.15.131:0/3891908611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f10108e70 0x7f1f1019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:29.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 >> 172.21.15.131:0/3891908611 conn(0x7f1f10077e00 msgr2=0x7f1f1010d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:29.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 shutdown_connections 2024-04-26T20:08:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:27.771+0000 7f1f0f200640 1 -- 172.21.15.131:0/3891908611 wait complete. 2024-04-26T20:08:29.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:08:29.855 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:27 smithi131 ceph-mon[27919]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:29.888 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:28 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3891908611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:08:29.923 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:27 smithi148 ceph-mon[32281]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:29.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:28 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3891908611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:08:29.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:27 smithi157 ceph-mon[32703]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:30.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:28 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3891908611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:08:30.125 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:29 smithi148 ceph-mon[32281]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:30.163 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:29 smithi157 ceph-mon[32703]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:30.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:29 smithi131 ceph-mon[27919]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:30.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 -- 172.21.15.131:0/3832694223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 msgr2=0x7fe8dc10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:30.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/3832694223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc10a400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c402a410 comp rx=0 tx=0).stop 2024-04-26T20:08:30.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 -- 172.21.15.131:0/3832694223 shutdown_connections 2024-04-26T20:08:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/3832694223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8dc10a940 0x7fe8dc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:30.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/3832694223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc10a400 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:30.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/3832694223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 0x7fe8dc1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:30.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 -- 172.21.15.131:0/3832694223 >> 172.21.15.131:0/3832694223 conn(0x7fe8dc077d30 msgr2=0x7fe8dc07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:30.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 -- 172.21.15.131:0/3832694223 shutdown_connections 2024-04-26T20:08:30.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.138+0000 7fe8e1c00640 1 -- 172.21.15.131:0/3832694223 wait complete. 2024-04-26T20:08:30.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 Processor -- start 2024-04-26T20:08:30.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 -- start start 2024-04-26T20:08:30.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 0x7fe8dc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:30.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:30.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8dc10a940 0x7fe8dc1a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc114910 con 0x7fe8dc109f80 2024-04-26T20:08:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc114790 con 0x7fe8dc108d90 2024-04-26T20:08:30.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.139+0000 7fe8e1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc114a90 con 0x7fe8dc10a940 2024-04-26T20:08:30.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 0x7fe8dc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8db400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:31.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 0x7fe8dc19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:35242/0 (socket says 172.21.15.131:35242) 2024-04-26T20:08:31.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 -- 172.21.15.131:0/1374569051 learned_addr learned my addr 172.21.15.131:0/1374569051 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:31.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8db400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38112/0 (socket says 172.21.15.131:38112) 2024-04-26T20:08:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8e1200640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8dc10a940 0x7fe8dc1a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:31.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 -- 172.21.15.131:0/1374569051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8dc10a940 msgr2=0x7fe8dc1a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:31.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8dc10a940 0x7fe8dc1a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 -- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 msgr2=0x7fe8dc19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:31.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 -- 172.21.15.131:0/1374569051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc1a6bd0 con 0x7fe8dc108d90 2024-04-26T20:08:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8db400640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:08:31.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.140+0000 7fe8dbe00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 0x7fe8dc19efb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00d950 tx=0x7fe8cc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.141+0000 7fe8d8c00640 1 -- 172.21.15.131:0/1374569051 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc017020 con 0x7fe8dc108d90 2024-04-26T20:08:31.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.141+0000 7fe8d8c00640 1 -- 172.21.15.131:0/1374569051 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8cc009c30 con 0x7fe8dc108d90 2024-04-26T20:08:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.141+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc1a6ec0 con 0x7fe8dc108d90 2024-04-26T20:08:31.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.141+0000 7fe8d8c00640 1 -- 172.21.15.131:0/1374569051 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00f600 con 0x7fe8dc108d90 2024-04-26T20:08:31.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.141+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc1a7330 con 0x7fe8dc108d90 2024-04-26T20:08:31.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.142+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8dc110100 con 0x7fe8dc108d90 2024-04-26T20:08:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.144+0000 7fe8d8c00640 1 -- 172.21.15.131:0/1374569051 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe8cc01a020 con 0x7fe8dc108d90 2024-04-26T20:08:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.145+0000 7fe8d8c00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe8c00a3c30 0x7fe8c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:31.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.146+0000 7fe8d8c00640 1 -- 172.21.15.131:0/1374569051 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fe8cc0d09c0 con 0x7fe8dc108d90 2024-04-26T20:08:31.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.146+0000 7fe8db400640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe8c00a3c30 0x7fe8c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:31.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.146+0000 7fe8db400640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe8c00a3c30 0x7fe8c00a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4031500 tx=0x7fe8c4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:31.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.148+0000 7fe8d8c00640 1 -- 172.21.15.131:0/1374569051 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe8cc021020 con 0x7fe8dc108d90 2024-04-26T20:08:32.024 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:30 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1374569051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:08:32.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.347+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8dc02dad0 con 0x7fe8dc108d90 2024-04-26T20:08:32.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.348+0000 7fe8d8c00640 1 -- 172.21.15.131:0/1374569051 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fe8cc09d060 con 0x7fe8dc108d90 2024-04-26T20:08:32.123 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:30 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1374569051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:08:32.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:31 smithi157 ceph-mon[32703]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:32.220 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:08:32.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe8c00a3c30 msgr2=0x7fe8c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:32.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe8c00a3c30 0x7fe8c00a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4031500 tx=0x7fe8c4005a20 comp rx=0 tx=0).stop 2024-04-26T20:08:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 msgr2=0x7fe8dc19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 0x7fe8dc19efb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00d950 tx=0x7fe8cc00de10 comp rx=0 tx=0).stop 2024-04-26T20:08:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 shutdown_connections 2024-04-26T20:08:32.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe8c00a3c30 0x7fe8c00a60f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8dc10a940 0x7fe8dc1a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:32.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8dc109f80 0x7fe8dc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 --2- 172.21.15.131:0/1374569051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8dc108d90 0x7fe8dc19efb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:32.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 >> 172.21.15.131:0/1374569051 conn(0x7fe8dc077d30 msgr2=0x7fe8dc10d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:32.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 shutdown_connections 2024-04-26T20:08:32.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:30.351+0000 7fe8e1c00640 1 -- 172.21.15.131:0/1374569051 wait complete. 2024-04-26T20:08:32.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:08:32.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:30 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1374569051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:08:32.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:31 smithi131 ceph-mon[27919]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:32.899 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:31 smithi148 ceph-mon[32281]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 -- 172.21.15.131:0/1858078302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c106740 msgr2=0x7f2e1c10cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1858078302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c106740 0x7f2e1c10cfd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009ca0 tx=0x7f2e1402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:08:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 -- 172.21.15.131:0/1858078302 shutdown_connections 2024-04-26T20:08:33.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1858078302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c106740 0x7f2e1c10cfd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:33.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1858078302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e1c105d80 0x7f2e1c106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:33.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1858078302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c104b90 0x7f2e1c104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 -- 172.21.15.131:0/1858078302 >> 172.21.15.131:0/1858078302 conn(0x7f2e1c100120 msgr2=0x7f2e1c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:33.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 -- 172.21.15.131:0/1858078302 shutdown_connections 2024-04-26T20:08:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.115+0000 7f2e23e00640 1 -- 172.21.15.131:0/1858078302 wait complete. 2024-04-26T20:08:33.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.116+0000 7f2e23e00640 1 Processor -- start 2024-04-26T20:08:33.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e23e00640 1 -- start start 2024-04-26T20:08:33.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e23e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c104b90 0x7f2e1c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:33.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e23e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e1c105d80 0x7f2e1c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:33.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e23e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c106740 0x7f2e1c1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:33.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e23e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c110710 con 0x7f2e1c104b90 2024-04-26T20:08:33.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e23e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c110590 con 0x7f2e1c105d80 2024-04-26T20:08:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e23e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c110890 con 0x7f2e1c106740 2024-04-26T20:08:33.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e21400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c104b90 0x7f2e1c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e21400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c104b90 0x7f2e1c19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38124/0 (socket says 172.21.15.131:38124) 2024-04-26T20:08:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e21400640 1 -- 172.21.15.131:0/1529503191 learned_addr learned my addr 172.21.15.131:0/1529503191 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:33.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e20a00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e1c105d80 0x7f2e1c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.117+0000 7f2e21e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c106740 0x7f2e1c1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:34.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e21400640 1 -- 172.21.15.131:0/1529503191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c106740 msgr2=0x7f2e1c1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:34.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e21400640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c106740 0x7f2e1c1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:34.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e21400640 1 -- 172.21.15.131:0/1529503191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e1c105d80 msgr2=0x7f2e1c19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:34.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e21400640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e1c105d80 0x7f2e1c19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:34.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e21400640 1 -- 172.21.15.131:0/1529503191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c1a2980 con 0x7f2e1c104b90 2024-04-26T20:08:34.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e21e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c106740 0x7f2e1c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:08:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e21400640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c104b90 0x7f2e1c19ad60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000e9c0 tx=0x7f2e1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:34.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e1a000640 1 -- 172.21.15.131:0/1529503191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000cd80 con 0x7f2e1c104b90 2024-04-26T20:08:34.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e1a000640 1 -- 172.21.15.131:0/1529503191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e100035f0 con 0x7f2e1c104b90 2024-04-26T20:08:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c1a2c70 con 0x7f2e1c104b90 2024-04-26T20:08:34.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.118+0000 7f2e1a000640 1 -- 172.21.15.131:0/1529503191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10010690 con 0x7f2e1c104b90 2024-04-26T20:08:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.119+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c1a3170 con 0x7f2e1c104b90 2024-04-26T20:08:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.120+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e1c10beb0 con 0x7f2e1c104b90 2024-04-26T20:08:34.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.122+0000 7f2e1a000640 1 -- 172.21.15.131:0/1529503191 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2e10004000 con 0x7f2e1c104b90 2024-04-26T20:08:34.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.124+0000 7f2e1a000640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e040a3bb0 0x7f2e040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:34.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.124+0000 7f2e1a000640 1 -- 172.21.15.131:0/1529503191 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f2e100d0d60 con 0x7f2e1c104b90 2024-04-26T20:08:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.124+0000 7f2e20a00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e040a3bb0 0x7f2e040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:34.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.125+0000 7f2e20a00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e040a3bb0 0x7f2e040a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c19c270 tx=0x7f2e08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:34.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.129+0000 7f2e1a000640 1 -- 172.21.15.131:0/1529503191 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2e1009d380 con 0x7f2e1c104b90 2024-04-26T20:08:34.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.333+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2e1c02dd10 con 0x7f2e1c104b90 2024-04-26T20:08:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.334+0000 7f2e1a000640 1 -- 172.21.15.131:0/1529503191 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f2e1009cc80 con 0x7f2e1c104b90 2024-04-26T20:08:34.935 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:08:34.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e040a3bb0 msgr2=0x7f2e040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:35.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e040a3bb0 0x7f2e040a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c19c270 tx=0x7f2e08005f90 comp rx=0 tx=0).stop 2024-04-26T20:08:35.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c104b90 msgr2=0x7f2e1c19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:35.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c104b90 0x7f2e1c19ad60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000e9c0 tx=0x7f2e1000ee80 comp rx=0 tx=0).stop 2024-04-26T20:08:35.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 shutdown_connections 2024-04-26T20:08:35.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e040a3bb0 0x7f2e040a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:35.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e1c106740 0x7f2e1c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:35.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e1c105d80 0x7f2e1c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:35.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 --2- 172.21.15.131:0/1529503191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c104b90 0x7f2e1c19ad60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:35.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 >> 172.21.15.131:0/1529503191 conn(0x7f2e1c100120 msgr2=0x7f2e1c109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:35.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.337+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 shutdown_connections 2024-04-26T20:08:35.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:33.338+0000 7f2e23e00640 1 -- 172.21.15.131:0/1529503191 wait complete. 2024-04-26T20:08:35.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:08:35.526 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:33 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1529503191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:08:35.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:34 smithi131 ceph-mon[27919]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:35.593 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:33 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1529503191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:08:35.626 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:34 smithi148 ceph-mon[32281]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:35.672 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:33 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1529503191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:08:35.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:34 smithi157 ceph-mon[32703]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:35.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.812+0000 7f4c52e00640 1 -- 172.21.15.131:0/192611577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c1029e0 msgr2=0x7f4c4c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:35.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.812+0000 7f4c52e00640 1 --2- 172.21.15.131:0/192611577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c1029e0 0x7f4c4c102e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009ca0 tx=0x7f4c3c02a8b0 comp rx=0 tx=0).stop 2024-04-26T20:08:35.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.813+0000 7f4c52e00640 1 -- 172.21.15.131:0/192611577 shutdown_connections 2024-04-26T20:08:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.813+0000 7f4c52e00640 1 --2- 172.21.15.131:0/192611577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c104590 0x7f4c4c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:35.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.813+0000 7f4c52e00640 1 --2- 172.21.15.131:0/192611577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c4c103bd0 0x7f4c4c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.813+0000 7f4c52e00640 1 --2- 172.21.15.131:0/192611577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c1029e0 0x7f4c4c102e00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:36.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.813+0000 7f4c52e00640 1 -- 172.21.15.131:0/192611577 >> 172.21.15.131:0/192611577 conn(0x7f4c4c0fdf70 msgr2=0x7f4c4c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:36.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.813+0000 7f4c52e00640 1 -- 172.21.15.131:0/192611577 shutdown_connections 2024-04-26T20:08:36.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.813+0000 7f4c52e00640 1 -- 172.21.15.131:0/192611577 wait complete. 2024-04-26T20:08:36.168 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:35 smithi148 ceph-mon[32281]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:36.201 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:35 smithi157 ceph-mon[32703]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 Processor -- start 2024-04-26T20:08:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 -- start start 2024-04-26T20:08:36.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c4c1029e0 0x7f4c4c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c103bd0 0x7f4c4c074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c104590 0x7f4c4c075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:36.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10e4f0 con 0x7f4c4c1029e0 2024-04-26T20:08:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10e370 con 0x7f4c4c104590 2024-04-26T20:08:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.814+0000 7f4c52e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10e670 con 0x7f4c4c103bd0 2024-04-26T20:08:36.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c4be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c4c1029e0 0x7f4c4c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c50e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c104590 0x7f4c4c075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c4be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c4c1029e0 0x7f4c4c0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46460/0 (socket says 172.21.15.131:46460) 2024-04-26T20:08:36.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c4be00640 1 -- 172.21.15.131:0/2403257414 learned_addr learned my addr 172.21.15.131:0/2403257414 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:36.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c43400640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c103bd0 0x7f4c4c074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c50e00640 1 -- 172.21.15.131:0/2403257414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c103bd0 msgr2=0x7f4c4c074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:36.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c50e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c103bd0 0x7f4c4c074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c50e00640 1 -- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c4c1029e0 msgr2=0x7f4c4c0766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:36.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c50e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c4c1029e0 0x7f4c4c0766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:36.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c50e00640 1 -- 172.21.15.131:0/2403257414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c075af0 con 0x7f4c4c104590 2024-04-26T20:08:36.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.815+0000 7f4c43400640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c103bd0 0x7f4c4c074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:08:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.816+0000 7f4c50e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c104590 0x7f4c4c075260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400e970 tx=0x7f4c4400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.816+0000 7f4c49600640 1 -- 172.21.15.131:0/2403257414 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400ccc0 con 0x7f4c4c104590 2024-04-26T20:08:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.816+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c1a2cc0 con 0x7f4c4c104590 2024-04-26T20:08:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.816+0000 7f4c49600640 1 -- 172.21.15.131:0/2403257414 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c44005020 con 0x7f4c4c104590 2024-04-26T20:08:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.816+0000 7f4c49600640 1 -- 172.21.15.131:0/2403257414 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44004a00 con 0x7f4c4c104590 2024-04-26T20:08:37.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.816+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c1a31b0 con 0x7f4c4c104590 2024-04-26T20:08:37.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.817+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c109d70 con 0x7f4c4c104590 2024-04-26T20:08:37.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.819+0000 7f4c49600640 1 -- 172.21.15.131:0/2403257414 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4c44004ba0 con 0x7f4c4c104590 2024-04-26T20:08:37.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.820+0000 7f4c49600640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4c300a3bb0 0x7f4c300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:37.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.820+0000 7f4c49600640 1 -- 172.21.15.131:0/2403257414 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f4c440d07d0 con 0x7f4c4c104590 2024-04-26T20:08:37.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.821+0000 7f4c4be00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4c300a3bb0 0x7f4c300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:37.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.821+0000 7f4c4be00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4c300a3bb0 0x7f4c300a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c031040 tx=0x7f4c3c005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:37.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:35.825+0000 7f4c49600640 1 -- 172.21.15.131:0/2403257414 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4c4409cdf0 con 0x7f4c4c104590 2024-04-26T20:08:37.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:35 smithi131 ceph-mon[27919]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.043+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c4c063530 con 0x7f4c4c104590 2024-04-26T20:08:37.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.044+0000 7f4c49600640 1 -- 172.21.15.131:0/2403257414 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) ==== 81+0+601 (secure 0 0 0) 0x7f4c4409c6f0 con 0x7f4c4c104590 2024-04-26T20:08:37.719 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3514900702 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:37.752 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/3789992830 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:37.830 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/3789992830 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:37.902 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/2560715060 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/2579641682 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:37.994 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/2560715060 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:38.027 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3589087841 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:38.061 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3946098705 2024-04-27T19:59:13.443197+0000 2024-04-26T20:08:38.094 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1232073682 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:38.128 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/537078740 2024-04-27T19:58:38.942812+0000 2024-04-26T20:08:38.169 INFO:teuthology.orchestra.run.smithi131.stderr:listed 10 entries 2024-04-26T20:08:38.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.047+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4c300a3bb0 msgr2=0x7f4c300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:38.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.047+0000 7f4c52e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4c300a3bb0 0x7f4c300a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c031040 tx=0x7f4c3c005f70 comp rx=0 tx=0).stop 2024-04-26T20:08:38.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.047+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c104590 msgr2=0x7f4c4c075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:38.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.047+0000 7f4c52e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c104590 0x7f4c4c075260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400e970 tx=0x7f4c4400ee30 comp rx=0 tx=0).stop 2024-04-26T20:08:38.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 shutdown_connections 2024-04-26T20:08:38.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c104590 0x7f4c4c075260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:38.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4c300a3bb0 0x7f4c300a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:38.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c4c103bd0 0x7f4c4c074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:38.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 --2- 172.21.15.131:0/2403257414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c4c1029e0 0x7f4c4c0766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:38.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 >> 172.21.15.131:0/2403257414 conn(0x7f4c4c0fdf70 msgr2=0x7f4c4c106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:38.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 shutdown_connections 2024-04-26T20:08:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:36.048+0000 7f4c52e00640 1 -- 172.21.15.131:0/2403257414 wait complete. 2024-04-26T20:08:38.714 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-04-26T20:08:38.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:38.820 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:08:38.862 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:36 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2403257414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:08:38.925 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:37 smithi131 ceph-mon[27919]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:38.963 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:36 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2403257414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:08:38.995 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:37 smithi157 ceph-mon[32703]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:39.028 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:36 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2403257414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:08:39.078 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:37 smithi148 ceph-mon[32281]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:39.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[71348]: fuse finished with error 0 and tester_r 0 2024-04-26T20:08:39.147 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:08:39.244 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:08:39.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:39.332 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T20:08:39.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:08:39.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 -- 172.21.15.131:0/2525121870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 msgr2=0x7f4a80074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 --2- 172.21.15.131:0/2525121870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a80074d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009900 tx=0x7f4a7002a480 comp rx=0 tx=0).stop 2024-04-26T20:08:39.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 -- 172.21.15.131:0/2525121870 shutdown_connections 2024-04-26T20:08:39.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 --2- 172.21.15.131:0/2525121870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8010a820 0x7f4a8010ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:39.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 --2- 172.21.15.131:0/2525121870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a80075310 0x7f4a80075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 --2- 172.21.15.131:0/2525121870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a80074d40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:39.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 -- 172.21.15.131:0/2525121870 >> 172.21.15.131:0/2525121870 conn(0x7f4a800fdee0 msgr2=0x7f4a80100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 -- 172.21.15.131:0/2525121870 shutdown_connections 2024-04-26T20:08:39.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.699+0000 7f4a86800640 1 -- 172.21.15.131:0/2525121870 wait complete. 2024-04-26T20:08:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:39 smithi157 ceph-mon[32703]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:40.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.700+0000 7f4a86800640 1 Processor -- start 2024-04-26T20:08:40.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a86800640 1 -- start start 2024-04-26T20:08:40.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a86800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a80075310 0x7f4a8019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:40.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a86800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a8019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a86800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8010a820 0x7f4a801a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:40.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a86800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80110680 con 0x7f4a8010a820 2024-04-26T20:08:40.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a86800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80110500 con 0x7f4a800768f0 2024-04-26T20:08:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a86800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80110800 con 0x7f4a80075310 2024-04-26T20:08:40.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a7f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a80075310 0x7f4a8019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:40.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a7fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8010a820 0x7f4a801a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:40.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a7ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a8019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a7f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a80075310 0x7f4a8019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34818/0 (socket says 172.21.15.131:34818) 2024-04-26T20:08:40.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a7ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a8019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42654/0 (socket says 172.21.15.131:42654) 2024-04-26T20:08:40.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.701+0000 7f4a7f400640 1 -- 172.21.15.131:0/1351718280 learned_addr learned my addr 172.21.15.131:0/1351718280 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:40.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a7fe00640 1 -- 172.21.15.131:0/1351718280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a80075310 msgr2=0x7f4a8019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:40.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a7fe00640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a80075310 0x7f4a8019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a7fe00640 1 -- 172.21.15.131:0/1351718280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 msgr2=0x7f4a8019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:40.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a7fe00640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a8019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a7fe00640 1 -- 172.21.15.131:0/1351718280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a801a29b0 con 0x7f4a8010a820 2024-04-26T20:08:40.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a7ea00640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a8019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:08:40.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a7fe00640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8010a820 0x7f4a801a2310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009fd0 tx=0x7f4a7800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a77e00640 1 -- 172.21.15.131:0/1351718280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800ce20 con 0x7f4a8010a820 2024-04-26T20:08:41.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a801a2ca0 con 0x7f4a8010a820 2024-04-26T20:08:41.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.702+0000 7f4a77e00640 1 -- 172.21.15.131:0/1351718280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a78005020 con 0x7f4a8010a820 2024-04-26T20:08:41.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.703+0000 7f4a77e00640 1 -- 172.21.15.131:0/1351718280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78010690 con 0x7f4a8010a820 2024-04-26T20:08:41.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.703+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a801a31a0 con 0x7f4a8010a820 2024-04-26T20:08:41.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.704+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a801095e0 con 0x7f4a8010a820 2024-04-26T20:08:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.706+0000 7f4a77e00640 1 -- 172.21.15.131:0/1351718280 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4a780054d0 con 0x7f4a8010a820 2024-04-26T20:08:41.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.707+0000 7f4a77e00640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a640a3bb0 0x7f4a640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.707+0000 7f4a77e00640 1 -- 172.21.15.131:0/1351718280 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f4a780d0ec0 con 0x7f4a8010a820 2024-04-26T20:08:41.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.708+0000 7f4a7f400640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a640a3bb0 0x7f4a640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:41.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.708+0000 7f4a7f400640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a640a3bb0 0x7f4a640a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a8019c270 tx=0x7f4a700058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.716+0000 7f4a77e00640 1 -- 172.21.15.131:0/1351718280 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4a7809d4e0 con 0x7f4a8010a820 2024-04-26T20:08:41.452 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:39 smithi131 ceph-mon[27919]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:41.488 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:39 smithi148 ceph-mon[32281]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:41.521 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:40 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1351718280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:41.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.923+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a80063560 con 0x7f4a8010a820 2024-04-26T20:08:41.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.926+0000 7f4a77e00640 1 -- 172.21.15.131:0/1351718280 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24715 (secure 0 0 0) 0x7f4a7809cde0 con 0x7f4a8010a820 2024-04-26T20:08:41.670 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:08:41.706 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:08:15.562795+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:08:41.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a640a3bb0 msgr2=0x7f4a640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:41.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a640a3bb0 0x7f4a640a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a8019c270 tx=0x7f4a700058d0 comp rx=0 tx=0).stop 2024-04-26T20:08:41.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8010a820 msgr2=0x7f4a801a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:41.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8010a820 0x7f4a801a2310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009fd0 tx=0x7f4a7800ef10 comp rx=0 tx=0).stop 2024-04-26T20:08:41.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 shutdown_connections 2024-04-26T20:08:41.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8010a820 0x7f4a801a2310 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:42.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a800768f0 0x7f4a8019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a640a3bb0 0x7f4a640a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:42.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 --2- 172.21.15.131:0/1351718280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a80075310 0x7f4a8019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:42.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.929+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 >> 172.21.15.131:0/1351718280 conn(0x7f4a800fdee0 msgr2=0x7f4a801067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.930+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 shutdown_connections 2024-04-26T20:08:42.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:39.930+0000 7f4a86800640 1 -- 172.21.15.131:0/1351718280 wait complete. 2024-04-26T20:08:42.390 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:08:42.423 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:40 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1351718280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:42.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:41 smithi131 ceph-mon[27919]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:42.590 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:41 smithi148 ceph-mon[32281]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:42.622 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:40 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1351718280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:42.656 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:41 smithi157 ceph-mon[32703]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:08:42.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 -- 172.21.15.131:0/707587447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 msgr2=0x7f49641091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 --2- 172.21.15.131:0/707587447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f49641091b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4958009900 tx=0x7f495802a400 comp rx=0 tx=0).stop 2024-04-26T20:08:42.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 -- 172.21.15.131:0/707587447 shutdown_connections 2024-04-26T20:08:43.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 --2- 172.21.15.131:0/707587447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496410a940 0x7f49641111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:43.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 --2- 172.21.15.131:0/707587447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964109f80 0x7f496410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:43.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 --2- 172.21.15.131:0/707587447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f49641091b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:43.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 -- 172.21.15.131:0/707587447 >> 172.21.15.131:0/707587447 conn(0x7f4964077d30 msgr2=0x7f496407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:43.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 -- 172.21.15.131:0/707587447 shutdown_connections 2024-04-26T20:08:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.907+0000 7f496ac00640 1 -- 172.21.15.131:0/707587447 wait complete. 2024-04-26T20:08:43.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.908+0000 7f496ac00640 1 Processor -- start 2024-04-26T20:08:43.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.908+0000 7f496ac00640 1 -- start start 2024-04-26T20:08:43.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f496419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:43.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496ac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964109f80 0x7f496419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:43.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496410a940 0x7f49641a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:43.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4964114890 con 0x7f4964108d90 2024-04-26T20:08:43.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496ac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4964114710 con 0x7f4964109f80 2024-04-26T20:08:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4964114a10 con 0x7f496410a940 2024-04-26T20:08:43.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f496419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:43.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f496419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46504/0 (socket says 172.21.15.131:46504) 2024-04-26T20:08:43.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 -- 172.21.15.131:0/773921654 learned_addr learned my addr 172.21.15.131:0/773921654 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:43.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4968e00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964109f80 0x7f496419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496a200640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496410a940 0x7f49641a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 -- 172.21.15.131:0/773921654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496410a940 msgr2=0x7f49641a6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:43.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496410a940 0x7f49641a6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:43.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 -- 172.21.15.131:0/773921654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964109f80 msgr2=0x7f496419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:43.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964109f80 0x7f496419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:43.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f4969800640 1 -- 172.21.15.131:0/773921654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49641a6bc0 con 0x7f4964108d90 2024-04-26T20:08:43.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.909+0000 7f496a200640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496410a940 0x7f49641a6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:08:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.910+0000 7f4969800640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f496419f040 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4958006010 tx=0x7f49580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.910+0000 7f4962000640 1 -- 172.21.15.131:0/773921654 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495803f030 con 0x7f4964108d90 2024-04-26T20:08:44.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.910+0000 7f4962000640 1 -- 172.21.15.131:0/773921654 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4958003cb0 con 0x7f4964108d90 2024-04-26T20:08:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.910+0000 7f4962000640 1 -- 172.21.15.131:0/773921654 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958035070 con 0x7f4964108d90 2024-04-26T20:08:44.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.910+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49641a6e50 con 0x7f4964108d90 2024-04-26T20:08:44.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.910+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641a7260 con 0x7f4964108d90 2024-04-26T20:08:44.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.911+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4964110180 con 0x7f4964108d90 2024-04-26T20:08:44.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.913+0000 7f4962000640 1 -- 172.21.15.131:0/773921654 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4958042030 con 0x7f4964108d90 2024-04-26T20:08:44.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.914+0000 7f4962000640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f494c0a3bb0 0x7f494c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:44.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.915+0000 7f4962000640 1 -- 172.21.15.131:0/773921654 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f49580f0e00 con 0x7f4964108d90 2024-04-26T20:08:44.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.915+0000 7f4968e00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f494c0a3bb0 0x7f494c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.916+0000 7f4968e00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f494c0a3bb0 0x7f494c0a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49641a04b0 tx=0x7f495000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:44.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:42.919+0000 7f4962000640 1 -- 172.21.15.131:0/773921654 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f49580bd420 con 0x7f4964108d90 2024-04-26T20:08:44.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.124+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4964063680 con 0x7f4964108d90 2024-04-26T20:08:44.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.128+0000 7f4962000640 1 -- 172.21.15.131:0/773921654 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24715 (secure 0 0 0) 0x7f49580bcd20 con 0x7f4964108d90 2024-04-26T20:08:44.701 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:08:44.732 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:08:15.562795+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:08:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f494c0a3bb0 msgr2=0x7f494c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:44.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f494c0a3bb0 0x7f494c0a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49641a04b0 tx=0x7f495000b390 comp rx=0 tx=0).stop 2024-04-26T20:08:44.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 msgr2=0x7f496419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:44.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f496419f040 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4958006010 tx=0x7f49580042f0 comp rx=0 tx=0).stop 2024-04-26T20:08:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 shutdown_connections 2024-04-26T20:08:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f494c0a3bb0 0x7f494c0a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496410a940 0x7f49641a6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964109f80 0x7f496419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:45.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 --2- 172.21.15.131:0/773921654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4964108d90 0x7f496419f040 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:45.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.131+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 >> 172.21.15.131:0/773921654 conn(0x7f4964077d30 msgr2=0x7f496410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:45.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.132+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 shutdown_connections 2024-04-26T20:08:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:43.132+0000 7f496ac00640 1 -- 172.21.15.131:0/773921654 wait complete. 2024-04-26T20:08:45.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:08:45.300 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:43 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/773921654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:45.350 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:44 smithi131 ceph-mon[27919]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:45.409 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:43 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/773921654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:45.441 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:44 smithi148 ceph-mon[32281]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:45.475 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:43 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/773921654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:08:45.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:44 smithi157 ceph-mon[32703]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:45.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 -- 172.21.15.131:0/3899837533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e0101a10 msgr2=0x7f34e0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:45.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 --2- 172.21.15.131:0/3899837533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e0101a10 0x7f34e0101e70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009860 tx=0x7f34cc02a340 comp rx=0 tx=0).stop 2024-04-26T20:08:45.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 -- 172.21.15.131:0/3899837533 shutdown_connections 2024-04-26T20:08:45.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 --2- 172.21.15.131:0/3899837533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e01023b0 0x7f34e0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 --2- 172.21.15.131:0/3899837533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e0101a10 0x7f34e0101e70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 --2- 172.21.15.131:0/3899837533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01010b0 0x7f34e01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:45.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 -- 172.21.15.131:0/3899837533 >> 172.21.15.131:0/3899837533 conn(0x7f34e00fccd0 msgr2=0x7f34e00ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 -- 172.21.15.131:0/3899837533 shutdown_connections 2024-04-26T20:08:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.625+0000 7f34e5600640 1 -- 172.21.15.131:0/3899837533 wait complete. 2024-04-26T20:08:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.626+0000 7f34e5600640 1 Processor -- start 2024-04-26T20:08:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e5600640 1 -- start start 2024-04-26T20:08:46.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e5600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e01010b0 0x7f34e019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:46.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0101a10 0x7f34e019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:46.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e5600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01023b0 0x7f34e01a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:46.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e5600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0110530 con 0x7f34e01023b0 2024-04-26T20:08:46.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e5600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01103b0 con 0x7f34e01010b0 2024-04-26T20:08:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01106b0 con 0x7f34e0101a10 2024-04-26T20:08:46.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01023b0 0x7f34e01a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:46.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e01010b0 0x7f34e019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:46.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01023b0 0x7f34e01a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43244/0 (socket says 172.21.15.131:43244) 2024-04-26T20:08:46.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.627+0000 7f34e4c00640 1 -- 172.21.15.131:0/2929022873 learned_addr learned my addr 172.21.15.131:0/2929022873 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:46.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34df400640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0101a10 0x7f34e019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:46.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34e4c00640 1 -- 172.21.15.131:0/2929022873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0101a10 msgr2=0x7f34e019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:46.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34e4c00640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0101a10 0x7f34e019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:46.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34e4c00640 1 -- 172.21.15.131:0/2929022873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e01010b0 msgr2=0x7f34e019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:46.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34e4c00640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e01010b0 0x7f34e019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34e4c00640 1 -- 172.21.15.131:0/2929022873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e01a2860 con 0x7f34e01023b0 2024-04-26T20:08:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34e4c00640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01023b0 0x7f34e01a21c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009fd0 tx=0x7f34d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.628+0000 7f34dcc00640 1 -- 172.21.15.131:0/2929022873 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8004b20 con 0x7f34e01023b0 2024-04-26T20:08:46.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.629+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e01a2b50 con 0x7f34e01023b0 2024-04-26T20:08:46.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.629+0000 7f34dcc00640 1 -- 172.21.15.131:0/2929022873 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d8003e20 con 0x7f34e01023b0 2024-04-26T20:08:46.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.629+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a30a0 con 0x7f34e01023b0 2024-04-26T20:08:47.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.629+0000 7f34dcc00640 1 -- 172.21.15.131:0/2929022873 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d801a020 con 0x7f34e01023b0 2024-04-26T20:08:47.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.630+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e010bfd0 con 0x7f34e01023b0 2024-04-26T20:08:47.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.632+0000 7f34dcc00640 1 -- 172.21.15.131:0/2929022873 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f34d801d020 con 0x7f34e01023b0 2024-04-26T20:08:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.634+0000 7f34dcc00640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34c40a3bb0 0x7f34c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:47.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.634+0000 7f34dcc00640 1 -- 172.21.15.131:0/2929022873 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f34d80d07c0 con 0x7f34e01023b0 2024-04-26T20:08:47.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.634+0000 7f34dfe00640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34c40a3bb0 0x7f34c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.635+0000 7f34dfe00640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34c40a3bb0 0x7f34c40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f34e019c120 tx=0x7f34d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.638+0000 7f34dcc00640 1 -- 172.21.15.131:0/2929022873 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f34d809cde0 con 0x7f34e01023b0 2024-04-26T20:08:47.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:45 smithi131 ceph-mon[27919]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:47.344 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:45 smithi148 ceph-mon[32281]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:47.404 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:46 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2929022873' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T20:08:47.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:45 smithi157 ceph-mon[32703]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:47.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:46 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2929022873' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T20:08:47.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:45.889+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f34e0063530 con 0x7f34e01023b0 2024-04-26T20:08:47.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.693+0000 7f34dcc00640 1 -- 172.21.15.131:0/2929022873 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f34d80047e0 con 0x7f34e01023b0 2024-04-26T20:08:47.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34c40a3bb0 msgr2=0x7f34c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:47.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34c40a3bb0 0x7f34c40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f34e019c120 tx=0x7f34d0005f90 comp rx=0 tx=0).stop 2024-04-26T20:08:47.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01023b0 msgr2=0x7f34e01a21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01023b0 0x7f34e01a21c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009fd0 tx=0x7f34d800c800 comp rx=0 tx=0).stop 2024-04-26T20:08:47.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 shutdown_connections 2024-04-26T20:08:47.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e01023b0 0x7f34e01a21c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:47.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34c40a3bb0 0x7f34c40a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:47.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0101a10 0x7f34e019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:47.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 --2- 172.21.15.131:0/2929022873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e01010b0 0x7f34e019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:48.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.697+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 >> 172.21.15.131:0/2929022873 conn(0x7f34e00fccd0 msgr2=0x7f34e00fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:48.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.698+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 shutdown_connections 2024-04-26T20:08:48.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:46.698+0000 7f34e5600640 1 -- 172.21.15.131:0/2929022873 wait complete. 2024-04-26T20:08:48.144 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-26T20:08:48.176 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-26T20:08:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:46 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2929022873' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T20:08:48.243 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:47 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2929022873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T20:08:48.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:47 smithi131 ceph-mon[27919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:08:48.356 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:47 smithi131 ceph-mon[27919]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:48.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:47 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2929022873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T20:08:48.427 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:47 smithi148 ceph-mon[32281]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:08:48.527 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:47 smithi148 ceph-mon[32281]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:48.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:47 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2929022873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T20:08:48.651 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:47 smithi157 ceph-mon[32703]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:08:48.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:47 smithi157 ceph-mon[32703]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-26T20:08:48.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.607+0000 7f9eb4a00640 1 -- 172.21.15.131:0/1207063073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0105c60 msgr2=0x7f9eb01060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:48.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.607+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/1207063073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0105c60 0x7f9eb01060e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009900 tx=0x7f9e9802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:08:48.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.607+0000 7f9eb4a00640 1 -- 172.21.15.131:0/1207063073 shutdown_connections 2024-04-26T20:08:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.607+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/1207063073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb0106620 0x7f9eb010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:48.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.607+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/1207063073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0105c60 0x7f9eb01060e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:49.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.608+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/1207063073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0104a70 0x7f9eb0104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:49.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.608+0000 7f9eb4a00640 1 -- 172.21.15.131:0/1207063073 >> 172.21.15.131:0/1207063073 conn(0x7f9eb00fffe0 msgr2=0x7f9eb0102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.608+0000 7f9eb4a00640 1 -- 172.21.15.131:0/1207063073 shutdown_connections 2024-04-26T20:08:49.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.608+0000 7f9eb4a00640 1 -- 172.21.15.131:0/1207063073 wait complete. 2024-04-26T20:08:49.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.609+0000 7f9eb4a00640 1 Processor -- start 2024-04-26T20:08:49.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.609+0000 7f9eb4a00640 1 -- start start 2024-04-26T20:08:49.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.609+0000 7f9eb4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0104a70 0x7f9eb019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:49.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.609+0000 7f9eb4a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb0105c60 0x7f9eb019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:49.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eb4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0106620 0x7f9eb01a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:49.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eb4a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01105f0 con 0x7f9eb0106620 2024-04-26T20:08:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eb4a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0110470 con 0x7f9eb0105c60 2024-04-26T20:08:49.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eb4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0110770 con 0x7f9eb0104a70 2024-04-26T20:08:49.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0104a70 0x7f9eb019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:49.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0104a70 0x7f9eb019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:39032/0 (socket says 172.21.15.131:39032) 2024-04-26T20:08:49.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 -- 172.21.15.131:0/3972441198 learned_addr learned my addr 172.21.15.131:0/3972441198 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9ea5e00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb0105c60 0x7f9eb019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:49.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eaf200640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0106620 0x7f9eb01a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:49.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 -- 172.21.15.131:0/3972441198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb0105c60 msgr2=0x7f9eb019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:49.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb0105c60 0x7f9eb019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:49.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 -- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0106620 msgr2=0x7f9eb01a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:49.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0106620 0x7f9eb01a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.610+0000 7f9eae800640 1 -- 172.21.15.131:0/3972441198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb01a28b0 con 0x7f9eb0104a70 2024-04-26T20:08:50.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.611+0000 7f9eae800640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0104a70 0x7f9eb019ac70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e9c0 tx=0x7f9ea000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.611+0000 7f9ea7e00640 1 -- 172.21.15.131:0/3972441198 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000cd80 con 0x7f9eb0104a70 2024-04-26T20:08:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.611+0000 7f9ea7e00640 1 -- 172.21.15.131:0/3972441198 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea00035f0 con 0x7f9eb0104a70 2024-04-26T20:08:50.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.611+0000 7f9ea7e00640 1 -- 172.21.15.131:0/3972441198 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0010690 con 0x7f9eb0104a70 2024-04-26T20:08:50.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.611+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb01a2ba0 con 0x7f9eb0104a70 2024-04-26T20:08:50.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.611+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01a3070 con 0x7f9eb0104a70 2024-04-26T20:08:50.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.612+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb010bdb0 con 0x7f9eb0104a70 2024-04-26T20:08:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.614+0000 7f9ea7e00640 1 -- 172.21.15.131:0/3972441198 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9ea00037a0 con 0x7f9eb0104a70 2024-04-26T20:08:50.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.615+0000 7f9ea7e00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9e940a3bb0 0x7f9e940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:50.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.616+0000 7f9ea7e00640 1 -- 172.21.15.131:0/3972441198 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f9ea00d0bb0 con 0x7f9eb0104a70 2024-04-26T20:08:50.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.616+0000 7f9ea5e00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9e940a3bb0 0x7f9e940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:50.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.617+0000 7f9ea5e00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9e940a3bb0 0x7f9e940a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9e98031500 tx=0x7f9e98005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:50.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.620+0000 7f9ea7e00640 1 -- 172.21.15.131:0/3972441198 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9ea009d1d0 con 0x7f9eb0104a70 2024-04-26T20:08:50.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:48 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3972441198' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T20:08:50.653 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:48 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T20:08:50.688 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:49 smithi148 ceph-mon[32281]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T20:08:50.720 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:49 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T20:08:50.753 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:49 smithi148 ceph-mon[32281]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T20:08:50.787 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:49 smithi148 ceph-mon[32281]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:08:50.820 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:49 smithi148 ceph-mon[32281]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-26T20:08:50.853 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:49 smithi148 ceph-mon[32281]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T20:08:50.887 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:49 smithi148 ceph-mon[32281]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-26T20:08:50.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:48 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3972441198' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T20:08:51.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:48 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T20:08:51.083 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:49 smithi157 ceph-mon[32703]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T20:08:51.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:49 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T20:08:51.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:49 smithi157 ceph-mon[32703]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T20:08:51.262 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:49 smithi157 ceph-mon[32703]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:08:51.295 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:49 smithi157 ceph-mon[32703]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-26T20:08:51.329 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:49 smithi157 ceph-mon[32703]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T20:08:51.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:49 smithi157 ceph-mon[32703]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-26T20:08:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.862+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f9eb0063560 con 0x7f9eb0104a70 2024-04-26T20:08:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.920+0000 7f9ea7e00640 1 -- 172.21.15.131:0/3972441198 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f9ea009cad0 con 0x7f9eb0104a70 2024-04-26T20:08:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9e940a3bb0 msgr2=0x7f9e940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:51.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9e940a3bb0 0x7f9e940a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9e98031500 tx=0x7f9e98005a20 comp rx=0 tx=0).stop 2024-04-26T20:08:51.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0104a70 msgr2=0x7f9eb019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:51.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0104a70 0x7f9eb019ac70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e9c0 tx=0x7f9ea000ee80 comp rx=0 tx=0).stop 2024-04-26T20:08:51.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 shutdown_connections 2024-04-26T20:08:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eb0106620 0x7f9eb01a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:51.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb0105c60 0x7f9eb019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:51.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9e940a3bb0 0x7f9e940a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:51.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 --2- 172.21.15.131:0/3972441198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb0104a70 0x7f9eb019ac70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:51.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.924+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 >> 172.21.15.131:0/3972441198 conn(0x7f9eb00fffe0 msgr2=0x7f9eb0108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:51.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.925+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 shutdown_connections 2024-04-26T20:08:51.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:48.925+0000 7f9eb4a00640 1 -- 172.21.15.131:0/3972441198 wait complete. 2024-04-26T20:08:52.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T20:08:52.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:48 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3972441198' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T20:08:52.104 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:48 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T20:08:52.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:49 smithi131 ceph-mon[27919]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T20:08:52.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:49 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T20:08:52.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:49 smithi131 ceph-mon[27919]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T20:08:52.249 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:49 smithi131 ceph-mon[27919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T20:08:52.296 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:49 smithi131 ceph-mon[27919]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-26T20:08:52.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:49 smithi131 ceph-mon[27919]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T20:08:52.363 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:49 smithi131 ceph-mon[27919]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-26T20:08:52.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:50 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] up:active 2024-04-26T20:08:52.456 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:50 smithi131 ceph-mon[27919]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T20:08:52.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:50 smithi131 ceph-mon[27919]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T20:08:52.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:50 smithi131 ceph-mon[27919]: Cluster is now healthy 2024-04-26T20:08:52.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:50 smithi131 ceph-mon[27919]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-26T20:08:52.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:50 smithi131 ceph-mon[27919]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-26T20:08:52.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:50 smithi131 ceph-mon[27919]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T20:08:52.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:51 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] up:active 2024-04-26T20:08:52.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:51 smithi131 ceph-mon[27919]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-26T20:08:52.780 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:51 smithi131 ceph-mon[27919]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-26T20:08:52.813 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:50 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] up:active 2024-04-26T20:08:52.846 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:50 smithi148 ceph-mon[32281]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T20:08:52.880 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:50 smithi148 ceph-mon[32281]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T20:08:52.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:50 smithi148 ceph-mon[32281]: Cluster is now healthy 2024-04-26T20:08:52.960 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:50 smithi148 ceph-mon[32281]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-26T20:08:52.996 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:50 smithi148 ceph-mon[32281]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-26T20:08:53.030 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:50 smithi148 ceph-mon[32281]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T20:08:53.064 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:51 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] up:active 2024-04-26T20:08:53.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:51 smithi148 ceph-mon[32281]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-26T20:08:53.180 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:51 smithi148 ceph-mon[32281]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-26T20:08:53.214 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:50 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] up:active 2024-04-26T20:08:53.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:50 smithi157 ceph-mon[32703]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T20:08:53.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:50 smithi157 ceph-mon[32703]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T20:08:53.339 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:50 smithi157 ceph-mon[32703]: Cluster is now healthy 2024-04-26T20:08:53.372 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:50 smithi157 ceph-mon[32703]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-26T20:08:53.409 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:50 smithi157 ceph-mon[32703]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-26T20:08:53.479 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:50 smithi157 ceph-mon[32703]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T20:08:53.514 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:51 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] up:active 2024-04-26T20:08:53.589 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:51 smithi157 ceph-mon[32703]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-26T20:08:53.681 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:51 smithi157 ceph-mon[32703]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-26T20:08:53.715 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:08:52 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b[63000]: 2024-04-26T20:08:52.673+0000 7f71fa892700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T20:08:53.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:53 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/140437608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T20:08:53.784 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:53 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/140437608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T20:08:53.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 -- 172.21.15.131:0/623163523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 msgr2=0x7fc3ac104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:53.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 --2- 172.21.15.131:0/623163523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac104fb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009900 tx=0x7fc39c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:08:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 -- 172.21.15.131:0/623163523 shutdown_connections 2024-04-26T20:08:53.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 --2- 172.21.15.131:0/623163523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ac106740 0x7fc3ac10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 --2- 172.21.15.131:0/623163523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac105d80 0x7fc3ac106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 --2- 172.21.15.131:0/623163523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac104fb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:54.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 -- 172.21.15.131:0/623163523 >> 172.21.15.131:0/623163523 conn(0x7fc3ac1000e0 msgr2=0x7fc3ac102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:54.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 -- 172.21.15.131:0/623163523 shutdown_connections 2024-04-26T20:08:54.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.928+0000 7fc3b1000640 1 -- 172.21.15.131:0/623163523 wait complete. 2024-04-26T20:08:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 Processor -- start 2024-04-26T20:08:54.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 -- start start 2024-04-26T20:08:54.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:54.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106740 0x7fc3ac1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:54.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1106b0 con 0x7fc3ac104b90 2024-04-26T20:08:54.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac110530 con 0x7fc3ac105d80 2024-04-26T20:08:54.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.929+0000 7fc3b1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac110830 con 0x7fc3ac106740 2024-04-26T20:08:54.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:54.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43296/0 (socket says 172.21.15.131:43296) 2024-04-26T20:08:54.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 -- 172.21.15.131:0/140437608 learned_addr learned my addr 172.21.15.131:0/140437608 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a3e00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3aa800640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106740 0x7fc3ac1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 -- 172.21.15.131:0/140437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106740 msgr2=0x7fc3ac1a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106740 0x7fc3ac1a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 -- 172.21.15.131:0/140437608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ac105d80 msgr2=0x7fc3ac19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:54.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:54.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 -- 172.21.15.131:0/140437608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1a2a30 con 0x7fc3ac104b90 2024-04-26T20:08:54.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a3e00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:08:54.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.930+0000 7fc3a9e00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac19ae10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc39c006010 tx=0x7fc39c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:55.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.931+0000 7fc3a2a00640 1 -- 172.21.15.131:0/140437608 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c03f030 con 0x7fc3ac104b90 2024-04-26T20:08:55.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.931+0000 7fc3a2a00640 1 -- 172.21.15.131:0/140437608 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc39c031910 con 0x7fc3ac104b90 2024-04-26T20:08:55.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.931+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1a2cc0 con 0x7fc3ac104b90 2024-04-26T20:08:55.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.931+0000 7fc3a2a00640 1 -- 172.21.15.131:0/140437608 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c035070 con 0x7fc3ac104b90 2024-04-26T20:08:55.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.931+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1a31b0 con 0x7fc3ac104b90 2024-04-26T20:08:55.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.932+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac10c020 con 0x7fc3ac104b90 2024-04-26T20:08:55.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.934+0000 7fc3a2a00640 1 -- 172.21.15.131:0/140437608 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc39c042030 con 0x7fc3ac104b90 2024-04-26T20:08:55.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.936+0000 7fc3a2a00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3900a3c30 0x7fc3900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:55.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.937+0000 7fc3a2a00640 1 -- 172.21.15.131:0/140437608 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fc39c0f0fe0 con 0x7fc3ac104b90 2024-04-26T20:08:55.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.937+0000 7fc3a3e00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3900a3c30 0x7fc3900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:55.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.938+0000 7fc3a3e00640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3900a3c30 0x7fc3900a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac19c320 tx=0x7fc394005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:52.940+0000 7fc3a2a00640 1 -- 172.21.15.131:0/140437608 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc39c0bcff0 con 0x7fc3ac104b90 2024-04-26T20:08:55.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.186+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc3ac02dcd0 con 0x7fc3ac104b90 2024-04-26T20:08:55.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.240+0000 7fc3a2a00640 1 -- 172.21.15.131:0/140437608 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fc39c0c12f0 con 0x7fc3ac104b90 2024-04-26T20:08:55.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.243+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3900a3c30 msgr2=0x7fc3900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:55.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.243+0000 7fc3b1000640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3900a3c30 0x7fc3900a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac19c320 tx=0x7fc394005f90 comp rx=0 tx=0).stop 2024-04-26T20:08:55.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.243+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 msgr2=0x7fc3ac19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:55.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.243+0000 7fc3b1000640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac19ae10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc39c006010 tx=0x7fc39c0042f0 comp rx=0 tx=0).stop 2024-04-26T20:08:55.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 shutdown_connections 2024-04-26T20:08:55.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3900a3c30 0x7fc3900a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:56.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106740 0x7fc3ac1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:56.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 --2- 172.21.15.131:0/140437608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3ac104b90 0x7fc3ac19ae10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:56.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 >> 172.21.15.131:0/140437608 conn(0x7fc3ac1000e0 msgr2=0x7fc3ac1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:56.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 shutdown_connections 2024-04-26T20:08:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:53.244+0000 7fc3b1000640 1 -- 172.21.15.131:0/140437608 wait complete. 2024-04-26T20:08:56.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T20:08:56.441 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:53 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/140437608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T20:08:56.474 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:54 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/140437608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T20:08:56.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:54 smithi131 ceph-mon[27919]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-26T20:08:56.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:54 smithi131 ceph-mon[27919]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:08:56.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:54 smithi131 ceph-mon[27919]: pgmap v375: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T20:08:56.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:55 smithi131 ceph-mon[27919]: pgmap v376: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T20:08:56.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:54 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/140437608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T20:08:56.750 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:54 smithi157 ceph-mon[32703]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-26T20:08:56.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:54 smithi157 ceph-mon[32703]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:08:56.833 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:54 smithi157 ceph-mon[32703]: pgmap v375: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T20:08:56.883 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:55 smithi157 ceph-mon[32703]: pgmap v376: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T20:08:56.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:54 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/140437608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T20:08:56.949 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:54 smithi148 ceph-mon[32281]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-26T20:08:57.029 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:54 smithi148 ceph-mon[32281]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:08:57.102 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:54 smithi148 ceph-mon[32281]: pgmap v375: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T20:08:57.133 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:55 smithi148 ceph-mon[32281]: pgmap v376: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T20:08:57.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 -- 172.21.15.131:0/269370242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58103a50 msgr2=0x7fed58103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:57.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 --2- 172.21.15.131:0/269370242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58103a50 0x7fed58103ed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fed40009900 tx=0x7fed4002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:08:57.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 -- 172.21.15.131:0/269370242 shutdown_connections 2024-04-26T20:08:57.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 --2- 172.21.15.131:0/269370242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed58104410 0x7fed5810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:57.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 --2- 172.21.15.131:0/269370242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58103a50 0x7fed58103ed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:57.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 --2- 172.21.15.131:0/269370242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed58102860 0x7fed58102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 -- 172.21.15.131:0/269370242 >> 172.21.15.131:0/269370242 conn(0x7fed580fddd0 msgr2=0x7fed58100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 -- 172.21.15.131:0/269370242 shutdown_connections 2024-04-26T20:08:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.932+0000 7fed5cc00640 1 -- 172.21.15.131:0/269370242 wait complete. 2024-04-26T20:08:57.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 Processor -- start 2024-04-26T20:08:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 -- start start 2024-04-26T20:08:57.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58102860 0x7fed5819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:57.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed58103a50 0x7fed5819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:57.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed58104410 0x7fed581a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:57.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810e3c0 con 0x7fed58102860 2024-04-26T20:08:57.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810e240 con 0x7fed58104410 2024-04-26T20:08:57.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.933+0000 7fed5cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810e540 con 0x7fed58103a50 2024-04-26T20:08:57.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58102860 0x7fed5819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:58.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58102860 0x7fed5819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41052/0 (socket says 172.21.15.131:41052) 2024-04-26T20:08:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 -- 172.21.15.131:0/1494039115 learned_addr learned my addr 172.21.15.131:0/1494039115 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:08:58.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed4ea00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed58103a50 0x7fed5819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:58.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57e00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed58104410 0x7fed581a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 -- 172.21.15.131:0/1494039115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed58103a50 msgr2=0x7fed5819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed58103a50 0x7fed5819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:58.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 -- 172.21.15.131:0/1494039115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed58104410 msgr2=0x7fed581a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:58.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed58104410 0x7fed581a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:58.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 -- 172.21.15.131:0/1494039115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed581a2880 con 0x7fed58102860 2024-04-26T20:08:58.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.934+0000 7fed57400640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58102860 0x7fed5819ac70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed4800ebc0 tx=0x7fed4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:58.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.935+0000 7fed54c00640 1 -- 172.21.15.131:0/1494039115 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48003b70 con 0x7fed58102860 2024-04-26T20:08:58.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.935+0000 7fed54c00640 1 -- 172.21.15.131:0/1494039115 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed48003d10 con 0x7fed58102860 2024-04-26T20:08:58.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.935+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed581a2b70 con 0x7fed58102860 2024-04-26T20:08:58.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.935+0000 7fed54c00640 1 -- 172.21.15.131:0/1494039115 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48019800 con 0x7fed58102860 2024-04-26T20:08:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.935+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed581a30c0 con 0x7fed58102860 2024-04-26T20:08:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.936+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed581a3290 con 0x7fed58102860 2024-04-26T20:08:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.938+0000 7fed54c00640 1 -- 172.21.15.131:0/1494039115 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fed480199a0 con 0x7fed58102860 2024-04-26T20:08:58.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.939+0000 7fed54c00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed3c0a3bb0 0x7fed3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:08:58.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.939+0000 7fed54c00640 1 -- 172.21.15.131:0/1494039115 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fed480d10d0 con 0x7fed58102860 2024-04-26T20:08:58.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.939+0000 7fed4ea00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed3c0a3bb0 0x7fed3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:08:58.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.940+0000 7fed4ea00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed3c0a3bb0 0x7fed3c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fed40031500 tx=0x7fed40005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:08:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:56.944+0000 7fed54c00640 1 -- 172.21.15.131:0/1494039115 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fed4809d6f0 con 0x7fed58102860 2024-04-26T20:08:58.852 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:57 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1494039115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T20:08:58.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:58 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1494039115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T20:08:58.968 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:58 smithi157 ceph-mon[32703]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:08:59.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:08:58 smithi157 ceph-mon[32703]: pgmap v377: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T20:08:59.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:57 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1494039115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T20:08:59.076 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:58 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1494039115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T20:08:59.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:58 smithi148 ceph-mon[32281]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:08:59.157 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:08:58 smithi148 ceph-mon[32281]: pgmap v377: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T20:08:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.195+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fed581a3460 con 0x7fed58102860 2024-04-26T20:08:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.248+0000 7fed54c00640 1 -- 172.21.15.131:0/1494039115 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fed480bd020 con 0x7fed58102860 2024-04-26T20:08:59.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed3c0a3bb0 msgr2=0x7fed3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:59.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed3c0a3bb0 0x7fed3c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fed40031500 tx=0x7fed40005a20 comp rx=0 tx=0).stop 2024-04-26T20:08:59.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58102860 msgr2=0x7fed5819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:08:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58102860 0x7fed5819ac70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed4800ebc0 tx=0x7fed4800c6a0 comp rx=0 tx=0).stop 2024-04-26T20:08:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 shutdown_connections 2024-04-26T20:08:59.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed58104410 0x7fed581a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fed3c0a3bb0 0x7fed3c0a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed58103a50 0x7fed5819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 --2- 172.21.15.131:0/1494039115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed58102860 0x7fed5819ac70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:08:59.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 >> 172.21.15.131:0/1494039115 conn(0x7fed580fddd0 msgr2=0x7fed58109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:08:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 shutdown_connections 2024-04-26T20:08:59.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:08:57.253+0000 7fed5cc00640 1 -- 172.21.15.131:0/1494039115 wait complete. 2024-04-26T20:08:59.785 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-26T20:08:59.818 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T20:08:59.852 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:57 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1494039115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T20:08:59.885 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:58 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1494039115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T20:08:59.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:58 smithi131 ceph-mon[27919]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:08:59.994 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:08:58 smithi131 ceph-mon[27919]: pgmap v377: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T20:09:00.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 -- 172.21.15.131:0/7679263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a38075310 msgr2=0x7f6a38075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:00.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 --2- 172.21.15.131:0/7679263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a38075310 0x7f6a38075770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009900 tx=0x7f6a2002a480 comp rx=0 tx=0).stop 2024-04-26T20:09:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 -- 172.21.15.131:0/7679263 shutdown_connections 2024-04-26T20:09:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 --2- 172.21.15.131:0/7679263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a3810a710 0x7f6a3810cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:00.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 --2- 172.21.15.131:0/7679263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a38075310 0x7f6a38075770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 --2- 172.21.15.131:0/7679263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a380768f0 0x7f6a38074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 -- 172.21.15.131:0/7679263 >> 172.21.15.131:0/7679263 conn(0x7f6a380fddd0 msgr2=0x7f6a38100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:00.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 -- 172.21.15.131:0/7679263 shutdown_connections 2024-04-26T20:09:00.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.234+0000 7f6a3d600640 1 -- 172.21.15.131:0/7679263 wait complete. 2024-04-26T20:09:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.235+0000 7f6a3d600640 1 Processor -- start 2024-04-26T20:09:00.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3d600640 1 -- start start 2024-04-26T20:09:00.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a38075310 0x7f6a3819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:00.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a380768f0 0x7f6a3819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:00.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3810a710 0x7f6a381a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:00.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381105e0 con 0x7f6a380768f0 2024-04-26T20:09:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38110460 con 0x7f6a38075310 2024-04-26T20:09:00.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38110760 con 0x7f6a3810a710 2024-04-26T20:09:01.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3810a710 0x7f6a381a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:01.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a2fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a380768f0 0x7f6a3819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:01.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3810a710 0x7f6a381a2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35042/0 (socket says 172.21.15.131:35042) 2024-04-26T20:09:01.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a3cc00640 1 -- 172.21.15.131:0/462372103 learned_addr learned my addr 172.21.15.131:0/462372103 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:01.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.236+0000 7f6a37e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a38075310 0x7f6a3819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:01.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3cc00640 1 -- 172.21.15.131:0/462372103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a38075310 msgr2=0x7f6a3819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3cc00640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a38075310 0x7f6a3819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:01.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3cc00640 1 -- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a380768f0 msgr2=0x7f6a3819b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3cc00640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a380768f0 0x7f6a3819b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:01.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3cc00640 1 -- 172.21.15.131:0/462372103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381a28d0 con 0x7f6a3810a710 2024-04-26T20:09:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a37e00640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a38075310 0x7f6a3819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3cc00640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3810a710 0x7f6a381a2230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009fd0 tx=0x7f6a3000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a35600640 1 -- 172.21.15.131:0/462372103 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30017070 con 0x7f6a3810a710 2024-04-26T20:09:01.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a35600640 1 -- 172.21.15.131:0/462372103 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a30003e20 con 0x7f6a3810a710 2024-04-26T20:09:01.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a35600640 1 -- 172.21.15.131:0/462372103 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30010460 con 0x7f6a3810a710 2024-04-26T20:09:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a381a2bc0 con 0x7f6a3810a710 2024-04-26T20:09:01.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.237+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381a3090 con 0x7f6a3810a710 2024-04-26T20:09:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.239+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a381094d0 con 0x7f6a3810a710 2024-04-26T20:09:01.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.240+0000 7f6a35600640 1 -- 172.21.15.131:0/462372103 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6a30003970 con 0x7f6a3810a710 2024-04-26T20:09:01.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.242+0000 7f6a35600640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6a1c0a3bb0 0x7f6a1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.242+0000 7f6a35600640 1 -- 172.21.15.131:0/462372103 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f6a300d04c0 con 0x7f6a3810a710 2024-04-26T20:09:01.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.242+0000 7f6a37e00640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6a1c0a3bb0 0x7f6a1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:01.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.243+0000 7f6a37e00640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6a1c0a3bb0 0x7f6a1c0a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a3819bb60 tx=0x7f6a28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:01.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.247+0000 7f6a35600640 1 -- 172.21.15.131:0/462372103 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6a3009c490 con 0x7f6a3810a710 2024-04-26T20:09:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.476+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6a38062d70 con 0x7f6a1c0a3bb0 2024-04-26T20:09:02.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.509+0000 7f6a35600640 1 -- 172.21.15.131:0/462372103 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a38062d70 con 0x7f6a1c0a3bb0 2024-04-26T20:09:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.511+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6a1c0a3bb0 msgr2=0x7f6a1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:02.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.511+0000 7f6a3d600640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6a1c0a3bb0 0x7f6a1c0a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a3819bb60 tx=0x7f6a28005f90 comp rx=0 tx=0).stop 2024-04-26T20:09:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.511+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3810a710 msgr2=0x7f6a381a2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:02.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.511+0000 7f6a3d600640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3810a710 0x7f6a381a2230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009fd0 tx=0x7f6a3000b260 comp rx=0 tx=0).stop 2024-04-26T20:09:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 shutdown_connections 2024-04-26T20:09:02.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6a1c0a3bb0 0x7f6a1c0a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:02.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3810a710 0x7f6a381a2230 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:02.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a380768f0 0x7f6a3819b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:02.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 --2- 172.21.15.131:0/462372103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a38075310 0x7f6a3819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 >> 172.21.15.131:0/462372103 conn(0x7f6a380fddd0 msgr2=0x7f6a381066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 shutdown_connections 2024-04-26T20:09:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:00.512+0000 7f6a3d600640 1 -- 172.21.15.131:0/462372103 wait complete. 2024-04-26T20:09:02.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T20:09:02.743 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:00 smithi148 ceph-mon[32281]: pgmap v378: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T20:09:02.779 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:00 smithi148 ceph-mon[32281]: from='client.24814 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:02.812 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:00 smithi157 ceph-mon[32703]: pgmap v378: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T20:09:02.848 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:00 smithi157 ceph-mon[32703]: from='client.24814 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:02.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:00 smithi131 ceph-mon[27919]: pgmap v378: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T20:09:02.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:00 smithi131 ceph-mon[27919]: from='client.24814 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:03.008 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:09:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:09:00.484+0000 7f635ba00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:09:03.037 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:09:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:09:00.484+0000 7f635ba00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:09:03.070 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:09:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:09:00.484+0000 7f635ba00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:09:03.110 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:09:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:09:00.484+0000 7f635ba00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:09:03.182 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:09:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:09:00.484+0000 7f635ba00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:09:03.221 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:02 smithi131 ceph-mon[27919]: pgmap v379: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T20:09:03.255 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:02 smithi148 ceph-mon[32281]: pgmap v379: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T20:09:03.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:02 smithi157 ceph-mon[32703]: pgmap v379: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T20:09:03.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 -- 172.21.15.131:0/2201165348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 msgr2=0x7f3e74104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:03.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 --2- 172.21.15.131:0/2201165348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e74104fb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009900 tx=0x7f3e6802a410 comp rx=0 tx=0).stop 2024-04-26T20:09:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 -- 172.21.15.131:0/2201165348 shutdown_connections 2024-04-26T20:09:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 --2- 172.21.15.131:0/2201165348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e74106740 0x7f3e7410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:03.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 --2- 172.21.15.131:0/2201165348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e74105d80 0x7f3e74106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 --2- 172.21.15.131:0/2201165348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e74104fb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:03.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 -- 172.21.15.131:0/2201165348 >> 172.21.15.131:0/2201165348 conn(0x7f3e74100120 msgr2=0x7f3e74102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:03.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 -- 172.21.15.131:0/2201165348 shutdown_connections 2024-04-26T20:09:03.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.439+0000 7f3e7be00640 1 -- 172.21.15.131:0/2201165348 wait complete. 2024-04-26T20:09:03.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 Processor -- start 2024-04-26T20:09:03.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 -- start start 2024-04-26T20:09:03.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e7419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:03.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e74105d80 0x7f3e7419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:04.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e74106740 0x7f3e741a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:04.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741106b0 con 0x7f3e74104b90 2024-04-26T20:09:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74110530 con 0x7f3e74105d80 2024-04-26T20:09:04.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.440+0000 7f3e7be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74110830 con 0x7f3e74106740 2024-04-26T20:09:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e7419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:04.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e78a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e74105d80 0x7f3e7419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:04.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e7419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41096/0 (socket says 172.21.15.131:41096) 2024-04-26T20:09:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 -- 172.21.15.131:0/3631062381 learned_addr learned my addr 172.21.15.131:0/3631062381 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:04.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79e00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e74106740 0x7f3e741a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:04.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 -- 172.21.15.131:0/3631062381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e74106740 msgr2=0x7f3e741a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:04.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e74106740 0x7f3e741a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 -- 172.21.15.131:0/3631062381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e74105d80 msgr2=0x7f3e7419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e74105d80 0x7f3e7419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e79400640 1 -- 172.21.15.131:0/3631062381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e741a2a10 con 0x7f3e74104b90 2024-04-26T20:09:04.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.441+0000 7f3e78a00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e74105d80 0x7f3e7419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:04.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.442+0000 7f3e79400640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e7419adf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006010 tx=0x7f3e680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:04.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.442+0000 7f3e72000640 1 -- 172.21.15.131:0/3631062381 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6803f030 con 0x7f3e74104b90 2024-04-26T20:09:04.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.442+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e741a2ca0 con 0x7f3e74104b90 2024-04-26T20:09:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.442+0000 7f3e72000640 1 -- 172.21.15.131:0/3631062381 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e68031910 con 0x7f3e74104b90 2024-04-26T20:09:04.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.442+0000 7f3e72000640 1 -- 172.21.15.131:0/3631062381 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68035070 con 0x7f3e74104b90 2024-04-26T20:09:04.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.442+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e741a3190 con 0x7f3e74104b90 2024-04-26T20:09:04.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.443+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7410bfd0 con 0x7f3e74104b90 2024-04-26T20:09:04.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.445+0000 7f3e72000640 1 -- 172.21.15.131:0/3631062381 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3e68042030 con 0x7f3e74104b90 2024-04-26T20:09:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.447+0000 7f3e72000640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3e5c0a3c40 0x7f3e5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.447+0000 7f3e72000640 1 -- 172.21.15.131:0/3631062381 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f3e680f0e20 con 0x7f3e74104b90 2024-04-26T20:09:05.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.447+0000 7f3e78a00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3e5c0a3c40 0x7f3e5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:05.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.448+0000 7f3e78a00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3e5c0a3c40 0x7f3e5c0a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e7419c300 tx=0x7f3e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:05.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.451+0000 7f3e72000640 1 -- 172.21.15.131:0/3631062381 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3e680bcc90 con 0x7f3e74104b90 2024-04-26T20:09:05.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.688+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3e74062e00 con 0x7f3e5c0a3c40 2024-04-26T20:09:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.732+0000 7f3e72000640 1 -- 172.21.15.131:0/3631062381 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e74062e00 con 0x7f3e5c0a3c40 2024-04-26T20:09:05.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.735+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3e5c0a3c40 msgr2=0x7f3e5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.735+0000 7f3e7be00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3e5c0a3c40 0x7f3e5c0a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e7419c300 tx=0x7f3e60005f90 comp rx=0 tx=0).stop 2024-04-26T20:09:05.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.735+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 msgr2=0x7f3e7419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:05.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.735+0000 7f3e7be00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e7419adf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006010 tx=0x7f3e680042f0 comp rx=0 tx=0).stop 2024-04-26T20:09:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 shutdown_connections 2024-04-26T20:09:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3e5c0a3c40 0x7f3e5c0a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:05.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e74106740 0x7f3e741a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:05.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e74105d80 0x7f3e7419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 --2- 172.21.15.131:0/3631062381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e74104b90 0x7f3e7419adf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:05.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 >> 172.21.15.131:0/3631062381 conn(0x7f3e74100120 msgr2=0x7f3e741091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:05.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 shutdown_connections 2024-04-26T20:09:05.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:03.736+0000 7f3e7be00640 1 -- 172.21.15.131:0/3631062381 wait complete. 2024-04-26T20:09:05.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T20:09:05.783 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:03 smithi157 ceph-mon[32703]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T20:09:05.820 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:04 smithi157 ceph-mon[32703]: pgmap v380: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T20:09:05.856 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:04 smithi157 ceph-mon[32703]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:05.890 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:03 smithi148 ceph-mon[32281]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T20:09:05.923 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:04 smithi148 ceph-mon[32281]: pgmap v380: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T20:09:05.977 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:04 smithi148 ceph-mon[32281]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:06.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:03 smithi131 ceph-mon[27919]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T20:09:06.065 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:04 smithi131 ceph-mon[27919]: pgmap v380: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T20:09:06.098 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:04 smithi131 ceph-mon[27919]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:06.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 -- 172.21.15.131:0/3000519352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 msgr2=0x7f15f0109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 --2- 172.21.15.131:0/3000519352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 0x7f15f0109290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009900 tx=0x7f15e002a410 comp rx=0 tx=0).stop 2024-04-26T20:09:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 -- 172.21.15.131:0/3000519352 shutdown_connections 2024-04-26T20:09:06.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 --2- 172.21.15.131:0/3000519352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f010a980 0x7f15f01112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:06.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 --2- 172.21.15.131:0/3000519352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f0109fc0 0x7f15f010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:06.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 --2- 172.21.15.131:0/3000519352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 0x7f15f0109290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:06.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 -- 172.21.15.131:0/3000519352 >> 172.21.15.131:0/3000519352 conn(0x7f15f0077e00 msgr2=0x7f15f007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 -- 172.21.15.131:0/3000519352 shutdown_connections 2024-04-26T20:09:06.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.351+0000 7f15f5200640 1 -- 172.21.15.131:0/3000519352 wait complete. 2024-04-26T20:09:06.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.352+0000 7f15f5200640 1 Processor -- start 2024-04-26T20:09:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15f5200640 1 -- start start 2024-04-26T20:09:06.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15f5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 0x7f15f019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:06.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15f5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f0109fc0 0x7f15f019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:07.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15f5200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f010a980 0x7f15f01a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:07.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15f5200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01149a0 con 0x7f15f0108e70 2024-04-26T20:09:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15f5200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0114820 con 0x7f15f0109fc0 2024-04-26T20:09:07.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15f5200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0114b20 con 0x7f15f010a980 2024-04-26T20:09:07.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15ef400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 0x7f15f019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15efe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f010a980 0x7f15f01a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15efe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f010a980 0x7f15f01a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33426/0 (socket says 172.21.15.131:33426) 2024-04-26T20:09:07.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15efe00640 1 -- 172.21.15.131:0/2977083390 learned_addr learned my addr 172.21.15.131:0/2977083390 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:07.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.353+0000 7f15e7e00640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f0109fc0 0x7f15f019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:07.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15efe00640 1 -- 172.21.15.131:0/2977083390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f0109fc0 msgr2=0x7f15f019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:07.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15efe00640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f0109fc0 0x7f15f019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:07.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15efe00640 1 -- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 msgr2=0x7f15f019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:07.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15efe00640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 0x7f15f019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15efe00640 1 -- 172.21.15.131:0/2977083390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f01a6ca0 con 0x7f15f010a980 2024-04-26T20:09:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15efe00640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f010a980 0x7f15f01a6630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15e800efd0 tx=0x7f15e800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15ecc00640 1 -- 172.21.15.131:0/2977083390 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8003b70 con 0x7f15f010a980 2024-04-26T20:09:07.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15ecc00640 1 -- 172.21.15.131:0/2977083390 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e8003d10 con 0x7f15f010a980 2024-04-26T20:09:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f01a6f90 con 0x7f15f010a980 2024-04-26T20:09:07.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15ecc00640 1 -- 172.21.15.131:0/2977083390 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80108a0 con 0x7f15f010a980 2024-04-26T20:09:07.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.354+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f01a7400 con 0x7f15f010a980 2024-04-26T20:09:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.355+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f0110200 con 0x7f15f010a980 2024-04-26T20:09:07.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.357+0000 7f15ecc00640 1 -- 172.21.15.131:0/2977083390 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f15e8004b40 con 0x7f15f010a980 2024-04-26T20:09:07.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.359+0000 7f15ecc00640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15d40a3c40 0x7f15d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:08.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.359+0000 7f15ecc00640 1 -- 172.21.15.131:0/2977083390 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f15e80d1100 con 0x7f15f010a980 2024-04-26T20:09:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.359+0000 7f15ef400640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15d40a3c40 0x7f15d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:08.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.360+0000 7f15ef400640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15d40a3c40 0x7f15d40a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15e0000c00 tx=0x7f15e00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:08.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.361+0000 7f15ecc00640 1 -- 172.21.15.131:0/2977083390 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f15e8043020 con 0x7f15f010a980 2024-04-26T20:09:08.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.596+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f15f0062d70 con 0x7f15d40a3c40 2024-04-26T20:09:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.607+0000 7f15ecc00640 1 -- 172.21.15.131:0/2977083390 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f15f0062d70 con 0x7f15d40a3c40 2024-04-26T20:09:08.358 INFO:teuthology.orchestra.run.smithi131.stdout:/volumes/qa/sv_0/bb326790-17c4-4431-a7a9-560eea459031 2024-04-26T20:09:08.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15d40a3c40 msgr2=0x7f15d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:08.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15d40a3c40 0x7f15d40a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15e0000c00 tx=0x7f15e00314a0 comp rx=0 tx=0).stop 2024-04-26T20:09:08.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f010a980 msgr2=0x7f15f01a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:08.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f010a980 0x7f15f01a6630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15e800efd0 tx=0x7f15e800c6a0 comp rx=0 tx=0).stop 2024-04-26T20:09:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 shutdown_connections 2024-04-26T20:09:08.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15d40a3c40 0x7f15d40a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:08.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15f010a980 0x7f15f01a6630 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f0109fc0 0x7f15f019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 --2- 172.21.15.131:0/2977083390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f0108e70 0x7f15f019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:08.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.610+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 >> 172.21.15.131:0/2977083390 conn(0x7f15f0077e00 msgr2=0x7f15f010d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:08.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.611+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 shutdown_connections 2024-04-26T20:09:08.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:06.611+0000 7f15f5200640 1 -- 172.21.15.131:0/2977083390 wait complete. 2024-04-26T20:09:08.901 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T20:09:08.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:06 smithi131 ceph-mon[27919]: pgmap v381: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:09:09.000 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:06 smithi131 ceph-mon[27919]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:06 smithi157 ceph-mon[32703]: pgmap v381: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:09:09.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:06 smithi157 ceph-mon[32703]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:09.108 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:06 smithi148 ceph-mon[32281]: pgmap v381: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:09:09.141 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:06 smithi148 ceph-mon[32281]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:09.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:08 smithi131 ceph-mon[27919]: pgmap v382: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:09:09.279 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:08 smithi157 ceph-mon[32703]: pgmap v382: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:09:09.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:08 smithi148 ceph-mon[32281]: pgmap v382: 129 pgs: 129 active+clean; 591 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:09:09.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 -- 172.21.15.131:0/761203880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac104a70 msgr2=0x7f82ac104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:09.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 --2- 172.21.15.131:0/761203880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac104a70 0x7f82ac104e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f829c009860 tx=0x7f829c02a340 comp rx=0 tx=0).stop 2024-04-26T20:09:09.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 -- 172.21.15.131:0/761203880 shutdown_connections 2024-04-26T20:09:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 --2- 172.21.15.131:0/761203880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82ac106620 0x7f82ac10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:09.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 --2- 172.21.15.131:0/761203880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82ac105c60 0x7f82ac1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:09.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 --2- 172.21.15.131:0/761203880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac104a70 0x7f82ac104e90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:09.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 -- 172.21.15.131:0/761203880 >> 172.21.15.131:0/761203880 conn(0x7f82ac0fffe0 msgr2=0x7f82ac102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:09.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 -- 172.21.15.131:0/761203880 shutdown_connections 2024-04-26T20:09:09.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.253+0000 7f82b3200640 1 -- 172.21.15.131:0/761203880 wait complete. 2024-04-26T20:09:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 Processor -- start 2024-04-26T20:09:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 -- start start 2024-04-26T20:09:09.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82ac104a70 0x7f82ac19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:09.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82ac105c60 0x7f82ac19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:09.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac106620 0x7f82ac1a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:10.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1105a0 con 0x7f82ac104a70 2024-04-26T20:09:10.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac110420 con 0x7f82ac106620 2024-04-26T20:09:10.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.254+0000 7f82b3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac110720 con 0x7f82ac105c60 2024-04-26T20:09:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac106620 0x7f82ac1a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:10.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82ac105c60 0x7f82ac19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:10.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82ac104a70 0x7f82ac19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:10.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac106620 0x7f82ac1a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:37776/0 (socket says 172.21.15.131:37776) 2024-04-26T20:09:10.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 -- 172.21.15.131:0/912755505 learned_addr learned my addr 172.21.15.131:0/912755505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:10.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 -- 172.21.15.131:0/912755505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82ac105c60 msgr2=0x7f82ac19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:10.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82ac105c60 0x7f82ac19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 -- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82ac104a70 msgr2=0x7f82ac19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:10.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82ac104a70 0x7f82ac19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:10.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 -- 172.21.15.131:0/912755505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1a28e0 con 0x7f82ac106620 2024-04-26T20:09:10.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.255+0000 7f82b2800640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac106620 0x7f82ac1a2240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82a800eb10 tx=0x7f82a800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:10.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.256+0000 7f82a6a00640 1 -- 172.21.15.131:0/912755505 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a800ce20 con 0x7f82ac106620 2024-04-26T20:09:10.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.256+0000 7f82a6a00640 1 -- 172.21.15.131:0/912755505 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a8005020 con 0x7f82ac106620 2024-04-26T20:09:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.256+0000 7f82a6a00640 1 -- 172.21.15.131:0/912755505 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8010690 con 0x7f82ac106620 2024-04-26T20:09:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.256+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1a2bd0 con 0x7f82ac106620 2024-04-26T20:09:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.256+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a30d0 con 0x7f82ac106620 2024-04-26T20:09:10.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.257+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac10be30 con 0x7f82ac106620 2024-04-26T20:09:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.259+0000 7f82a6a00640 1 -- 172.21.15.131:0/912755505 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f82a8005510 con 0x7f82ac106620 2024-04-26T20:09:10.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.260+0000 7f82a6a00640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f82940a3cc0 0x7f82940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:11.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.261+0000 7f82a6a00640 1 -- 172.21.15.131:0/912755505 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f82a80d0fe0 con 0x7f82ac106620 2024-04-26T20:09:11.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.261+0000 7f82b1e00640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f82940a3cc0 0x7f82940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:11.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.261+0000 7f82b1e00640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f82940a3cc0 0x7f82940a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f829c009830 tx=0x7f829c030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.263+0000 7f82a6a00640 1 -- 172.21.15.131:0/912755505 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f82a809d500 con 0x7f82ac106620 2024-04-26T20:09:11.152 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:09 smithi157 ceph-mon[32703]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:11.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:09 smithi148 ceph-mon[32281]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:11.218 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:10 smithi148 ceph-mon[32281]: pgmap v383: 129 pgs: 129 active+clean; 610 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-04-26T20:09:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.495+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f82ac062d70 con 0x7f82940a3cc0 2024-04-26T20:09:11.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.541+0000 7f82a6a00640 1 -- 172.21.15.131:0/912755505 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82ac062d70 con 0x7f82940a3cc0 2024-04-26T20:09:11.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f82940a3cc0 msgr2=0x7f82940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:11.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f82940a3cc0 0x7f82940a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f829c009830 tx=0x7f829c030470 comp rx=0 tx=0).stop 2024-04-26T20:09:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac106620 msgr2=0x7f82ac1a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac106620 0x7f82ac1a2240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82a800eb10 tx=0x7f82a800ef10 comp rx=0 tx=0).stop 2024-04-26T20:09:11.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 shutdown_connections 2024-04-26T20:09:11.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ac106620 0x7f82ac1a2240 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f82940a3cc0 0x7f82940a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:11.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82ac105c60 0x7f82ac19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:11.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 --2- 172.21.15.131:0/912755505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82ac104a70 0x7f82ac19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 >> 172.21.15.131:0/912755505 conn(0x7f82ac0fffe0 msgr2=0x7f82ac109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:11.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 shutdown_connections 2024-04-26T20:09:11.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:09.544+0000 7f82b3200640 1 -- 172.21.15.131:0/912755505 wait complete. 2024-04-26T20:09:11.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T20:09:11.902 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:09 smithi131 ceph-mon[27919]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:11.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:10 smithi131 ceph-mon[27919]: pgmap v383: 129 pgs: 129 active+clean; 610 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-04-26T20:09:12.019 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:10 smithi157 ceph-mon[32703]: pgmap v383: 129 pgs: 129 active+clean; 610 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-04-26T20:09:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.237+0000 7faaa8000640 1 -- 172.21.15.131:0/1542771097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0106740 msgr2=0x7faaa010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.237+0000 7faaa8000640 1 --2- 172.21.15.131:0/1542771097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0106740 0x7faaa010cfd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faa98009900 tx=0x7faa9802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.237+0000 7faaa8000640 1 -- 172.21.15.131:0/1542771097 shutdown_connections 2024-04-26T20:09:12.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.237+0000 7faaa8000640 1 --2- 172.21.15.131:0/1542771097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0106740 0x7faaa010cfd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.237+0000 7faaa8000640 1 --2- 172.21.15.131:0/1542771097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa0105d80 0x7faaa0106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.237+0000 7faaa8000640 1 --2- 172.21.15.131:0/1542771097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa0104b90 0x7faaa0104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.237+0000 7faaa8000640 1 -- 172.21.15.131:0/1542771097 >> 172.21.15.131:0/1542771097 conn(0x7faaa0100120 msgr2=0x7faaa0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:12.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.238+0000 7faaa8000640 1 -- 172.21.15.131:0/1542771097 shutdown_connections 2024-04-26T20:09:12.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.238+0000 7faaa8000640 1 -- 172.21.15.131:0/1542771097 wait complete. 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 Processor -- start 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 -- start start 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa0104b90 0x7faaa019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0105d80 0x7faaa019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa0106740 0x7faaa01a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01106c0 con 0x7faaa0105d80 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0110540 con 0x7faaa0104b90 2024-04-26T20:09:12.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa8000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0110840 con 0x7faaa0106740 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0105d80 0x7faaa019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0105d80 0x7faaa019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57280/0 (socket says 172.21.15.131:57280) 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa4c00640 1 -- 172.21.15.131:0/3291303462 learned_addr learned my addr 172.21.15.131:0/3291303462 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa5600640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa0104b90 0x7faaa019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.239+0000 7faaa6000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa0106740 0x7faaa01a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faaa4c00640 1 -- 172.21.15.131:0/3291303462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa0106740 msgr2=0x7faaa01a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faaa4c00640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa0106740 0x7faaa01a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faaa4c00640 1 -- 172.21.15.131:0/3291303462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa0104b90 msgr2=0x7faaa019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faaa4c00640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa0104b90 0x7faaa019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faaa4c00640 1 -- 172.21.15.131:0/3291303462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa01a29b0 con 0x7faaa0105d80 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faaa4c00640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0105d80 0x7faaa019b2e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e9f0 tx=0x7faa8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faa9e000640 1 -- 172.21.15.131:0/3291303462 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00cd80 con 0x7faaa0105d80 2024-04-26T20:09:12.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faa9e000640 1 -- 172.21.15.131:0/3291303462 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa8c0054d0 con 0x7faaa0105d80 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa01a2ca0 con 0x7faaa0105d80 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.240+0000 7faa9e000640 1 -- 172.21.15.131:0/3291303462 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c010690 con 0x7faaa0105d80 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.241+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa01a31a0 con 0x7faaa0105d80 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.242+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa010bf80 con 0x7faaa0105d80 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.244+0000 7faa9e000640 1 -- 172.21.15.131:0/3291303462 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7faa8c004e90 con 0x7faaa0105d80 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.245+0000 7faa9e000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7faa880a3c80 0x7faa880a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.245+0000 7faa9e000640 1 -- 172.21.15.131:0/3291303462 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7faa8c0d0dd0 con 0x7faaa0105d80 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.245+0000 7faaa5600640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7faa880a3c80 0x7faa880a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.246+0000 7faaa5600640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7faa880a3c80 0x7faa880a6140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faaa019c270 tx=0x7faa94009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:12.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.249+0000 7faa9e000640 1 -- 172.21.15.131:0/3291303462 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faa8c09d290 con 0x7faaa0105d80 2024-04-26T20:09:12.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.484+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7faaa0062d70 con 0x7faa880a3c80 2024-04-26T20:09:12.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.495+0000 7faa9e000640 1 -- 172.21.15.131:0/3291303462 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7faaa0062d70 con 0x7faa880a3c80 2024-04-26T20:09:12.496 INFO:teuthology.orchestra.run.smithi131.stdout:/volumes/qa/sv_1/a29373f3-2ca2-48d2-9585-6ab9ff653e9e 2024-04-26T20:09:12.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.497+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7faa880a3c80 msgr2=0x7faa880a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.497+0000 7faaa8000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7faa880a3c80 0x7faa880a6140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faaa019c270 tx=0x7faa94009500 comp rx=0 tx=0).stop 2024-04-26T20:09:12.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.497+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0105d80 msgr2=0x7faaa019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.497+0000 7faaa8000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0105d80 0x7faaa019b2e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e9f0 tx=0x7faa8c00edf0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 shutdown_connections 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7faa880a3c80 0x7faa880a6140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa0106740 0x7faaa01a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa0105d80 0x7faaa019b2e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 --2- 172.21.15.131:0/3291303462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa0104b90 0x7faaa019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 >> 172.21.15.131:0/3291303462 conn(0x7faaa0100120 msgr2=0x7faaa0109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 shutdown_connections 2024-04-26T20:09:12.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.498+0000 7faaa8000640 1 -- 172.21.15.131:0/3291303462 wait complete. 2024-04-26T20:09:12.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:09:12.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 -- 172.21.15.131:0/3450341044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410a180 msgr2=0x7f477410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 --2- 172.21.15.131:0/3450341044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410a180 0x7f477410a600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4760009ca0 tx=0x7f476002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 -- 172.21.15.131:0/3450341044 shutdown_connections 2024-04-26T20:09:12.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 --2- 172.21.15.131:0/3450341044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f477410ab40 0x7f47741113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 --2- 172.21.15.131:0/3450341044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410a180 0x7f477410a600 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 --2- 172.21.15.131:0/3450341044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 0x7f47741093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 -- 172.21.15.131:0/3450341044 >> 172.21.15.131:0/3450341044 conn(0x7f4774077f40 msgr2=0x7f477407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:12.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 -- 172.21.15.131:0/3450341044 shutdown_connections 2024-04-26T20:09:12.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.721+0000 7f477be00640 1 -- 172.21.15.131:0/3450341044 wait complete. 2024-04-26T20:09:12.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.722+0000 7f477be00640 1 Processor -- start 2024-04-26T20:09:12.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.722+0000 7f477be00640 1 -- start start 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f477be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 0x7f477419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f477be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f477410a180 0x7f477419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f477be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410ab40 0x7f47741a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f477be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4774114ad0 con 0x7f4774108f90 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f477be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4774114950 con 0x7f477410ab40 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f477be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4774114c50 con 0x7f477410a180 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f4779400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 0x7f477419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f4778a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f477410a180 0x7f477419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f4779400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 0x7f477419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57298/0 (socket says 172.21.15.131:57298) 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f4779400640 1 -- 172.21.15.131:0/1552967138 learned_addr learned my addr 172.21.15.131:0/1552967138 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:12.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.723+0000 7f4779e00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410ab40 0x7f47741a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4779400640 1 -- 172.21.15.131:0/1552967138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f477410a180 msgr2=0x7f477419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4779400640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f477410a180 0x7f477419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4779400640 1 -- 172.21.15.131:0/1552967138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410ab40 msgr2=0x7f47741a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4779400640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410ab40 0x7f47741a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4779400640 1 -- 172.21.15.131:0/1552967138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47741a6d70 con 0x7f4774108f90 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4779400640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 0x7f477419f150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f476400e9c0 tx=0x7f476400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4772000640 1 -- 172.21.15.131:0/1552967138 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476400cd80 con 0x7f4774108f90 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4772000640 1 -- 172.21.15.131:0/1552967138 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47640035f0 con 0x7f4774108f90 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47741a7060 con 0x7f4774108f90 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f4772000640 1 -- 172.21.15.131:0/1552967138 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764010690 con 0x7f4774108f90 2024-04-26T20:09:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.724+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47741a75b0 con 0x7f4774108f90 2024-04-26T20:09:12.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.726+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47741102b0 con 0x7f4774108f90 2024-04-26T20:09:12.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.727+0000 7f4772000640 1 -- 172.21.15.131:0/1552967138 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f47640037a0 con 0x7f4774108f90 2024-04-26T20:09:12.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.729+0000 7f4772000640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47580a3c40 0x7f47580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:12.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.729+0000 7f4772000640 1 -- 172.21.15.131:0/1552967138 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f47640d0cb0 con 0x7f4774108f90 2024-04-26T20:09:12.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.729+0000 7f4778a00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47580a3c40 0x7f47580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:12.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.730+0000 7f4778a00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47580a3c40 0x7f47580a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47741a0660 tx=0x7f4760005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:12.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.733+0000 7f4772000640 1 -- 172.21.15.131:0/1552967138 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f476409d170 con 0x7f4774108f90 2024-04-26T20:09:12.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.968+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f477402dad0 con 0x7f4774108f90 2024-04-26T20:09:12.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.970+0000 7f4772000640 1 -- 172.21.15.131:0/1552967138 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11895 (secure 0 0 0) 0x7f476409ca70 con 0x7f4774108f90 2024-04-26T20:09:12.972 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:09:12.972 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:08:57.196498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24557,24571]},"id":1}]} 2024-04-26T20:09:12.972 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47580a3c40 msgr2=0x7f47580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47580a3c40 0x7f47580a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47741a0660 tx=0x7f4760005dc0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 msgr2=0x7f477419f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 0x7f477419f150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f476400e9c0 tx=0x7f476400ee80 comp rx=0 tx=0).stop 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 shutdown_connections 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f477410ab40 0x7f47741a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47580a3c40 0x7f47580a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f477410a180 0x7f477419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 --2- 172.21.15.131:0/1552967138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4774108f90 0x7f477419f150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:12.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 >> 172.21.15.131:0/1552967138 conn(0x7f4774077f40 msgr2=0x7f477410d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:12.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 shutdown_connections 2024-04-26T20:09:12.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:12.972+0000 7f477be00640 1 -- 172.21.15.131:0/1552967138 wait complete. 2024-04-26T20:09:12.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:09:13.055 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:12 smithi131 ceph-mon[27919]: pgmap v384: 129 pgs: 129 active+clean; 610 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-04-26T20:09:13.056 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:12 smithi131 ceph-mon[27919]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:13.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:12 smithi148 ceph-mon[32281]: pgmap v384: 129 pgs: 129 active+clean; 610 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-04-26T20:09:13.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:12 smithi148 ceph-mon[32281]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 -- 172.21.15.131:0/633588217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614106740 msgr2=0x7fc61410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 --2- 172.21.15.131:0/633588217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614106740 0x7fc61410cfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009be0 tx=0x7fc60c02a6c0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 -- 172.21.15.131:0/633588217 shutdown_connections 2024-04-26T20:09:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 --2- 172.21.15.131:0/633588217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614106740 0x7fc61410cfd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 --2- 172.21.15.131:0/633588217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614105d80 0x7fc614106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 --2- 172.21.15.131:0/633588217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614104b90 0x7fc614104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 -- 172.21.15.131:0/633588217 >> 172.21.15.131:0/633588217 conn(0x7fc614100120 msgr2=0x7fc614102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.180+0000 7fc619600640 1 -- 172.21.15.131:0/633588217 shutdown_connections 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.181+0000 7fc619600640 1 -- 172.21.15.131:0/633588217 wait complete. 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.181+0000 7fc619600640 1 Processor -- start 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.181+0000 7fc619600640 1 -- start start 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc619600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614104b90 0x7fc61419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc619600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614105d80 0x7fc61419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc619600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614106740 0x7fc6141a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc619600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6141106c0 con 0x7fc614105d80 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc619600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc614110540 con 0x7fc614104b90 2024-04-26T20:09:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc619600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc614110840 con 0x7fc614106740 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc60be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614105d80 0x7fc61419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614106740 0x7fc6141a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614106740 0x7fc6141a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33518/0 (socket says 172.21.15.131:33518) 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 -- 172.21.15.131:0/144386107 learned_addr learned my addr 172.21.15.131:0/144386107 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc612a00640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614104b90 0x7fc61419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 -- 172.21.15.131:0/144386107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614104b90 msgr2=0x7fc61419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614104b90 0x7fc61419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 -- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614105d80 msgr2=0x7fc61419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614105d80 0x7fc61419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc613400640 1 -- 172.21.15.131:0/144386107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6141a2960 con 0x7fc614106740 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.182+0000 7fc612a00640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614104b90 0x7fc61419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.183+0000 7fc613400640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614106740 0x7fc6141a22c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc60c004af0 tx=0x7fc60c030d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.183+0000 7fc60b400640 1 -- 172.21.15.131:0/144386107 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c041020 con 0x7fc614106740 2024-04-26T20:09:13.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.183+0000 7fc60b400640 1 -- 172.21.15.131:0/144386107 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60c030ec0 con 0x7fc614106740 2024-04-26T20:09:13.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.183+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6141a2bf0 con 0x7fc614106740 2024-04-26T20:09:13.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.183+0000 7fc60b400640 1 -- 172.21.15.131:0/144386107 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c036040 con 0x7fc614106740 2024-04-26T20:09:13.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.183+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6141a3130 con 0x7fc614106740 2024-04-26T20:09:13.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.184+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61410beb0 con 0x7fc614106740 2024-04-26T20:09:13.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.186+0000 7fc60b400640 1 -- 172.21.15.131:0/144386107 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc60c044020 con 0x7fc614106740 2024-04-26T20:09:13.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.187+0000 7fc60b400640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5f80a3cc0 0x7fc5f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.188+0000 7fc60b400640 1 -- 172.21.15.131:0/144386107 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fc60c0f16a0 con 0x7fc614106740 2024-04-26T20:09:13.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.188+0000 7fc612a00640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5f80a3cc0 0x7fc5f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.188+0000 7fc612a00640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5f80a3cc0 0x7fc5f80a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc61419c220 tx=0x7fc604005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:13.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.192+0000 7fc60b400640 1 -- 172.21.15.131:0/144386107 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc60c04b020 con 0x7fc614106740 2024-04-26T20:09:13.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:12 smithi157 ceph-mon[32703]: pgmap v384: 129 pgs: 129 active+clean; 610 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-04-26T20:09:13.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:12 smithi157 ceph-mon[32703]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:09:13.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.380+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc61402dcb0 con 0x7fc614106740 2024-04-26T20:09:13.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.383+0000 7fc60b400640 1 -- 172.21.15.131:0/144386107 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24715 (secure 0 0 0) 0x7fc60c04e350 con 0x7fc614106740 2024-04-26T20:09:13.384 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:09:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:08:15.562795+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:09:13.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5f80a3cc0 msgr2=0x7fc5f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5f80a3cc0 0x7fc5f80a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc61419c220 tx=0x7fc604005f90 comp rx=0 tx=0).stop 2024-04-26T20:09:13.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614106740 msgr2=0x7fc6141a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614106740 0x7fc6141a22c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc60c004af0 tx=0x7fc60c030d50 comp rx=0 tx=0).stop 2024-04-26T20:09:13.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 shutdown_connections 2024-04-26T20:09:13.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5f80a3cc0 0x7fc5f80a6180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc614106740 0x7fc6141a22c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc614105d80 0x7fc61419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 --2- 172.21.15.131:0/144386107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc614104b90 0x7fc61419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 >> 172.21.15.131:0/144386107 conn(0x7fc614100120 msgr2=0x7fc614109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:13.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 shutdown_connections 2024-04-26T20:09:13.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.385+0000 7fc619600640 1 -- 172.21.15.131:0/144386107 wait complete. 2024-04-26T20:09:13.405 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:09:13.405 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:09:13.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 -- 172.21.15.131:0/3581462418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10a180 msgr2=0x7f6f7c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 --2- 172.21.15.131:0/3581462418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10a180 0x7f6f7c10a600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009860 tx=0x7f6f6402a340 comp rx=0 tx=0).stop 2024-04-26T20:09:13.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 -- 172.21.15.131:0/3581462418 shutdown_connections 2024-04-26T20:09:13.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 --2- 172.21.15.131:0/3581462418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7c10ab40 0x7f6f7c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 --2- 172.21.15.131:0/3581462418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10a180 0x7f6f7c10a600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 --2- 172.21.15.131:0/3581462418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 0x7f6f7c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 -- 172.21.15.131:0/3581462418 >> 172.21.15.131:0/3581462418 conn(0x7f6f7c077f40 msgr2=0x7f6f7c07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:13.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 -- 172.21.15.131:0/3581462418 shutdown_connections 2024-04-26T20:09:13.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.604+0000 7f6f81400640 1 -- 172.21.15.131:0/3581462418 wait complete. 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.605+0000 7f6f81400640 1 Processor -- start 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f81400640 1 -- start start 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f81400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 0x7f6f7c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f81400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7c10a180 0x7f6f7c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f81400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10ab40 0x7f6f7c1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f81400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c114ad0 con 0x7f6f7c10a180 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f81400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c114950 con 0x7f6f7c10ab40 2024-04-26T20:09:13.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f81400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c114c50 con 0x7f6f7c108f90 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f7a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 0x7f6f7c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f71e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7c10a180 0x7f6f7c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f7a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 0x7f6f7c19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33536/0 (socket says 172.21.15.131:33536) 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f7a800640 1 -- 172.21.15.131:0/461771067 learned_addr learned my addr 172.21.15.131:0/461771067 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.606+0000 7f6f7b200640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10ab40 0x7f6f7c1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f7a800640 1 -- 172.21.15.131:0/461771067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10ab40 msgr2=0x7f6f7c1a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f7a800640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10ab40 0x7f6f7c1a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f7a800640 1 -- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7c10a180 msgr2=0x7f6f7c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f7a800640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7c10a180 0x7f6f7c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f7a800640 1 -- 172.21.15.131:0/461771067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c1a6d90 con 0x7f6f7c108f90 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f7b200640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10ab40 0x7f6f7c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f7a800640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 0x7f6f7c19f170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00eb70 tx=0x7f6f6c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f73e00640 1 -- 172.21.15.131:0/461771067 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c003b70 con 0x7f6f7c108f90 2024-04-26T20:09:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7c1a7080 con 0x7f6f7c108f90 2024-04-26T20:09:13.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f73e00640 1 -- 172.21.15.131:0/461771067 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6c00ee70 con 0x7f6f7c108f90 2024-04-26T20:09:13.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f73e00640 1 -- 172.21.15.131:0/461771067 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c010650 con 0x7f6f7c108f90 2024-04-26T20:09:13.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.607+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7c1a75a0 con 0x7f6f7c108f90 2024-04-26T20:09:13.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.608+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7c1102b0 con 0x7f6f7c108f90 2024-04-26T20:09:13.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.610+0000 7f6f73e00640 1 -- 172.21.15.131:0/461771067 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6f6c004a40 con 0x7f6f7c108f90 2024-04-26T20:09:13.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.612+0000 7f6f73e00640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f600a3cc0 0x7f6f600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:13.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.612+0000 7f6f73e00640 1 -- 172.21.15.131:0/461771067 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f6f6c0d10b0 con 0x7f6f7c108f90 2024-04-26T20:09:13.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.612+0000 7f6f71e00640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f600a3cc0 0x7f6f600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:13.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.613+0000 7f6f71e00640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f600a3cc0 0x7f6f600a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f6402a930 tx=0x7f6f64030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:13.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.614+0000 7f6f73e00640 1 -- 172.21.15.131:0/461771067 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6f6c09d570 con 0x7f6f7c108f90 2024-04-26T20:09:13.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.807+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f7c062d70 con 0x7f6f600a3cc0 2024-04-26T20:09:13.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.819+0000 7f6f73e00640 1 -- 172.21.15.131:0/461771067 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7f6f7c062d70 con 0x7f6f600a3cc0 2024-04-26T20:09:13.826 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:09:13.828 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f600a3cc0 msgr2=0x7f6f600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f600a3cc0 0x7f6f600a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f6402a930 tx=0x7f6f64030450 comp rx=0 tx=0).stop 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 msgr2=0x7f6f7c19f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 0x7f6f7c19f170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00eb70 tx=0x7f6f6c00c6a0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 shutdown_connections 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f7c10ab40 0x7f6f7c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7c10a180 0x7f6f7c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f600a3cc0 0x7f6f600a6180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 --2- 172.21.15.131:0/461771067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f7c108f90 0x7f6f7c19f170 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 >> 172.21.15.131:0/461771067 conn(0x7f6f7c077f40 msgr2=0x7f6f7c10d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 shutdown_connections 2024-04-26T20:09:13.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:13.826+0000 7f6f81400640 1 -- 172.21.15.131:0/461771067 wait complete. 2024-04-26T20:09:13.847 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-04-26T20:09:13.633527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647083,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":271,"num_read_kb":242,"num_write":340,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368036,"kb_used_data":14724,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573788,"statfs":{"total":1151940427776,"available":1151563558912,"internally_reserved":0,"allocated":15077376,"data_stored":8330834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49263,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":95,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006261"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571774+0000","last_change":"2024-04-26T20:07:34.771977+0000","last_active":"2024-04-26T20:08:15.571774+0000","last_peered":"2024-04-26T20:08:15.571774+0000","last_clean":"2024-04-26T20:08:15.571774+0000","last_became_active":"2024-04-26T20:07:34.771658+0000","last_became_peered":"2024-04-26T20:07:34.771658+0000","last_unstale":"2024-04-26T20:08:15.571774+0000","last_undegraded":"2024-04-26T20:08:15.571774+0000","last_fullsized":"2024-04-26T20:08:15.571774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:38.419057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111647+0000","last_change":"2024-04-26T20:07:34.768774+0000","last_active":"2024-04-26T20:08:16.111647+0000","last_peered":"2024-04-26T20:08:16.111647+0000","last_clean":"2024-04-26T20:08:16.111647+0000","last_became_active":"2024-04-26T20:07:34.768383+0000","last_became_peered":"2024-04-26T20:07:34.768383+0000","last_unstale":"2024-04-26T20:08:16.111647+0000","last_undegraded":"2024-04-26T20:08:16.111647+0000","last_fullsized":"2024-04-26T20:08:16.111647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:43.813261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111048+0000","last_change":"2024-04-26T20:07:34.761796+0000","last_active":"2024-04-26T20:08:16.111048+0000","last_peered":"2024-04-26T20:08:16.111048+0000","last_clean":"2024-04-26T20:08:16.111048+0000","last_became_active":"2024-04-26T20:07:34.761403+0000","last_became_peered":"2024-04-26T20:07:34.761403+0000","last_unstale":"2024-04-26T20:08:16.111048+0000","last_undegraded":"2024-04-26T20:08:16.111048+0000","last_fullsized":"2024-04-26T20:08:16.111048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:06.832304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573588+0000","last_change":"2024-04-26T20:07:34.756728+0000","last_active":"2024-04-26T20:08:15.573588+0000","last_peered":"2024-04-26T20:08:15.573588+0000","last_clean":"2024-04-26T20:08:15.573588+0000","last_became_active":"2024-04-26T20:07:34.756406+0000","last_became_peered":"2024-04-26T20:07:34.756406+0000","last_unstale":"2024-04-26T20:08:15.573588+0000","last_undegraded":"2024-04-26T20:08:15.573588+0000","last_fullsized":"2024-04-26T20:08:15.573588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:27.753674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568635+0000","last_change":"2024-04-26T20:07:34.758243+0000","last_active":"2024-04-26T20:08:15.568635+0000","last_peered":"2024-04-26T20:08:15.568635+0000","last_clean":"2024-04-26T20:08:15.568635+0000","last_became_active":"2024-04-26T20:07:34.757868+0000","last_became_peered":"2024-04-26T20:07:34.757868+0000","last_unstale":"2024-04-26T20:08:15.568635+0000","last_undegraded":"2024-04-26T20:08:15.568635+0000","last_fullsized":"2024-04-26T20:08:15.568635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:53.361835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.585311+0000","last_change":"2024-04-26T20:08:15.585311+0000","last_active":"2024-04-26T20:08:15.585311+0000","last_peered":"2024-04-26T20:08:15.585311+0000","last_clean":"2024-04-26T20:08:15.585311+0000","last_became_active":"2024-04-26T20:08:15.584982+0000","last_became_peered":"2024-04-26T20:08:15.584982+0000","last_unstale":"2024-04-26T20:08:15.585311+0000","last_undegraded":"2024-04-26T20:08:15.585311+0000","last_fullsized":"2024-04-26T20:08:15.585311+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:56.828218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569002+0000","last_change":"2024-04-26T20:07:34.757529+0000","last_active":"2024-04-26T20:08:15.569002+0000","last_peered":"2024-04-26T20:08:15.569002+0000","last_clean":"2024-04-26T20:08:15.569002+0000","last_became_active":"2024-04-26T20:07:34.757166+0000","last_became_peered":"2024-04-26T20:07:34.757166+0000","last_unstale":"2024-04-26T20:08:15.569002+0000","last_undegraded":"2024-04-26T20:08:15.569002+0000","last_fullsized":"2024-04-26T20:08:15.569002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:01.309698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.570536+0000","last_change":"2024-04-26T20:07:34.765352+0000","last_active":"2024-04-26T20:08:15.570536+0000","last_peered":"2024-04-26T20:08:15.570536+0000","last_clean":"2024-04-26T20:08:15.570536+0000","last_became_active":"2024-04-26T20:07:34.758400+0000","last_became_peered":"2024-04-26T20:07:34.758400+0000","last_unstale":"2024-04-26T20:08:15.570536+0000","last_undegraded":"2024-04-26T20:08:15.570536+0000","last_fullsized":"2024-04-26T20:08:15.570536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:43.940746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374668+0000","last_change":"2024-04-26T20:07:34.754413+0000","last_active":"2024-04-26T20:08:16.374668+0000","last_peered":"2024-04-26T20:08:16.374668+0000","last_clean":"2024-04-26T20:08:16.374668+0000","last_became_active":"2024-04-26T20:07:34.754073+0000","last_became_peered":"2024-04-26T20:07:34.754073+0000","last_unstale":"2024-04-26T20:08:16.374668+0000","last_undegraded":"2024-04-26T20:08:16.374668+0000","last_fullsized":"2024-04-26T20:08:16.374668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:25.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.576145+0000","last_change":"2024-04-26T20:07:34.748739+0000","last_active":"2024-04-26T20:08:15.576145+0000","last_peered":"2024-04-26T20:08:15.576145+0000","last_clean":"2024-04-26T20:08:15.576145+0000","last_became_active":"2024-04-26T20:07:34.748378+0000","last_became_peered":"2024-04-26T20:07:34.748378+0000","last_unstale":"2024-04-26T20:08:15.576145+0000","last_undegraded":"2024-04-26T20:08:15.576145+0000","last_fullsized":"2024-04-26T20:08:15.576145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:13.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569570+0000","last_change":"2024-04-26T20:07:34.747402+0000","last_active":"2024-04-26T20:08:15.569570+0000","last_peered":"2024-04-26T20:08:15.569570+0000","last_clean":"2024-04-26T20:08:15.569570+0000","last_became_active":"2024-04-26T20:07:34.746965+0000","last_became_peered":"2024-04-26T20:07:34.746965+0000","last_unstale":"2024-04-26T20:08:15.569570+0000","last_undegraded":"2024-04-26T20:08:15.569570+0000","last_fullsized":"2024-04-26T20:08:15.569570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:32.066286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573453+0000","last_change":"2024-04-26T20:07:34.755848+0000","last_active":"2024-04-26T20:08:15.573453+0000","last_peered":"2024-04-26T20:08:15.573453+0000","last_clean":"2024-04-26T20:08:15.573453+0000","last_became_active":"2024-04-26T20:07:34.755403+0000","last_became_peered":"2024-04-26T20:07:34.755403+0000","last_unstale":"2024-04-26T20:08:15.573453+0000","last_undegraded":"2024-04-26T20:08:15.573453+0000","last_fullsized":"2024-04-26T20:08:15.573453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:30.576290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.585014+0000","last_change":"2024-04-26T20:07:34.766542+0000","last_active":"2024-04-26T20:08:15.585014+0000","last_peered":"2024-04-26T20:08:15.585014+0000","last_clean":"2024-04-26T20:08:15.585014+0000","last_became_active":"2024-04-26T20:07:34.766185+0000","last_became_peered":"2024-04-26T20:07:34.766185+0000","last_unstale":"2024-04-26T20:08:15.585014+0000","last_undegraded":"2024-04-26T20:08:15.585014+0000","last_fullsized":"2024-04-26T20:08:15.585014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:34.499145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111075+0000","last_change":"2024-04-26T20:07:34.762610+0000","last_active":"2024-04-26T20:08:16.111075+0000","last_peered":"2024-04-26T20:08:16.111075+0000","last_clean":"2024-04-26T20:08:16.111075+0000","last_became_active":"2024-04-26T20:07:34.762100+0000","last_became_peered":"2024-04-26T20:07:34.762100+0000","last_unstale":"2024-04-26T20:08:16.111075+0000","last_undegraded":"2024-04-26T20:08:16.111075+0000","last_fullsized":"2024-04-26T20:08:16.111075+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:18.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111002+0000","last_change":"2024-04-26T20:07:34.766754+0000","last_active":"2024-04-26T20:08:16.111002+0000","last_peered":"2024-04-26T20:08:16.111002+0000","last_clean":"2024-04-26T20:08:16.111002+0000","last_became_active":"2024-04-26T20:07:34.762204+0000","last_became_peered":"2024-04-26T20:07:34.762204+0000","last_unstale":"2024-04-26T20:08:16.111002+0000","last_undegraded":"2024-04-26T20:08:16.111002+0000","last_fullsized":"2024-04-26T20:08:16.111002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:25.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374070+0000","last_change":"2024-04-26T20:07:34.769739+0000","last_active":"2024-04-26T20:08:16.374070+0000","last_peered":"2024-04-26T20:08:16.374070+0000","last_clean":"2024-04-26T20:08:16.374070+0000","last_became_active":"2024-04-26T20:07:34.769383+0000","last_became_peered":"2024-04-26T20:07:34.769383+0000","last_unstale":"2024-04-26T20:08:16.374070+0000","last_undegraded":"2024-04-26T20:08:16.374070+0000","last_fullsized":"2024-04-26T20:08:16.374070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:08.771590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571611+0000","last_change":"2024-04-26T20:07:34.772010+0000","last_active":"2024-04-26T20:08:15.571611+0000","last_peered":"2024-04-26T20:08:15.571611+0000","last_clean":"2024-04-26T20:08:15.571611+0000","last_became_active":"2024-04-26T20:07:34.771682+0000","last_became_peered":"2024-04-26T20:07:34.771682+0000","last_unstale":"2024-04-26T20:08:15.571611+0000","last_undegraded":"2024-04-26T20:08:15.571611+0000","last_fullsized":"2024-04-26T20:08:15.571611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:57.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583019+0000","last_change":"2024-04-26T20:07:34.757547+0000","last_active":"2024-04-26T20:08:15.583019+0000","last_peered":"2024-04-26T20:08:15.583019+0000","last_clean":"2024-04-26T20:08:15.583019+0000","last_became_active":"2024-04-26T20:07:34.757180+0000","last_became_peered":"2024-04-26T20:07:34.757180+0000","last_unstale":"2024-04-26T20:08:15.583019+0000","last_undegraded":"2024-04-26T20:08:15.583019+0000","last_fullsized":"2024-04-26T20:08:15.583019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:32.365944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837548+0000","last_change":"2024-04-26T20:07:34.767363+0000","last_active":"2024-04-26T20:08:15.837548+0000","last_peered":"2024-04-26T20:08:15.837548+0000","last_clean":"2024-04-26T20:08:15.837548+0000","last_became_active":"2024-04-26T20:07:34.766909+0000","last_became_peered":"2024-04-26T20:07:34.766909+0000","last_unstale":"2024-04-26T20:08:15.837548+0000","last_undegraded":"2024-04-26T20:08:15.837548+0000","last_fullsized":"2024-04-26T20:08:15.837548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:32:31.500141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.836872+0000","last_change":"2024-04-26T20:07:38.823959+0000","last_active":"2024-04-26T20:08:15.836872+0000","last_peered":"2024-04-26T20:08:15.836872+0000","last_clean":"2024-04-26T20:08:15.836872+0000","last_became_active":"2024-04-26T20:07:38.823524+0000","last_became_peered":"2024-04-26T20:07:38.823524+0000","last_unstale":"2024-04-26T20:08:15.836872+0000","last_undegraded":"2024-04-26T20:08:15.836872+0000","last_fullsized":"2024-04-26T20:08:15.836872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:23.348376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.964850+0000","last_change":"2024-04-26T20:07:31.721324+0000","last_active":"2024-04-26T20:08:49.964850+0000","last_peered":"2024-04-26T20:08:49.964850+0000","last_clean":"2024-04-26T20:08:49.964850+0000","last_became_active":"2024-04-26T20:07:31.720140+0000","last_became_peered":"2024-04-26T20:07:31.720140+0000","last_unstale":"2024-04-26T20:08:49.964850+0000","last_undegraded":"2024-04-26T20:08:49.964850+0000","last_fullsized":"2024-04-26T20:08:49.964850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:27.636660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571683+0000","last_change":"2024-04-26T20:07:34.760387+0000","last_active":"2024-04-26T20:08:15.571683+0000","last_peered":"2024-04-26T20:08:15.571683+0000","last_clean":"2024-04-26T20:08:15.571683+0000","last_became_active":"2024-04-26T20:07:34.760042+0000","last_became_peered":"2024-04-26T20:07:34.760042+0000","last_unstale":"2024-04-26T20:08:15.571683+0000","last_undegraded":"2024-04-26T20:08:15.571683+0000","last_fullsized":"2024-04-26T20:08:15.571683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:39.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.587181+0000","last_change":"2024-04-26T20:08:15.587181+0000","last_active":"2024-04-26T20:08:15.587181+0000","last_peered":"2024-04-26T20:08:15.587181+0000","last_clean":"2024-04-26T20:08:15.587181+0000","last_became_active":"2024-04-26T20:08:15.586847+0000","last_became_peered":"2024-04-26T20:08:15.586847+0000","last_unstale":"2024-04-26T20:08:15.587181+0000","last_undegraded":"2024-04-26T20:08:15.587181+0000","last_fullsized":"2024-04-26T20:08:15.587181+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:12.495523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573157+0000","last_change":"2024-04-26T20:07:31.711623+0000","last_active":"2024-04-26T20:08:15.573157+0000","last_peered":"2024-04-26T20:08:15.573157+0000","last_clean":"2024-04-26T20:08:15.573157+0000","last_became_active":"2024-04-26T20:07:31.710488+0000","last_became_peered":"2024-04-26T20:07:31.710488+0000","last_unstale":"2024-04-26T20:08:15.573157+0000","last_undegraded":"2024-04-26T20:08:15.573157+0000","last_fullsized":"2024-04-26T20:08:15.573157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:04.747170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111751+0000","last_change":"2024-04-26T20:07:34.762383+0000","last_active":"2024-04-26T20:08:16.111751+0000","last_peered":"2024-04-26T20:08:16.111751+0000","last_clean":"2024-04-26T20:08:16.111751+0000","last_became_active":"2024-04-26T20:07:34.762012+0000","last_became_peered":"2024-04-26T20:07:34.762012+0000","last_unstale":"2024-04-26T20:08:16.111751+0000","last_undegraded":"2024-04-26T20:08:16.111751+0000","last_fullsized":"2024-04-26T20:08:16.111751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:27:16.715387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573094+0000","last_change":"2024-04-26T20:07:38.822443+0000","last_active":"2024-04-26T20:08:15.573094+0000","last_peered":"2024-04-26T20:08:15.573094+0000","last_clean":"2024-04-26T20:08:15.573094+0000","last_became_active":"2024-04-26T20:07:38.822046+0000","last_became_peered":"2024-04-26T20:07:38.822046+0000","last_unstale":"2024-04-26T20:08:15.573094+0000","last_undegraded":"2024-04-26T20:08:15.573094+0000","last_fullsized":"2024-04-26T20:08:15.573094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:26.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.961193+0000","last_change":"2024-04-26T20:07:31.715319+0000","last_active":"2024-04-26T20:08:49.961193+0000","last_peered":"2024-04-26T20:08:49.961193+0000","last_clean":"2024-04-26T20:08:49.961193+0000","last_became_active":"2024-04-26T20:07:31.714915+0000","last_became_peered":"2024-04-26T20:07:31.714915+0000","last_unstale":"2024-04-26T20:08:49.961193+0000","last_undegraded":"2024-04-26T20:08:49.961193+0000","last_fullsized":"2024-04-26T20:08:49.961193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:27.898652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374346+0000","last_change":"2024-04-26T20:07:34.753422+0000","last_active":"2024-04-26T20:08:16.374346+0000","last_peered":"2024-04-26T20:08:16.374346+0000","last_clean":"2024-04-26T20:08:16.374346+0000","last_became_active":"2024-04-26T20:07:34.753009+0000","last_became_peered":"2024-04-26T20:07:34.753009+0000","last_unstale":"2024-04-26T20:08:16.374346+0000","last_undegraded":"2024-04-26T20:08:16.374346+0000","last_fullsized":"2024-04-26T20:08:16.374346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.369715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573079+0000","last_change":"2024-04-26T20:07:38.830239+0000","last_active":"2024-04-26T20:08:15.573079+0000","last_peered":"2024-04-26T20:08:15.573079+0000","last_clean":"2024-04-26T20:08:15.573079+0000","last_became_active":"2024-04-26T20:07:38.829509+0000","last_became_peered":"2024-04-26T20:07:38.829509+0000","last_unstale":"2024-04-26T20:08:15.573079+0000","last_undegraded":"2024-04-26T20:08:15.573079+0000","last_fullsized":"2024-04-26T20:08:15.573079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:53.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571416+0000","last_change":"2024-04-26T20:08:14.576129+0000","last_active":"2024-04-26T20:08:15.571416+0000","last_peered":"2024-04-26T20:08:15.571416+0000","last_clean":"2024-04-26T20:08:15.571416+0000","last_became_active":"2024-04-26T20:08:14.573168+0000","last_became_peered":"2024-04-26T20:08:14.573168+0000","last_unstale":"2024-04-26T20:08:15.571416+0000","last_undegraded":"2024-04-26T20:08:15.571416+0000","last_fullsized":"2024-04-26T20:08:15.571416+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:38.234154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155226+0000","last_change":"2024-04-26T20:07:34.749270+0000","last_active":"2024-04-26T20:08:16.155226+0000","last_peered":"2024-04-26T20:08:16.155226+0000","last_clean":"2024-04-26T20:08:16.155226+0000","last_became_active":"2024-04-26T20:07:34.748585+0000","last_became_peered":"2024-04-26T20:07:34.748585+0000","last_unstale":"2024-04-26T20:08:16.155226+0000","last_undegraded":"2024-04-26T20:08:16.155226+0000","last_fullsized":"2024-04-26T20:08:16.155226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:03.091720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837290+0000","last_change":"2024-04-26T20:07:38.806247+0000","last_active":"2024-04-26T20:08:15.837290+0000","last_peered":"2024-04-26T20:08:15.837290+0000","last_clean":"2024-04-26T20:08:15.837290+0000","last_became_active":"2024-04-26T20:07:38.805748+0000","last_became_peered":"2024-04-26T20:07:38.805748+0000","last_unstale":"2024-04-26T20:08:15.837290+0000","last_undegraded":"2024-04-26T20:08:15.837290+0000","last_fullsized":"2024-04-26T20:08:15.837290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:16.962494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":65,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:11.287089+0000","last_change":"2024-04-26T20:07:31.713618+0000","last_active":"2024-04-26T20:09:11.287089+0000","last_peered":"2024-04-26T20:09:11.287089+0000","last_clean":"2024-04-26T20:09:11.287089+0000","last_became_active":"2024-04-26T20:07:31.713259+0000","last_became_peered":"2024-04-26T20:07:31.713259+0000","last_unstale":"2024-04-26T20:09:11.287089+0000","last_undegraded":"2024-04-26T20:09:11.287089+0000","last_fullsized":"2024-04-26T20:09:11.287089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:25.215087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568618+0000","last_change":"2024-04-26T20:08:14.585688+0000","last_active":"2024-04-26T20:08:15.568618+0000","last_peered":"2024-04-26T20:08:15.568618+0000","last_clean":"2024-04-26T20:08:15.568618+0000","last_became_active":"2024-04-26T20:08:14.585337+0000","last_became_peered":"2024-04-26T20:08:14.585337+0000","last_unstale":"2024-04-26T20:08:15.568618+0000","last_undegraded":"2024-04-26T20:08:15.568618+0000","last_fullsized":"2024-04-26T20:08:15.568618+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:03.515457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837120+0000","last_change":"2024-04-26T20:07:38.829772+0000","last_active":"2024-04-26T20:08:15.837120+0000","last_peered":"2024-04-26T20:08:15.837120+0000","last_clean":"2024-04-26T20:08:15.837120+0000","last_became_active":"2024-04-26T20:07:38.829327+0000","last_became_peered":"2024-04-26T20:07:38.829327+0000","last_unstale":"2024-04-26T20:08:15.837120+0000","last_undegraded":"2024-04-26T20:08:15.837120+0000","last_fullsized":"2024-04-26T20:08:15.837120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:37.252863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.965820+0000","last_change":"2024-04-26T20:07:31.728654+0000","last_active":"2024-04-26T20:08:49.965820+0000","last_peered":"2024-04-26T20:08:49.965820+0000","last_clean":"2024-04-26T20:08:49.965820+0000","last_became_active":"2024-04-26T20:07:31.728278+0000","last_became_peered":"2024-04-26T20:07:31.728278+0000","last_unstale":"2024-04-26T20:08:49.965820+0000","last_undegraded":"2024-04-26T20:08:49.965820+0000","last_fullsized":"2024-04-26T20:08:49.965820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155076+0000","last_change":"2024-04-26T20:07:34.739575+0000","last_active":"2024-04-26T20:08:16.155076+0000","last_peered":"2024-04-26T20:08:16.155076+0000","last_clean":"2024-04-26T20:08:16.155076+0000","last_became_active":"2024-04-26T20:07:34.739264+0000","last_became_peered":"2024-04-26T20:07:34.739264+0000","last_unstale":"2024-04-26T20:08:16.155076+0000","last_undegraded":"2024-04-26T20:08:16.155076+0000","last_fullsized":"2024-04-26T20:08:16.155076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:25.130064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571551+0000","last_change":"2024-04-26T20:07:38.822669+0000","last_active":"2024-04-26T20:08:15.571551+0000","last_peered":"2024-04-26T20:08:15.571551+0000","last_clean":"2024-04-26T20:08:15.571551+0000","last_became_active":"2024-04-26T20:07:38.822197+0000","last_became_peered":"2024-04-26T20:07:38.822197+0000","last_unstale":"2024-04-26T20:08:15.571551+0000","last_undegraded":"2024-04-26T20:08:15.571551+0000","last_fullsized":"2024-04-26T20:08:15.571551+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:56.782794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"95'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.967402+0000","last_change":"2024-04-26T20:07:31.723527+0000","last_active":"2024-04-26T20:08:49.967402+0000","last_peered":"2024-04-26T20:08:49.967402+0000","last_clean":"2024-04-26T20:08:49.967402+0000","last_became_active":"2024-04-26T20:07:31.722574+0000","last_became_peered":"2024-04-26T20:07:31.722574+0000","last_unstale":"2024-04-26T20:08:49.967402+0000","last_undegraded":"2024-04-26T20:08:49.967402+0000","last_fullsized":"2024-04-26T20:08:49.967402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.521613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571094+0000","last_change":"2024-04-26T20:07:34.771459+0000","last_active":"2024-04-26T20:08:15.571094+0000","last_peered":"2024-04-26T20:08:15.571094+0000","last_clean":"2024-04-26T20:08:15.571094+0000","last_became_active":"2024-04-26T20:07:34.771124+0000","last_became_peered":"2024-04-26T20:07:34.771124+0000","last_unstale":"2024-04-26T20:08:15.571094+0000","last_undegraded":"2024-04-26T20:08:15.571094+0000","last_fullsized":"2024-04-26T20:08:15.571094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.560440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111642+0000","last_change":"2024-04-26T20:07:38.838858+0000","last_active":"2024-04-26T20:08:16.111642+0000","last_peered":"2024-04-26T20:08:16.111642+0000","last_clean":"2024-04-26T20:08:16.111642+0000","last_became_active":"2024-04-26T20:07:38.838125+0000","last_became_peered":"2024-04-26T20:07:38.838125+0000","last_unstale":"2024-04-26T20:08:16.111642+0000","last_undegraded":"2024-04-26T20:08:16.111642+0000","last_fullsized":"2024-04-26T20:08:16.111642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:01.856633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"95'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154860+0000","last_change":"2024-04-26T20:07:31.710690+0000","last_active":"2024-04-26T20:08:16.154860+0000","last_peered":"2024-04-26T20:08:16.154860+0000","last_clean":"2024-04-26T20:08:16.154860+0000","last_became_active":"2024-04-26T20:07:31.710353+0000","last_became_peered":"2024-04-26T20:07:31.710353+0000","last_unstale":"2024-04-26T20:08:16.154860+0000","last_undegraded":"2024-04-26T20:08:16.154860+0000","last_fullsized":"2024-04-26T20:08:16.154860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:47.442673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.578985+0000","last_change":"2024-04-26T20:07:34.776023+0000","last_active":"2024-04-26T20:08:15.578985+0000","last_peered":"2024-04-26T20:08:15.578985+0000","last_clean":"2024-04-26T20:08:15.578985+0000","last_became_active":"2024-04-26T20:07:34.775571+0000","last_became_peered":"2024-04-26T20:07:34.775571+0000","last_unstale":"2024-04-26T20:08:15.578985+0000","last_undegraded":"2024-04-26T20:08:15.578985+0000","last_fullsized":"2024-04-26T20:08:15.578985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:40.573046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.570964+0000","last_change":"2024-04-26T20:07:38.820636+0000","last_active":"2024-04-26T20:08:15.570964+0000","last_peered":"2024-04-26T20:08:15.570964+0000","last_clean":"2024-04-26T20:08:15.570964+0000","last_became_active":"2024-04-26T20:07:38.819390+0000","last_became_peered":"2024-04-26T20:07:38.819390+0000","last_unstale":"2024-04-26T20:08:15.570964+0000","last_undegraded":"2024-04-26T20:08:15.570964+0000","last_fullsized":"2024-04-26T20:08:15.570964+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:58.793479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'3","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:11.286188+0000","last_change":"2024-04-26T20:07:31.724192+0000","last_active":"2024-04-26T20:09:11.286188+0000","last_peered":"2024-04-26T20:09:11.286188+0000","last_clean":"2024-04-26T20:09:11.286188+0000","last_became_active":"2024-04-26T20:07:31.723816+0000","last_became_peered":"2024-04-26T20:07:31.723816+0000","last_unstale":"2024-04-26T20:09:11.286188+0000","last_undegraded":"2024-04-26T20:09:11.286188+0000","last_fullsized":"2024-04-26T20:09:11.286188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:31.707282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374611+0000","last_change":"2024-04-26T20:07:34.759027+0000","last_active":"2024-04-26T20:08:16.374611+0000","last_peered":"2024-04-26T20:08:16.374611+0000","last_clean":"2024-04-26T20:08:16.374611+0000","last_became_active":"2024-04-26T20:07:34.758689+0000","last_became_peered":"2024-04-26T20:07:34.758689+0000","last_unstale":"2024-04-26T20:08:16.374611+0000","last_undegraded":"2024-04-26T20:08:16.374611+0000","last_fullsized":"2024-04-26T20:08:16.374611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:29.110222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.579328+0000","last_change":"2024-04-26T20:07:38.819000+0000","last_active":"2024-04-26T20:08:15.579328+0000","last_peered":"2024-04-26T20:08:15.579328+0000","last_clean":"2024-04-26T20:08:15.579328+0000","last_became_active":"2024-04-26T20:07:38.818526+0000","last_became_peered":"2024-04-26T20:07:38.818526+0000","last_unstale":"2024-04-26T20:08:15.579328+0000","last_undegraded":"2024-04-26T20:08:15.579328+0000","last_fullsized":"2024-04-26T20:08:15.579328+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:28.830096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940017+0000","last_change":"2024-04-26T20:07:31.712952+0000","last_active":"2024-04-26T20:08:48.940017+0000","last_peered":"2024-04-26T20:08:48.940017+0000","last_clean":"2024-04-26T20:08:48.940017+0000","last_became_active":"2024-04-26T20:07:31.712081+0000","last_became_peered":"2024-04-26T20:07:31.712081+0000","last_unstale":"2024-04-26T20:08:48.940017+0000","last_undegraded":"2024-04-26T20:08:48.940017+0000","last_fullsized":"2024-04-26T20:08:48.940017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:46.923536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111838+0000","last_change":"2024-04-26T20:08:14.585054+0000","last_active":"2024-04-26T20:08:16.111838+0000","last_peered":"2024-04-26T20:08:16.111838+0000","last_clean":"2024-04-26T20:08:16.111838+0000","last_became_active":"2024-04-26T20:08:14.584525+0000","last_became_peered":"2024-04-26T20:08:14.584525+0000","last_unstale":"2024-04-26T20:08:16.111838+0000","last_undegraded":"2024-04-26T20:08:16.111838+0000","last_fullsized":"2024-04-26T20:08:16.111838+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:58.518819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573478+0000","last_change":"2024-04-26T20:07:38.822520+0000","last_active":"2024-04-26T20:08:15.573478+0000","last_peered":"2024-04-26T20:08:15.573478+0000","last_clean":"2024-04-26T20:08:15.573478+0000","last_became_active":"2024-04-26T20:07:38.822056+0000","last_became_peered":"2024-04-26T20:07:38.822056+0000","last_unstale":"2024-04-26T20:08:15.573478+0000","last_undegraded":"2024-04-26T20:08:15.573478+0000","last_fullsized":"2024-04-26T20:08:15.573478+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.324996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"95'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940181+0000","last_change":"2024-04-26T20:07:31.724015+0000","last_active":"2024-04-26T20:08:48.940181+0000","last_peered":"2024-04-26T20:08:48.940181+0000","last_clean":"2024-04-26T20:08:48.940181+0000","last_became_active":"2024-04-26T20:07:31.723686+0000","last_became_peered":"2024-04-26T20:07:31.723686+0000","last_unstale":"2024-04-26T20:08:48.940181+0000","last_undegraded":"2024-04-26T20:08:48.940181+0000","last_fullsized":"2024-04-26T20:08:48.940181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:55.925280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374527+0000","last_change":"2024-04-26T20:07:34.750276+0000","last_active":"2024-04-26T20:08:16.374527+0000","last_peered":"2024-04-26T20:08:16.374527+0000","last_clean":"2024-04-26T20:08:16.374527+0000","last_became_active":"2024-04-26T20:07:34.749716+0000","last_became_peered":"2024-04-26T20:07:34.749716+0000","last_unstale":"2024-04-26T20:08:16.374527+0000","last_undegraded":"2024-04-26T20:08:16.374527+0000","last_fullsized":"2024-04-26T20:08:16.374527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:22.429780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.579105+0000","last_change":"2024-04-26T20:07:38.815636+0000","last_active":"2024-04-26T20:08:15.579105+0000","last_peered":"2024-04-26T20:08:15.579105+0000","last_clean":"2024-04-26T20:08:15.579105+0000","last_became_active":"2024-04-26T20:07:38.815278+0000","last_became_peered":"2024-04-26T20:07:38.815278+0000","last_unstale":"2024-04-26T20:08:15.579105+0000","last_undegraded":"2024-04-26T20:08:15.579105+0000","last_fullsized":"2024-04-26T20:08:15.579105+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:27.522132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'30","reported_seq":112,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:12.289544+0000","last_change":"2024-04-26T20:07:31.711393+0000","last_active":"2024-04-26T20:09:12.289544+0000","last_peered":"2024-04-26T20:09:12.289544+0000","last_clean":"2024-04-26T20:09:12.289544+0000","last_became_active":"2024-04-26T20:07:31.710998+0000","last_became_peered":"2024-04-26T20:07:31.710998+0000","last_unstale":"2024-04-26T20:09:12.289544+0000","last_undegraded":"2024-04-26T20:09:12.289544+0000","last_fullsized":"2024-04-26T20:09:12.289544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:00.674950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52480,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":47,"num_read_kb":74,"num_write":31,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568876+0000","last_change":"2024-04-26T20:07:34.752774+0000","last_active":"2024-04-26T20:08:15.568876+0000","last_peered":"2024-04-26T20:08:15.568876+0000","last_clean":"2024-04-26T20:08:15.568876+0000","last_became_active":"2024-04-26T20:07:34.752471+0000","last_became_peered":"2024-04-26T20:07:34.752471+0000","last_unstale":"2024-04-26T20:08:15.568876+0000","last_undegraded":"2024-04-26T20:08:15.568876+0000","last_fullsized":"2024-04-26T20:08:15.568876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.646231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155471+0000","last_change":"2024-04-26T20:07:38.830322+0000","last_active":"2024-04-26T20:08:16.155471+0000","last_peered":"2024-04-26T20:08:16.155471+0000","last_clean":"2024-04-26T20:08:16.155471+0000","last_became_active":"2024-04-26T20:07:38.829910+0000","last_became_peered":"2024-04-26T20:07:38.829910+0000","last_unstale":"2024-04-26T20:08:16.155471+0000","last_undegraded":"2024-04-26T20:08:16.155471+0000","last_fullsized":"2024-04-26T20:08:16.155471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:01.979793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.965190+0000","last_change":"2024-04-26T20:07:31.712054+0000","last_active":"2024-04-26T20:08:49.965190+0000","last_peered":"2024-04-26T20:08:49.965190+0000","last_clean":"2024-04-26T20:08:49.965190+0000","last_became_active":"2024-04-26T20:07:31.711618+0000","last_became_peered":"2024-04-26T20:07:31.711618+0000","last_unstale":"2024-04-26T20:08:49.965190+0000","last_undegraded":"2024-04-26T20:08:49.965190+0000","last_fullsized":"2024-04-26T20:08:49.965190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:46.465777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584167+0000","last_change":"2024-04-26T20:07:34.772993+0000","last_active":"2024-04-26T20:08:15.584167+0000","last_peered":"2024-04-26T20:08:15.584167+0000","last_clean":"2024-04-26T20:08:15.584167+0000","last_became_active":"2024-04-26T20:07:34.772587+0000","last_became_peered":"2024-04-26T20:07:34.772587+0000","last_unstale":"2024-04-26T20:08:15.584167+0000","last_undegraded":"2024-04-26T20:08:15.584167+0000","last_fullsized":"2024-04-26T20:08:15.584167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:38.459869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.588307+0000","last_change":"2024-04-26T20:08:15.588307+0000","last_active":"2024-04-26T20:08:15.588307+0000","last_peered":"2024-04-26T20:08:15.588307+0000","last_clean":"2024-04-26T20:08:15.588307+0000","last_became_active":"2024-04-26T20:08:15.587978+0000","last_became_peered":"2024-04-26T20:08:15.587978+0000","last_unstale":"2024-04-26T20:08:15.588307+0000","last_undegraded":"2024-04-26T20:08:15.588307+0000","last_fullsized":"2024-04-26T20:08:15.588307+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:36.491892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"95'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940179+0000","last_change":"2024-04-26T20:07:31.720938+0000","last_active":"2024-04-26T20:08:48.940179+0000","last_peered":"2024-04-26T20:08:48.940179+0000","last_clean":"2024-04-26T20:08:48.940179+0000","last_became_active":"2024-04-26T20:07:31.720579+0000","last_became_peered":"2024-04-26T20:07:31.720579+0000","last_unstale":"2024-04-26T20:08:48.940179+0000","last_undegraded":"2024-04-26T20:08:48.940179+0000","last_fullsized":"2024-04-26T20:08:48.940179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:55.737976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569252+0000","last_change":"2024-04-26T20:07:34.764802+0000","last_active":"2024-04-26T20:08:15.569252+0000","last_peered":"2024-04-26T20:08:15.569252+0000","last_clean":"2024-04-26T20:08:15.569252+0000","last_became_active":"2024-04-26T20:07:34.763468+0000","last_became_peered":"2024-04-26T20:07:34.763468+0000","last_unstale":"2024-04-26T20:08:15.569252+0000","last_undegraded":"2024-04-26T20:08:15.569252+0000","last_fullsized":"2024-04-26T20:08:15.569252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:23.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583848+0000","last_change":"2024-04-26T20:07:38.830392+0000","last_active":"2024-04-26T20:08:15.583848+0000","last_peered":"2024-04-26T20:08:15.583848+0000","last_clean":"2024-04-26T20:08:15.583848+0000","last_became_active":"2024-04-26T20:07:38.822964+0000","last_became_peered":"2024-04-26T20:07:38.822964+0000","last_unstale":"2024-04-26T20:08:15.583848+0000","last_undegraded":"2024-04-26T20:08:15.583848+0000","last_fullsized":"2024-04-26T20:08:15.583848+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:17.992368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'3","reported_seq":59,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:10.296275+0000","last_change":"2024-04-26T20:07:31.710567+0000","last_active":"2024-04-26T20:09:10.296275+0000","last_peered":"2024-04-26T20:09:10.296275+0000","last_clean":"2024-04-26T20:09:10.296275+0000","last_became_active":"2024-04-26T20:07:31.710197+0000","last_became_peered":"2024-04-26T20:07:31.710197+0000","last_unstale":"2024-04-26T20:09:10.296275+0000","last_undegraded":"2024-04-26T20:09:10.296275+0000","last_fullsized":"2024-04-26T20:09:10.296275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:10.648320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569128+0000","last_change":"2024-04-26T20:07:34.743237+0000","last_active":"2024-04-26T20:08:15.569128+0000","last_peered":"2024-04-26T20:08:15.569128+0000","last_clean":"2024-04-26T20:08:15.569128+0000","last_became_active":"2024-04-26T20:07:34.742146+0000","last_became_peered":"2024-04-26T20:07:34.742146+0000","last_unstale":"2024-04-26T20:08:15.569128+0000","last_undegraded":"2024-04-26T20:08:15.569128+0000","last_fullsized":"2024-04-26T20:08:15.569128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:15.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.591942+0000","last_change":"2024-04-26T20:08:15.591942+0000","last_active":"2024-04-26T20:08:15.591942+0000","last_peered":"2024-04-26T20:08:15.591942+0000","last_clean":"2024-04-26T20:08:15.591942+0000","last_became_active":"2024-04-26T20:08:15.590842+0000","last_became_peered":"2024-04-26T20:08:15.590842+0000","last_unstale":"2024-04-26T20:08:15.591942+0000","last_undegraded":"2024-04-26T20:08:15.591942+0000","last_fullsized":"2024-04-26T20:08:15.591942+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:44.344152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"95'15","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.964519+0000","last_change":"2024-04-26T20:07:31.716877+0000","last_active":"2024-04-26T20:08:49.964519+0000","last_peered":"2024-04-26T20:08:49.964519+0000","last_clean":"2024-04-26T20:08:49.964519+0000","last_became_active":"2024-04-26T20:07:31.716560+0000","last_became_peered":"2024-04-26T20:07:31.716560+0000","last_unstale":"2024-04-26T20:08:49.964519+0000","last_undegraded":"2024-04-26T20:08:49.964519+0000","last_fullsized":"2024-04-26T20:08:49.964519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:08:47.680991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154746+0000","last_change":"2024-04-26T20:07:34.749893+0000","last_active":"2024-04-26T20:08:16.154746+0000","last_peered":"2024-04-26T20:08:16.154746+0000","last_clean":"2024-04-26T20:08:16.154746+0000","last_became_active":"2024-04-26T20:07:34.748535+0000","last_became_peered":"2024-04-26T20:07:34.748535+0000","last_unstale":"2024-04-26T20:08:16.154746+0000","last_undegraded":"2024-04-26T20:08:16.154746+0000","last_fullsized":"2024-04-26T20:08:16.154746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:32.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154813+0000","last_change":"2024-04-26T20:07:38.824046+0000","last_active":"2024-04-26T20:08:16.154813+0000","last_peered":"2024-04-26T20:08:16.154813+0000","last_clean":"2024-04-26T20:08:16.154813+0000","last_became_active":"2024-04-26T20:07:38.823549+0000","last_became_peered":"2024-04-26T20:07:38.823549+0000","last_unstale":"2024-04-26T20:08:16.154813+0000","last_undegraded":"2024-04-26T20:08:16.154813+0000","last_fullsized":"2024-04-26T20:08:16.154813+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:26.218201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:50.955777+0000","last_change":"2024-04-26T20:07:31.711968+0000","last_active":"2024-04-26T20:08:50.955777+0000","last_peered":"2024-04-26T20:08:50.955777+0000","last_clean":"2024-04-26T20:08:50.955777+0000","last_became_active":"2024-04-26T20:07:31.710982+0000","last_became_peered":"2024-04-26T20:07:31.710982+0000","last_unstale":"2024-04-26T20:08:50.955777+0000","last_undegraded":"2024-04-26T20:08:50.955777+0000","last_fullsized":"2024-04-26T20:08:50.955777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:29.526536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571515+0000","last_change":"2024-04-26T20:07:34.758101+0000","last_active":"2024-04-26T20:08:15.571515+0000","last_peered":"2024-04-26T20:08:15.571515+0000","last_clean":"2024-04-26T20:08:15.571515+0000","last_became_active":"2024-04-26T20:07:34.757747+0000","last_became_peered":"2024-04-26T20:07:34.757747+0000","last_unstale":"2024-04-26T20:08:15.571515+0000","last_undegraded":"2024-04-26T20:08:15.571515+0000","last_fullsized":"2024-04-26T20:08:15.571515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.337040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568753+0000","last_change":"2024-04-26T20:07:39.312976+0000","last_active":"2024-04-26T20:08:15.568753+0000","last_peered":"2024-04-26T20:08:15.568753+0000","last_clean":"2024-04-26T20:08:15.568753+0000","last_became_active":"2024-04-26T20:07:39.312443+0000","last_became_peered":"2024-04-26T20:07:39.312443+0000","last_unstale":"2024-04-26T20:08:15.568753+0000","last_undegraded":"2024-04-26T20:08:15.568753+0000","last_fullsized":"2024-04-26T20:08:15.568753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:23.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.581507+0000","last_change":"2024-04-26T20:08:14.587592+0000","last_active":"2024-04-26T20:08:15.581507+0000","last_peered":"2024-04-26T20:08:15.581507+0000","last_clean":"2024-04-26T20:08:15.581507+0000","last_became_active":"2024-04-26T20:08:14.587157+0000","last_became_peered":"2024-04-26T20:08:14.587157+0000","last_unstale":"2024-04-26T20:08:15.581507+0000","last_undegraded":"2024-04-26T20:08:15.581507+0000","last_fullsized":"2024-04-26T20:08:15.581507+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:22.184416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.575875+0000","last_change":"2024-04-26T20:07:34.764376+0000","last_active":"2024-04-26T20:08:15.575875+0000","last_peered":"2024-04-26T20:08:15.575875+0000","last_clean":"2024-04-26T20:08:15.575875+0000","last_became_active":"2024-04-26T20:07:34.764040+0000","last_became_peered":"2024-04-26T20:07:34.764040+0000","last_unstale":"2024-04-26T20:08:15.575875+0000","last_undegraded":"2024-04-26T20:08:15.575875+0000","last_fullsized":"2024-04-26T20:08:15.575875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:43.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583488+0000","last_change":"2024-04-26T20:07:38.837075+0000","last_active":"2024-04-26T20:08:15.583488+0000","last_peered":"2024-04-26T20:08:15.583488+0000","last_clean":"2024-04-26T20:08:15.583488+0000","last_became_active":"2024-04-26T20:07:38.835912+0000","last_became_peered":"2024-04-26T20:07:38.835912+0000","last_unstale":"2024-04-26T20:08:15.583488+0000","last_undegraded":"2024-04-26T20:08:15.583488+0000","last_fullsized":"2024-04-26T20:08:15.583488+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.853262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569209+0000","last_change":"2024-04-26T20:07:31.728087+0000","last_active":"2024-04-26T20:08:15.569209+0000","last_peered":"2024-04-26T20:08:15.569209+0000","last_clean":"2024-04-26T20:08:15.569209+0000","last_became_active":"2024-04-26T20:07:31.727688+0000","last_became_peered":"2024-04-26T20:07:31.727688+0000","last_unstale":"2024-04-26T20:08:15.569209+0000","last_undegraded":"2024-04-26T20:08:15.569209+0000","last_fullsized":"2024-04-26T20:08:15.569209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:16.220049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.575695+0000","last_change":"2024-04-26T20:07:34.765161+0000","last_active":"2024-04-26T20:08:15.575695+0000","last_peered":"2024-04-26T20:08:15.575695+0000","last_clean":"2024-04-26T20:08:15.575695+0000","last_became_active":"2024-04-26T20:07:34.764789+0000","last_became_peered":"2024-04-26T20:07:34.764789+0000","last_unstale":"2024-04-26T20:08:15.575695+0000","last_undegraded":"2024-04-26T20:08:15.575695+0000","last_fullsized":"2024-04-26T20:08:15.575695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:52.417157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584352+0000","last_change":"2024-04-26T20:07:39.313159+0000","last_active":"2024-04-26T20:08:15.584352+0000","last_peered":"2024-04-26T20:08:15.584352+0000","last_clean":"2024-04-26T20:08:15.584352+0000","last_became_active":"2024-04-26T20:07:39.312637+0000","last_became_peered":"2024-04-26T20:07:39.312637+0000","last_unstale":"2024-04-26T20:08:15.584352+0000","last_undegraded":"2024-04-26T20:08:15.584352+0000","last_fullsized":"2024-04-26T20:08:15.584352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:10.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155127+0000","last_change":"2024-04-26T20:08:14.586352+0000","last_active":"2024-04-26T20:08:16.155127+0000","last_peered":"2024-04-26T20:08:16.155127+0000","last_clean":"2024-04-26T20:08:16.155127+0000","last_became_active":"2024-04-26T20:08:14.585975+0000","last_became_peered":"2024-04-26T20:08:14.585975+0000","last_unstale":"2024-04-26T20:08:16.155127+0000","last_undegraded":"2024-04-26T20:08:16.155127+0000","last_fullsized":"2024-04-26T20:08:16.155127+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:27.676683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'4","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:11.297935+0000","last_change":"2024-04-26T20:07:31.726272+0000","last_active":"2024-04-26T20:09:11.297935+0000","last_peered":"2024-04-26T20:09:11.297935+0000","last_clean":"2024-04-26T20:09:11.297935+0000","last_became_active":"2024-04-26T20:07:31.725889+0000","last_became_peered":"2024-04-26T20:07:31.725889+0000","last_unstale":"2024-04-26T20:09:11.297935+0000","last_undegraded":"2024-04-26T20:09:11.297935+0000","last_fullsized":"2024-04-26T20:09:11.297935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:03.091138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569447+0000","last_change":"2024-04-26T20:07:38.815171+0000","last_active":"2024-04-26T20:08:15.569447+0000","last_peered":"2024-04-26T20:08:15.569447+0000","last_clean":"2024-04-26T20:08:15.569447+0000","last_became_active":"2024-04-26T20:07:38.814769+0000","last_became_peered":"2024-04-26T20:07:38.814769+0000","last_unstale":"2024-04-26T20:08:15.569447+0000","last_undegraded":"2024-04-26T20:08:15.569447+0000","last_fullsized":"2024-04-26T20:08:15.569447+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571131+0000","last_change":"2024-04-26T20:07:34.766538+0000","last_active":"2024-04-26T20:08:15.571131+0000","last_peered":"2024-04-26T20:08:15.571131+0000","last_clean":"2024-04-26T20:08:15.571131+0000","last_became_active":"2024-04-26T20:07:34.761651+0000","last_became_peered":"2024-04-26T20:07:34.761651+0000","last_unstale":"2024-04-26T20:08:15.571131+0000","last_undegraded":"2024-04-26T20:08:15.571131+0000","last_fullsized":"2024-04-26T20:08:15.571131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:11.213046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'3","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:53.256297+0000","last_change":"2024-04-26T20:07:31.720351+0000","last_active":"2024-04-26T20:08:53.256297+0000","last_peered":"2024-04-26T20:08:53.256297+0000","last_clean":"2024-04-26T20:08:53.256297+0000","last_became_active":"2024-04-26T20:07:31.719966+0000","last_became_peered":"2024-04-26T20:07:31.719966+0000","last_unstale":"2024-04-26T20:08:53.256297+0000","last_undegraded":"2024-04-26T20:08:53.256297+0000","last_fullsized":"2024-04-26T20:08:53.256297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:17.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569035+0000","last_change":"2024-04-26T20:05:56.337783+0000","last_active":"2024-04-26T20:08:15.569035+0000","last_peered":"2024-04-26T20:08:15.569035+0000","last_clean":"2024-04-26T20:08:15.569035+0000","last_became_active":"2024-04-26T20:05:56.337301+0000","last_became_peered":"2024-04-26T20:05:56.337301+0000","last_unstale":"2024-04-26T20:08:15.569035+0000","last_undegraded":"2024-04-26T20:08:15.569035+0000","last_fullsized":"2024-04-26T20:08:15.569035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_clean_scrub_stamp":"2024-04-26T20:01:58.332403+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:45.808606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155395+0000","last_change":"2024-04-26T20:07:39.313210+0000","last_active":"2024-04-26T20:08:16.155395+0000","last_peered":"2024-04-26T20:08:16.155395+0000","last_clean":"2024-04-26T20:08:16.155395+0000","last_became_active":"2024-04-26T20:07:39.312575+0000","last_became_peered":"2024-04-26T20:07:39.312575+0000","last_unstale":"2024-04-26T20:08:16.155395+0000","last_undegraded":"2024-04-26T20:08:16.155395+0000","last_fullsized":"2024-04-26T20:08:16.155395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:02.877951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.582422+0000","last_change":"2024-04-26T20:07:34.759474+0000","last_active":"2024-04-26T20:08:15.582422+0000","last_peered":"2024-04-26T20:08:15.582422+0000","last_clean":"2024-04-26T20:08:15.582422+0000","last_became_active":"2024-04-26T20:07:34.759127+0000","last_became_peered":"2024-04-26T20:07:34.759127+0000","last_unstale":"2024-04-26T20:08:15.582422+0000","last_undegraded":"2024-04-26T20:08:15.582422+0000","last_fullsized":"2024-04-26T20:08:15.582422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.874866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.957623+0000","last_change":"2024-04-26T20:07:31.710633+0000","last_active":"2024-04-26T20:08:49.957623+0000","last_peered":"2024-04-26T20:08:49.957623+0000","last_clean":"2024-04-26T20:08:49.957623+0000","last_became_active":"2024-04-26T20:07:31.710290+0000","last_became_peered":"2024-04-26T20:07:31.710290+0000","last_unstale":"2024-04-26T20:08:49.957623+0000","last_undegraded":"2024-04-26T20:08:49.957623+0000","last_fullsized":"2024-04-26T20:08:49.957623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:18.114770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571337+0000","last_change":"2024-04-26T20:07:38.829768+0000","last_active":"2024-04-26T20:08:15.571337+0000","last_peered":"2024-04-26T20:08:15.571337+0000","last_clean":"2024-04-26T20:08:15.571337+0000","last_became_active":"2024-04-26T20:07:38.829364+0000","last_became_peered":"2024-04-26T20:07:38.829364+0000","last_unstale":"2024-04-26T20:08:15.571337+0000","last_undegraded":"2024-04-26T20:08:15.571337+0000","last_fullsized":"2024-04-26T20:08:15.571337+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:56.887062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"95'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.937916+0000","last_change":"2024-04-26T20:07:31.720296+0000","last_active":"2024-04-26T20:08:48.937916+0000","last_peered":"2024-04-26T20:08:48.937916+0000","last_clean":"2024-04-26T20:08:48.937916+0000","last_became_active":"2024-04-26T20:07:31.719918+0000","last_became_peered":"2024-04-26T20:07:31.719918+0000","last_unstale":"2024-04-26T20:08:48.937916+0000","last_undegraded":"2024-04-26T20:08:48.937916+0000","last_fullsized":"2024-04-26T20:08:48.937916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:43.783104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583862+0000","last_change":"2024-04-26T20:07:34.770634+0000","last_active":"2024-04-26T20:08:15.583862+0000","last_peered":"2024-04-26T20:08:15.583862+0000","last_clean":"2024-04-26T20:08:15.583862+0000","last_became_active":"2024-04-26T20:07:34.770170+0000","last_became_peered":"2024-04-26T20:07:34.770170+0000","last_unstale":"2024-04-26T20:08:15.583862+0000","last_undegraded":"2024-04-26T20:08:15.583862+0000","last_fullsized":"2024-04-26T20:08:15.583862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:21.528735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571088+0000","last_change":"2024-04-26T20:07:39.317345+0000","last_active":"2024-04-26T20:08:15.571088+0000","last_peered":"2024-04-26T20:08:15.571088+0000","last_clean":"2024-04-26T20:08:15.571088+0000","last_became_active":"2024-04-26T20:07:39.315998+0000","last_became_peered":"2024-04-26T20:07:39.315998+0000","last_unstale":"2024-04-26T20:08:15.571088+0000","last_undegraded":"2024-04-26T20:08:15.571088+0000","last_fullsized":"2024-04-26T20:08:15.571088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:18.715213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:07.746531+0000","last_change":"2024-04-26T20:07:31.729195+0000","last_active":"2024-04-26T20:09:07.746531+0000","last_peered":"2024-04-26T20:09:07.746531+0000","last_clean":"2024-04-26T20:09:07.746531+0000","last_became_active":"2024-04-26T20:07:31.728746+0000","last_became_peered":"2024-04-26T20:07:31.728746+0000","last_unstale":"2024-04-26T20:09:07.746531+0000","last_undegraded":"2024-04-26T20:09:07.746531+0000","last_fullsized":"2024-04-26T20:09:07.746531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:34.443091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.572210+0000","last_change":"2024-04-26T20:07:34.765013+0000","last_active":"2024-04-26T20:08:15.572210+0000","last_peered":"2024-04-26T20:08:15.572210+0000","last_clean":"2024-04-26T20:08:15.572210+0000","last_became_active":"2024-04-26T20:07:34.764597+0000","last_became_peered":"2024-04-26T20:07:34.764597+0000","last_unstale":"2024-04-26T20:08:15.572210+0000","last_undegraded":"2024-04-26T20:08:15.572210+0000","last_fullsized":"2024-04-26T20:08:15.572210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:51.513026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.836755+0000","last_change":"2024-04-26T20:07:38.805910+0000","last_active":"2024-04-26T20:08:15.836755+0000","last_peered":"2024-04-26T20:08:15.836755+0000","last_clean":"2024-04-26T20:08:15.836755+0000","last_became_active":"2024-04-26T20:07:38.804580+0000","last_became_peered":"2024-04-26T20:07:38.804580+0000","last_unstale":"2024-04-26T20:08:15.836755+0000","last_undegraded":"2024-04-26T20:08:15.836755+0000","last_fullsized":"2024-04-26T20:08:15.836755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:41.321084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.942068+0000","last_change":"2024-04-26T20:07:31.915771+0000","last_active":"2024-04-26T20:08:48.942068+0000","last_peered":"2024-04-26T20:08:48.942068+0000","last_clean":"2024-04-26T20:08:48.942068+0000","last_became_active":"2024-04-26T20:07:31.915422+0000","last_became_peered":"2024-04-26T20:07:31.915422+0000","last_unstale":"2024-04-26T20:08:48.942068+0000","last_undegraded":"2024-04-26T20:08:48.942068+0000","last_fullsized":"2024-04-26T20:08:48.942068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:33.309270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111862+0000","last_change":"2024-04-26T20:07:34.775090+0000","last_active":"2024-04-26T20:08:16.111862+0000","last_peered":"2024-04-26T20:08:16.111862+0000","last_clean":"2024-04-26T20:08:16.111862+0000","last_became_active":"2024-04-26T20:07:34.774431+0000","last_became_peered":"2024-04-26T20:07:34.774431+0000","last_unstale":"2024-04-26T20:08:16.111862+0000","last_undegraded":"2024-04-26T20:08:16.111862+0000","last_fullsized":"2024-04-26T20:08:16.111862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:47.554047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573424+0000","last_change":"2024-04-26T20:07:39.313079+0000","last_active":"2024-04-26T20:08:15.573424+0000","last_peered":"2024-04-26T20:08:15.573424+0000","last_clean":"2024-04-26T20:08:15.573424+0000","last_became_active":"2024-04-26T20:07:39.312603+0000","last_became_peered":"2024-04-26T20:07:39.312603+0000","last_unstale":"2024-04-26T20:08:15.573424+0000","last_undegraded":"2024-04-26T20:08:15.573424+0000","last_fullsized":"2024-04-26T20:08:15.573424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:45.717375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940438+0000","last_change":"2024-04-26T20:07:31.715373+0000","last_active":"2024-04-26T20:08:48.940438+0000","last_peered":"2024-04-26T20:08:48.940438+0000","last_clean":"2024-04-26T20:08:48.940438+0000","last_became_active":"2024-04-26T20:07:31.714973+0000","last_became_peered":"2024-04-26T20:07:31.714973+0000","last_unstale":"2024-04-26T20:08:48.940438+0000","last_undegraded":"2024-04-26T20:08:48.940438+0000","last_fullsized":"2024-04-26T20:08:48.940438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.772671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568431+0000","last_change":"2024-04-26T20:07:34.753619+0000","last_active":"2024-04-26T20:08:15.568431+0000","last_peered":"2024-04-26T20:08:15.568431+0000","last_clean":"2024-04-26T20:08:15.568431+0000","last_became_active":"2024-04-26T20:07:34.753263+0000","last_became_peered":"2024-04-26T20:07:34.753263+0000","last_unstale":"2024-04-26T20:08:15.568431+0000","last_undegraded":"2024-04-26T20:08:15.568431+0000","last_fullsized":"2024-04-26T20:08:15.568431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.358153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.840243+0000","last_change":"2024-04-26T20:08:15.840243+0000","last_active":"2024-04-26T20:08:15.840243+0000","last_peered":"2024-04-26T20:08:15.840243+0000","last_clean":"2024-04-26T20:08:15.840243+0000","last_became_active":"2024-04-26T20:08:15.838891+0000","last_became_peered":"2024-04-26T20:08:15.838891+0000","last_unstale":"2024-04-26T20:08:15.840243+0000","last_undegraded":"2024-04-26T20:08:15.840243+0000","last_fullsized":"2024-04-26T20:08:15.840243+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:15.018199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"95'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.941460+0000","last_change":"2024-04-26T20:07:31.722516+0000","last_active":"2024-04-26T20:08:48.941460+0000","last_peered":"2024-04-26T20:08:48.941460+0000","last_clean":"2024-04-26T20:08:48.941460+0000","last_became_active":"2024-04-26T20:07:31.722096+0000","last_became_peered":"2024-04-26T20:07:31.722096+0000","last_unstale":"2024-04-26T20:08:48.941460+0000","last_undegraded":"2024-04-26T20:08:48.941460+0000","last_fullsized":"2024-04-26T20:08:48.941460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:57.693947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571173+0000","last_change":"2024-04-26T20:07:34.768754+0000","last_active":"2024-04-26T20:08:15.571173+0000","last_peered":"2024-04-26T20:08:15.571173+0000","last_clean":"2024-04-26T20:08:15.571173+0000","last_became_active":"2024-04-26T20:07:34.768405+0000","last_became_peered":"2024-04-26T20:07:34.768405+0000","last_unstale":"2024-04-26T20:08:15.571173+0000","last_undegraded":"2024-04-26T20:08:15.571173+0000","last_fullsized":"2024-04-26T20:08:15.571173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:08.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573877+0000","last_change":"2024-04-26T20:07:38.828804+0000","last_active":"2024-04-26T20:08:15.573877+0000","last_peered":"2024-04-26T20:08:15.573877+0000","last_clean":"2024-04-26T20:08:15.573877+0000","last_became_active":"2024-04-26T20:07:38.828452+0000","last_became_peered":"2024-04-26T20:07:38.828452+0000","last_unstale":"2024-04-26T20:08:15.573877+0000","last_undegraded":"2024-04-26T20:08:15.573877+0000","last_fullsized":"2024-04-26T20:08:15.573877+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:29.905102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837033+0000","last_change":"2024-04-26T20:07:31.710696+0000","last_active":"2024-04-26T20:08:15.837033+0000","last_peered":"2024-04-26T20:08:15.837033+0000","last_clean":"2024-04-26T20:08:15.837033+0000","last_became_active":"2024-04-26T20:07:31.710039+0000","last_became_peered":"2024-04-26T20:07:31.710039+0000","last_unstale":"2024-04-26T20:08:15.837033+0000","last_undegraded":"2024-04-26T20:08:15.837033+0000","last_fullsized":"2024-04-26T20:08:15.837033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:50.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571825+0000","last_change":"2024-04-26T20:07:34.767482+0000","last_active":"2024-04-26T20:08:15.571825+0000","last_peered":"2024-04-26T20:08:15.571825+0000","last_clean":"2024-04-26T20:08:15.571825+0000","last_became_active":"2024-04-26T20:07:34.767169+0000","last_became_peered":"2024-04-26T20:07:34.767169+0000","last_unstale":"2024-04-26T20:08:15.571825+0000","last_undegraded":"2024-04-26T20:08:15.571825+0000","last_fullsized":"2024-04-26T20:08:15.571825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:02.117983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573346+0000","last_change":"2024-04-26T20:07:39.312800+0000","last_active":"2024-04-26T20:08:15.573346+0000","last_peered":"2024-04-26T20:08:15.573346+0000","last_clean":"2024-04-26T20:08:15.573346+0000","last_became_active":"2024-04-26T20:07:39.312309+0000","last_became_peered":"2024-04-26T20:07:39.312309+0000","last_unstale":"2024-04-26T20:08:15.573346+0000","last_undegraded":"2024-04-26T20:08:15.573346+0000","last_fullsized":"2024-04-26T20:08:15.573346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:02.013468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.939380+0000","last_change":"2024-04-26T20:07:31.715509+0000","last_active":"2024-04-26T20:08:48.939380+0000","last_peered":"2024-04-26T20:08:48.939380+0000","last_clean":"2024-04-26T20:08:48.939380+0000","last_became_active":"2024-04-26T20:07:31.714606+0000","last_became_peered":"2024-04-26T20:07:31.714606+0000","last_unstale":"2024-04-26T20:08:48.939380+0000","last_undegraded":"2024-04-26T20:08:48.939380+0000","last_fullsized":"2024-04-26T20:08:48.939380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:03.979681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569206+0000","last_change":"2024-04-26T20:07:34.763771+0000","last_active":"2024-04-26T20:08:15.569206+0000","last_peered":"2024-04-26T20:08:15.569206+0000","last_clean":"2024-04-26T20:08:15.569206+0000","last_became_active":"2024-04-26T20:07:34.763422+0000","last_became_peered":"2024-04-26T20:07:34.763422+0000","last_unstale":"2024-04-26T20:08:15.569206+0000","last_undegraded":"2024-04-26T20:08:15.569206+0000","last_fullsized":"2024-04-26T20:08:15.569206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:27.224024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374185+0000","last_change":"2024-04-26T20:07:38.830156+0000","last_active":"2024-04-26T20:08:16.374185+0000","last_peered":"2024-04-26T20:08:16.374185+0000","last_clean":"2024-04-26T20:08:16.374185+0000","last_became_active":"2024-04-26T20:07:38.829530+0000","last_became_peered":"2024-04-26T20:07:38.829530+0000","last_unstale":"2024-04-26T20:08:16.374185+0000","last_undegraded":"2024-04-26T20:08:16.374185+0000","last_fullsized":"2024-04-26T20:08:16.374185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:21.720974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.965661+0000","last_change":"2024-04-26T20:07:31.915638+0000","last_active":"2024-04-26T20:08:49.965661+0000","last_peered":"2024-04-26T20:08:49.965661+0000","last_clean":"2024-04-26T20:08:49.965661+0000","last_became_active":"2024-04-26T20:07:31.915266+0000","last_became_peered":"2024-04-26T20:07:31.915266+0000","last_unstale":"2024-04-26T20:08:49.965661+0000","last_undegraded":"2024-04-26T20:08:49.965661+0000","last_fullsized":"2024-04-26T20:08:49.965661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:51.452828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569604+0000","last_change":"2024-04-26T20:07:34.754476+0000","last_active":"2024-04-26T20:08:15.569604+0000","last_peered":"2024-04-26T20:08:15.569604+0000","last_clean":"2024-04-26T20:08:15.569604+0000","last_became_active":"2024-04-26T20:07:34.754054+0000","last_became_peered":"2024-04-26T20:07:34.754054+0000","last_unstale":"2024-04-26T20:08:15.569604+0000","last_undegraded":"2024-04-26T20:08:15.569604+0000","last_fullsized":"2024-04-26T20:08:15.569604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.497083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.110783+0000","last_change":"2024-04-26T20:07:38.835265+0000","last_active":"2024-04-26T20:08:16.110783+0000","last_peered":"2024-04-26T20:08:16.110783+0000","last_clean":"2024-04-26T20:08:16.110783+0000","last_became_active":"2024-04-26T20:07:38.833873+0000","last_became_peered":"2024-04-26T20:07:38.833873+0000","last_unstale":"2024-04-26T20:08:16.110783+0000","last_undegraded":"2024-04-26T20:08:16.110783+0000","last_fullsized":"2024-04-26T20:08:16.110783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:17.850236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155313+0000","last_change":"2024-04-26T20:07:31.725576+0000","last_active":"2024-04-26T20:08:16.155313+0000","last_peered":"2024-04-26T20:08:16.155313+0000","last_clean":"2024-04-26T20:08:16.155313+0000","last_became_active":"2024-04-26T20:07:31.725166+0000","last_became_peered":"2024-04-26T20:07:31.725166+0000","last_unstale":"2024-04-26T20:08:16.155313+0000","last_undegraded":"2024-04-26T20:08:16.155313+0000","last_fullsized":"2024-04-26T20:08:16.155313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:04.539215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569433+0000","last_change":"2024-04-26T20:07:34.748263+0000","last_active":"2024-04-26T20:08:15.569433+0000","last_peered":"2024-04-26T20:08:15.569433+0000","last_clean":"2024-04-26T20:08:15.569433+0000","last_became_active":"2024-04-26T20:07:34.747889+0000","last_became_peered":"2024-04-26T20:07:34.747889+0000","last_unstale":"2024-04-26T20:08:15.569433+0000","last_undegraded":"2024-04-26T20:08:15.569433+0000","last_fullsized":"2024-04-26T20:08:15.569433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.257405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154483+0000","last_change":"2024-04-26T20:07:38.803999+0000","last_active":"2024-04-26T20:08:16.154483+0000","last_peered":"2024-04-26T20:08:16.154483+0000","last_clean":"2024-04-26T20:08:16.154483+0000","last_became_active":"2024-04-26T20:07:38.803585+0000","last_became_peered":"2024-04-26T20:07:38.803585+0000","last_unstale":"2024-04-26T20:08:16.154483+0000","last_undegraded":"2024-04-26T20:08:16.154483+0000","last_fullsized":"2024-04-26T20:08:16.154483+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:58.863511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.587505+0000","last_change":"2024-04-26T20:07:31.730445+0000","last_active":"2024-04-26T20:08:15.587505+0000","last_peered":"2024-04-26T20:08:15.587505+0000","last_clean":"2024-04-26T20:08:15.587505+0000","last_became_active":"2024-04-26T20:07:31.729944+0000","last_became_peered":"2024-04-26T20:07:31.729944+0000","last_unstale":"2024-04-26T20:08:15.587505+0000","last_undegraded":"2024-04-26T20:08:15.587505+0000","last_fullsized":"2024-04-26T20:08:15.587505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:06.274097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569790+0000","last_change":"2024-04-26T20:07:34.755449+0000","last_active":"2024-04-26T20:08:15.569790+0000","last_peered":"2024-04-26T20:08:15.569790+0000","last_clean":"2024-04-26T20:08:15.569790+0000","last_became_active":"2024-04-26T20:07:34.755100+0000","last_became_peered":"2024-04-26T20:07:34.755100+0000","last_unstale":"2024-04-26T20:08:15.569790+0000","last_undegraded":"2024-04-26T20:08:15.569790+0000","last_fullsized":"2024-04-26T20:08:15.569790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.774751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571204+0000","last_change":"2024-04-26T20:07:34.760161+0000","last_active":"2024-04-26T20:08:15.571204+0000","last_peered":"2024-04-26T20:08:15.571204+0000","last_clean":"2024-04-26T20:08:15.571204+0000","last_became_active":"2024-04-26T20:07:34.759709+0000","last_became_peered":"2024-04-26T20:07:34.759709+0000","last_unstale":"2024-04-26T20:08:15.571204+0000","last_undegraded":"2024-04-26T20:08:15.571204+0000","last_fullsized":"2024-04-26T20:08:15.571204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:54.468818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837080+0000","last_change":"2024-04-26T20:07:34.747723+0000","last_active":"2024-04-26T20:08:15.837080+0000","last_peered":"2024-04-26T20:08:15.837080+0000","last_clean":"2024-04-26T20:08:15.837080+0000","last_became_active":"2024-04-26T20:07:34.747411+0000","last_became_peered":"2024-04-26T20:07:34.747411+0000","last_unstale":"2024-04-26T20:08:15.837080+0000","last_undegraded":"2024-04-26T20:08:15.837080+0000","last_fullsized":"2024-04-26T20:08:15.837080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.836846+0000","last_change":"2024-04-26T20:07:34.773012+0000","last_active":"2024-04-26T20:08:15.836846+0000","last_peered":"2024-04-26T20:08:15.836846+0000","last_clean":"2024-04-26T20:08:15.836846+0000","last_became_active":"2024-04-26T20:07:34.772735+0000","last_became_peered":"2024-04-26T20:07:34.772735+0000","last_unstale":"2024-04-26T20:08:15.836846+0000","last_undegraded":"2024-04-26T20:08:15.836846+0000","last_fullsized":"2024-04-26T20:08:15.836846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:08.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111234+0000","last_change":"2024-04-26T20:07:34.762977+0000","last_active":"2024-04-26T20:08:16.111234+0000","last_peered":"2024-04-26T20:08:16.111234+0000","last_clean":"2024-04-26T20:08:16.111234+0000","last_became_active":"2024-04-26T20:07:34.762530+0000","last_became_peered":"2024-04-26T20:07:34.762530+0000","last_unstale":"2024-04-26T20:08:16.111234+0000","last_undegraded":"2024-04-26T20:08:16.111234+0000","last_fullsized":"2024-04-26T20:08:16.111234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:24.811825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584059+0000","last_change":"2024-04-26T20:07:34.769986+0000","last_active":"2024-04-26T20:08:15.584059+0000","last_peered":"2024-04-26T20:08:15.584059+0000","last_clean":"2024-04-26T20:08:15.584059+0000","last_became_active":"2024-04-26T20:07:34.769591+0000","last_became_peered":"2024-04-26T20:07:34.769591+0000","last_unstale":"2024-04-26T20:08:15.584059+0000","last_undegraded":"2024-04-26T20:08:15.584059+0000","last_fullsized":"2024-04-26T20:08:15.584059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:27.466675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571565+0000","last_change":"2024-04-26T20:07:34.754733+0000","last_active":"2024-04-26T20:08:15.571565+0000","last_peered":"2024-04-26T20:08:15.571565+0000","last_clean":"2024-04-26T20:08:15.571565+0000","last_became_active":"2024-04-26T20:07:34.754374+0000","last_became_peered":"2024-04-26T20:07:34.754374+0000","last_unstale":"2024-04-26T20:08:15.571565+0000","last_undegraded":"2024-04-26T20:08:15.571565+0000","last_fullsized":"2024-04-26T20:08:15.571565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:02.732025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837375+0000","last_change":"2024-04-26T20:07:34.750329+0000","last_active":"2024-04-26T20:08:15.837375+0000","last_peered":"2024-04-26T20:08:15.837375+0000","last_clean":"2024-04-26T20:08:15.837375+0000","last_became_active":"2024-04-26T20:07:34.749970+0000","last_became_peered":"2024-04-26T20:07:34.749970+0000","last_unstale":"2024-04-26T20:08:15.837375+0000","last_undegraded":"2024-04-26T20:08:15.837375+0000","last_fullsized":"2024-04-26T20:08:15.837375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:50.375128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.572512+0000","last_change":"2024-04-26T20:07:34.750623+0000","last_active":"2024-04-26T20:08:15.572512+0000","last_peered":"2024-04-26T20:08:15.572512+0000","last_clean":"2024-04-26T20:08:15.572512+0000","last_became_active":"2024-04-26T20:07:34.750314+0000","last_became_peered":"2024-04-26T20:07:34.750314+0000","last_unstale":"2024-04-26T20:08:15.572512+0000","last_undegraded":"2024-04-26T20:08:15.572512+0000","last_fullsized":"2024-04-26T20:08:15.572512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:16.773986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154828+0000","last_change":"2024-04-26T20:07:34.755474+0000","last_active":"2024-04-26T20:08:16.154828+0000","last_peered":"2024-04-26T20:08:16.154828+0000","last_clean":"2024-04-26T20:08:16.154828+0000","last_became_active":"2024-04-26T20:07:34.755133+0000","last_became_peered":"2024-04-26T20:07:34.755133+0000","last_unstale":"2024-04-26T20:08:16.154828+0000","last_undegraded":"2024-04-26T20:08:16.154828+0000","last_fullsized":"2024-04-26T20:08:16.154828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:02.861643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571466+0000","last_change":"2024-04-26T20:07:34.752727+0000","last_active":"2024-04-26T20:08:15.571466+0000","last_peered":"2024-04-26T20:08:15.571466+0000","last_clean":"2024-04-26T20:08:15.571466+0000","last_became_active":"2024-04-26T20:07:34.752420+0000","last_became_peered":"2024-04-26T20:07:34.752420+0000","last_unstale":"2024-04-26T20:08:15.571466+0000","last_undegraded":"2024-04-26T20:08:15.571466+0000","last_fullsized":"2024-04-26T20:08:15.571466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:45.076837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374461+0000","last_change":"2024-04-26T20:07:34.775116+0000","last_active":"2024-04-26T20:08:16.374461+0000","last_peered":"2024-04-26T20:08:16.374461+0000","last_clean":"2024-04-26T20:08:16.374461+0000","last_became_active":"2024-04-26T20:07:34.774710+0000","last_became_peered":"2024-04-26T20:07:34.774710+0000","last_unstale":"2024-04-26T20:08:16.374461+0000","last_undegraded":"2024-04-26T20:08:16.374461+0000","last_fullsized":"2024-04-26T20:08:16.374461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:53.203918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584777+0000","last_change":"2024-04-26T20:07:34.773245+0000","last_active":"2024-04-26T20:08:15.584777+0000","last_peered":"2024-04-26T20:08:15.584777+0000","last_clean":"2024-04-26T20:08:15.584777+0000","last_became_active":"2024-04-26T20:07:34.772879+0000","last_became_peered":"2024-04-26T20:07:34.772879+0000","last_unstale":"2024-04-26T20:08:15.584777+0000","last_undegraded":"2024-04-26T20:08:15.584777+0000","last_fullsized":"2024-04-26T20:08:15.584777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:54.450454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571986+0000","last_change":"2024-04-26T20:07:34.753716+0000","last_active":"2024-04-26T20:08:15.571986+0000","last_peered":"2024-04-26T20:08:15.571986+0000","last_clean":"2024-04-26T20:08:15.571986+0000","last_became_active":"2024-04-26T20:07:34.753378+0000","last_became_peered":"2024-04-26T20:07:34.753378+0000","last_unstale":"2024-04-26T20:08:15.571986+0000","last_undegraded":"2024-04-26T20:08:15.571986+0000","last_fullsized":"2024-04-26T20:08:15.571986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:47.165305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56481,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":145,"num_read_kb":133,"num_write":157,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":169443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":535138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1174886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":539470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":535294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1131203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":530550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":582940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":531938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40620,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704532,"statfs":{"total":95995035648,"available":95953440768,"internally_reserved":0,"allocated":1150976,"data_stored":576680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":530744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":535200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1699840,"data_stored":1126791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:09:13.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:09:14.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:13 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1552967138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:09:14.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:13 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/144386107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 -- 172.21.15.131:0/583540196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 msgr2=0x7f92b0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 --2- 172.21.15.131:0/583540196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b0103dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f929c009ca0 tx=0x7f929c02a8c0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 -- 172.21.15.131:0/583540196 shutdown_connections 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 --2- 172.21.15.131:0/583540196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 0x7f92b010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 --2- 172.21.15.131:0/583540196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b0103dc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 --2- 172.21.15.131:0/583540196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92b0102750 0x7f92b0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 -- 172.21.15.131:0/583540196 >> 172.21.15.131:0/583540196 conn(0x7f92b00fdcc0 msgr2=0x7f92b0100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:14.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 -- 172.21.15.131:0/583540196 shutdown_connections 2024-04-26T20:09:14.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.076+0000 7f92af200640 1 -- 172.21.15.131:0/583540196 wait complete. 2024-04-26T20:09:14.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.077+0000 7f92af200640 1 Processor -- start 2024-04-26T20:09:14.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92af200640 1 -- start start 2024-04-26T20:09:14.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92af200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92b0102750 0x7f92b019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:14.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92af200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:14.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92af200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 0x7f92b01a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:14.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92af200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f92b010e2b0 con 0x7f92b0102750 2024-04-26T20:09:14.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92af200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92b010e130 con 0x7f92b0103940 2024-04-26T20:09:14.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92af200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f92b010e430 con 0x7f92b0104300 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92ad400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92ae800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 0x7f92b01a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92ade00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92b0102750 0x7f92b019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92ae800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 0x7f92b01a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35584/0 (socket says 172.21.15.131:35584) 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.078+0000 7f92ad400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b019b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:54006/0 (socket says 172.21.15.131:54006) 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ae800640 1 -- 172.21.15.131:0/121821426 learned_addr learned my addr 172.21.15.131:0/121821426 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ae800640 1 -- 172.21.15.131:0/121821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 msgr2=0x7f92b019b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ae800640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b019b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ae800640 1 -- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92b0102750 msgr2=0x7f92b019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ae800640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92b0102750 0x7f92b019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ae800640 1 -- 172.21.15.131:0/121821426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b01a2780 con 0x7f92b0104300 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ad400640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92ae800640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 0x7f92b01a2110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92a800d750 tx=0x7f92a800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:14.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.079+0000 7f92a6a00640 1 -- 172.21.15.131:0/121821426 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8017070 con 0x7f92b0104300 2024-04-26T20:09:14.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.080+0000 7f92a6a00640 1 -- 172.21.15.131:0/121821426 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a80054d0 con 0x7f92b0104300 2024-04-26T20:09:14.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.080+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b01a2a70 con 0x7f92b0104300 2024-04-26T20:09:14.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.080+0000 7f92a6a00640 1 -- 172.21.15.131:0/121821426 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8004c60 con 0x7f92b0104300 2024-04-26T20:09:14.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.080+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01a2f90 con 0x7f92b0104300 2024-04-26T20:09:14.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.081+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b01a3220 con 0x7f92b0104300 2024-04-26T20:09:14.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.083+0000 7f92a6a00640 1 -- 172.21.15.131:0/121821426 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f92a8009bd0 con 0x7f92b0104300 2024-04-26T20:09:14.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.084+0000 7f92a6a00640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92940a3cc0 0x7f92940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:14.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.084+0000 7f92a6a00640 1 -- 172.21.15.131:0/121821426 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f92a80d01b0 con 0x7f92b0104300 2024-04-26T20:09:14.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.084+0000 7f92ade00640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92940a3cc0 0x7f92940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:14.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.085+0000 7f92ade00640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92940a3cc0 0x7f92940a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f92b01037a0 tx=0x7f92a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:14.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.088+0000 7f92a6a00640 1 -- 172.21.15.131:0/121821426 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f92a809c020 con 0x7f92b0104300 2024-04-26T20:09:14.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:13 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1552967138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:09:14.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:13 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/144386107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:14.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:13 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1552967138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:09:14.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:13 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/144386107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:14.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.273+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92b0076280 con 0x7f92940a3cc0 2024-04-26T20:09:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.282+0000 7f92a6a00640 1 -- 172.21.15.131:0/121821426 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7f92b0076280 con 0x7f92940a3cc0 2024-04-26T20:09:14.292 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:09:14.294 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92940a3cc0 msgr2=0x7f92940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92940a3cc0 0x7f92940a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f92b01037a0 tx=0x7f92a000b390 comp rx=0 tx=0).stop 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 msgr2=0x7f92b01a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 0x7f92b01a2110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92a800d750 tx=0x7f92a800dc10 comp rx=0 tx=0).stop 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 shutdown_connections 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92940a3cc0 0x7f92940a6180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92b0104300 0x7f92b01a2110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b0103940 0x7f92b019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 --2- 172.21.15.131:0/121821426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92b0102750 0x7f92b019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.289+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 >> 172.21.15.131:0/121821426 conn(0x7f92b00fdcc0 msgr2=0x7f92b0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.290+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 shutdown_connections 2024-04-26T20:09:14.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:14.290+0000 7f92af200640 1 -- 172.21.15.131:0/121821426 wait complete. 2024-04-26T20:09:14.310 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-04-26T20:09:13.633527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647083,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":271,"num_read_kb":242,"num_write":340,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368036,"kb_used_data":14724,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573788,"statfs":{"total":1151940427776,"available":1151563558912,"internally_reserved":0,"allocated":15077376,"data_stored":8330834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49263,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":95,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006261"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571774+0000","last_change":"2024-04-26T20:07:34.771977+0000","last_active":"2024-04-26T20:08:15.571774+0000","last_peered":"2024-04-26T20:08:15.571774+0000","last_clean":"2024-04-26T20:08:15.571774+0000","last_became_active":"2024-04-26T20:07:34.771658+0000","last_became_peered":"2024-04-26T20:07:34.771658+0000","last_unstale":"2024-04-26T20:08:15.571774+0000","last_undegraded":"2024-04-26T20:08:15.571774+0000","last_fullsized":"2024-04-26T20:08:15.571774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:38.419057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111647+0000","last_change":"2024-04-26T20:07:34.768774+0000","last_active":"2024-04-26T20:08:16.111647+0000","last_peered":"2024-04-26T20:08:16.111647+0000","last_clean":"2024-04-26T20:08:16.111647+0000","last_became_active":"2024-04-26T20:07:34.768383+0000","last_became_peered":"2024-04-26T20:07:34.768383+0000","last_unstale":"2024-04-26T20:08:16.111647+0000","last_undegraded":"2024-04-26T20:08:16.111647+0000","last_fullsized":"2024-04-26T20:08:16.111647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:43.813261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111048+0000","last_change":"2024-04-26T20:07:34.761796+0000","last_active":"2024-04-26T20:08:16.111048+0000","last_peered":"2024-04-26T20:08:16.111048+0000","last_clean":"2024-04-26T20:08:16.111048+0000","last_became_active":"2024-04-26T20:07:34.761403+0000","last_became_peered":"2024-04-26T20:07:34.761403+0000","last_unstale":"2024-04-26T20:08:16.111048+0000","last_undegraded":"2024-04-26T20:08:16.111048+0000","last_fullsized":"2024-04-26T20:08:16.111048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:06.832304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573588+0000","last_change":"2024-04-26T20:07:34.756728+0000","last_active":"2024-04-26T20:08:15.573588+0000","last_peered":"2024-04-26T20:08:15.573588+0000","last_clean":"2024-04-26T20:08:15.573588+0000","last_became_active":"2024-04-26T20:07:34.756406+0000","last_became_peered":"2024-04-26T20:07:34.756406+0000","last_unstale":"2024-04-26T20:08:15.573588+0000","last_undegraded":"2024-04-26T20:08:15.573588+0000","last_fullsized":"2024-04-26T20:08:15.573588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:27.753674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568635+0000","last_change":"2024-04-26T20:07:34.758243+0000","last_active":"2024-04-26T20:08:15.568635+0000","last_peered":"2024-04-26T20:08:15.568635+0000","last_clean":"2024-04-26T20:08:15.568635+0000","last_became_active":"2024-04-26T20:07:34.757868+0000","last_became_peered":"2024-04-26T20:07:34.757868+0000","last_unstale":"2024-04-26T20:08:15.568635+0000","last_undegraded":"2024-04-26T20:08:15.568635+0000","last_fullsized":"2024-04-26T20:08:15.568635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:53.361835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.585311+0000","last_change":"2024-04-26T20:08:15.585311+0000","last_active":"2024-04-26T20:08:15.585311+0000","last_peered":"2024-04-26T20:08:15.585311+0000","last_clean":"2024-04-26T20:08:15.585311+0000","last_became_active":"2024-04-26T20:08:15.584982+0000","last_became_peered":"2024-04-26T20:08:15.584982+0000","last_unstale":"2024-04-26T20:08:15.585311+0000","last_undegraded":"2024-04-26T20:08:15.585311+0000","last_fullsized":"2024-04-26T20:08:15.585311+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:56.828218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569002+0000","last_change":"2024-04-26T20:07:34.757529+0000","last_active":"2024-04-26T20:08:15.569002+0000","last_peered":"2024-04-26T20:08:15.569002+0000","last_clean":"2024-04-26T20:08:15.569002+0000","last_became_active":"2024-04-26T20:07:34.757166+0000","last_became_peered":"2024-04-26T20:07:34.757166+0000","last_unstale":"2024-04-26T20:08:15.569002+0000","last_undegraded":"2024-04-26T20:08:15.569002+0000","last_fullsized":"2024-04-26T20:08:15.569002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:01.309698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.570536+0000","last_change":"2024-04-26T20:07:34.765352+0000","last_active":"2024-04-26T20:08:15.570536+0000","last_peered":"2024-04-26T20:08:15.570536+0000","last_clean":"2024-04-26T20:08:15.570536+0000","last_became_active":"2024-04-26T20:07:34.758400+0000","last_became_peered":"2024-04-26T20:07:34.758400+0000","last_unstale":"2024-04-26T20:08:15.570536+0000","last_undegraded":"2024-04-26T20:08:15.570536+0000","last_fullsized":"2024-04-26T20:08:15.570536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:43.940746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374668+0000","last_change":"2024-04-26T20:07:34.754413+0000","last_active":"2024-04-26T20:08:16.374668+0000","last_peered":"2024-04-26T20:08:16.374668+0000","last_clean":"2024-04-26T20:08:16.374668+0000","last_became_active":"2024-04-26T20:07:34.754073+0000","last_became_peered":"2024-04-26T20:07:34.754073+0000","last_unstale":"2024-04-26T20:08:16.374668+0000","last_undegraded":"2024-04-26T20:08:16.374668+0000","last_fullsized":"2024-04-26T20:08:16.374668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:25.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.576145+0000","last_change":"2024-04-26T20:07:34.748739+0000","last_active":"2024-04-26T20:08:15.576145+0000","last_peered":"2024-04-26T20:08:15.576145+0000","last_clean":"2024-04-26T20:08:15.576145+0000","last_became_active":"2024-04-26T20:07:34.748378+0000","last_became_peered":"2024-04-26T20:07:34.748378+0000","last_unstale":"2024-04-26T20:08:15.576145+0000","last_undegraded":"2024-04-26T20:08:15.576145+0000","last_fullsized":"2024-04-26T20:08:15.576145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:13.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569570+0000","last_change":"2024-04-26T20:07:34.747402+0000","last_active":"2024-04-26T20:08:15.569570+0000","last_peered":"2024-04-26T20:08:15.569570+0000","last_clean":"2024-04-26T20:08:15.569570+0000","last_became_active":"2024-04-26T20:07:34.746965+0000","last_became_peered":"2024-04-26T20:07:34.746965+0000","last_unstale":"2024-04-26T20:08:15.569570+0000","last_undegraded":"2024-04-26T20:08:15.569570+0000","last_fullsized":"2024-04-26T20:08:15.569570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:32.066286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573453+0000","last_change":"2024-04-26T20:07:34.755848+0000","last_active":"2024-04-26T20:08:15.573453+0000","last_peered":"2024-04-26T20:08:15.573453+0000","last_clean":"2024-04-26T20:08:15.573453+0000","last_became_active":"2024-04-26T20:07:34.755403+0000","last_became_peered":"2024-04-26T20:07:34.755403+0000","last_unstale":"2024-04-26T20:08:15.573453+0000","last_undegraded":"2024-04-26T20:08:15.573453+0000","last_fullsized":"2024-04-26T20:08:15.573453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:30.576290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.585014+0000","last_change":"2024-04-26T20:07:34.766542+0000","last_active":"2024-04-26T20:08:15.585014+0000","last_peered":"2024-04-26T20:08:15.585014+0000","last_clean":"2024-04-26T20:08:15.585014+0000","last_became_active":"2024-04-26T20:07:34.766185+0000","last_became_peered":"2024-04-26T20:07:34.766185+0000","last_unstale":"2024-04-26T20:08:15.585014+0000","last_undegraded":"2024-04-26T20:08:15.585014+0000","last_fullsized":"2024-04-26T20:08:15.585014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:34.499145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111075+0000","last_change":"2024-04-26T20:07:34.762610+0000","last_active":"2024-04-26T20:08:16.111075+0000","last_peered":"2024-04-26T20:08:16.111075+0000","last_clean":"2024-04-26T20:08:16.111075+0000","last_became_active":"2024-04-26T20:07:34.762100+0000","last_became_peered":"2024-04-26T20:07:34.762100+0000","last_unstale":"2024-04-26T20:08:16.111075+0000","last_undegraded":"2024-04-26T20:08:16.111075+0000","last_fullsized":"2024-04-26T20:08:16.111075+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:18.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111002+0000","last_change":"2024-04-26T20:07:34.766754+0000","last_active":"2024-04-26T20:08:16.111002+0000","last_peered":"2024-04-26T20:08:16.111002+0000","last_clean":"2024-04-26T20:08:16.111002+0000","last_became_active":"2024-04-26T20:07:34.762204+0000","last_became_peered":"2024-04-26T20:07:34.762204+0000","last_unstale":"2024-04-26T20:08:16.111002+0000","last_undegraded":"2024-04-26T20:08:16.111002+0000","last_fullsized":"2024-04-26T20:08:16.111002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:25.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374070+0000","last_change":"2024-04-26T20:07:34.769739+0000","last_active":"2024-04-26T20:08:16.374070+0000","last_peered":"2024-04-26T20:08:16.374070+0000","last_clean":"2024-04-26T20:08:16.374070+0000","last_became_active":"2024-04-26T20:07:34.769383+0000","last_became_peered":"2024-04-26T20:07:34.769383+0000","last_unstale":"2024-04-26T20:08:16.374070+0000","last_undegraded":"2024-04-26T20:08:16.374070+0000","last_fullsized":"2024-04-26T20:08:16.374070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:08.771590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571611+0000","last_change":"2024-04-26T20:07:34.772010+0000","last_active":"2024-04-26T20:08:15.571611+0000","last_peered":"2024-04-26T20:08:15.571611+0000","last_clean":"2024-04-26T20:08:15.571611+0000","last_became_active":"2024-04-26T20:07:34.771682+0000","last_became_peered":"2024-04-26T20:07:34.771682+0000","last_unstale":"2024-04-26T20:08:15.571611+0000","last_undegraded":"2024-04-26T20:08:15.571611+0000","last_fullsized":"2024-04-26T20:08:15.571611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:57.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583019+0000","last_change":"2024-04-26T20:07:34.757547+0000","last_active":"2024-04-26T20:08:15.583019+0000","last_peered":"2024-04-26T20:08:15.583019+0000","last_clean":"2024-04-26T20:08:15.583019+0000","last_became_active":"2024-04-26T20:07:34.757180+0000","last_became_peered":"2024-04-26T20:07:34.757180+0000","last_unstale":"2024-04-26T20:08:15.583019+0000","last_undegraded":"2024-04-26T20:08:15.583019+0000","last_fullsized":"2024-04-26T20:08:15.583019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:32.365944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837548+0000","last_change":"2024-04-26T20:07:34.767363+0000","last_active":"2024-04-26T20:08:15.837548+0000","last_peered":"2024-04-26T20:08:15.837548+0000","last_clean":"2024-04-26T20:08:15.837548+0000","last_became_active":"2024-04-26T20:07:34.766909+0000","last_became_peered":"2024-04-26T20:07:34.766909+0000","last_unstale":"2024-04-26T20:08:15.837548+0000","last_undegraded":"2024-04-26T20:08:15.837548+0000","last_fullsized":"2024-04-26T20:08:15.837548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:32:31.500141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.836872+0000","last_change":"2024-04-26T20:07:38.823959+0000","last_active":"2024-04-26T20:08:15.836872+0000","last_peered":"2024-04-26T20:08:15.836872+0000","last_clean":"2024-04-26T20:08:15.836872+0000","last_became_active":"2024-04-26T20:07:38.823524+0000","last_became_peered":"2024-04-26T20:07:38.823524+0000","last_unstale":"2024-04-26T20:08:15.836872+0000","last_undegraded":"2024-04-26T20:08:15.836872+0000","last_fullsized":"2024-04-26T20:08:15.836872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:23.348376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.964850+0000","last_change":"2024-04-26T20:07:31.721324+0000","last_active":"2024-04-26T20:08:49.964850+0000","last_peered":"2024-04-26T20:08:49.964850+0000","last_clean":"2024-04-26T20:08:49.964850+0000","last_became_active":"2024-04-26T20:07:31.720140+0000","last_became_peered":"2024-04-26T20:07:31.720140+0000","last_unstale":"2024-04-26T20:08:49.964850+0000","last_undegraded":"2024-04-26T20:08:49.964850+0000","last_fullsized":"2024-04-26T20:08:49.964850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:27.636660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571683+0000","last_change":"2024-04-26T20:07:34.760387+0000","last_active":"2024-04-26T20:08:15.571683+0000","last_peered":"2024-04-26T20:08:15.571683+0000","last_clean":"2024-04-26T20:08:15.571683+0000","last_became_active":"2024-04-26T20:07:34.760042+0000","last_became_peered":"2024-04-26T20:07:34.760042+0000","last_unstale":"2024-04-26T20:08:15.571683+0000","last_undegraded":"2024-04-26T20:08:15.571683+0000","last_fullsized":"2024-04-26T20:08:15.571683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:39.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.587181+0000","last_change":"2024-04-26T20:08:15.587181+0000","last_active":"2024-04-26T20:08:15.587181+0000","last_peered":"2024-04-26T20:08:15.587181+0000","last_clean":"2024-04-26T20:08:15.587181+0000","last_became_active":"2024-04-26T20:08:15.586847+0000","last_became_peered":"2024-04-26T20:08:15.586847+0000","last_unstale":"2024-04-26T20:08:15.587181+0000","last_undegraded":"2024-04-26T20:08:15.587181+0000","last_fullsized":"2024-04-26T20:08:15.587181+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:12.495523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573157+0000","last_change":"2024-04-26T20:07:31.711623+0000","last_active":"2024-04-26T20:08:15.573157+0000","last_peered":"2024-04-26T20:08:15.573157+0000","last_clean":"2024-04-26T20:08:15.573157+0000","last_became_active":"2024-04-26T20:07:31.710488+0000","last_became_peered":"2024-04-26T20:07:31.710488+0000","last_unstale":"2024-04-26T20:08:15.573157+0000","last_undegraded":"2024-04-26T20:08:15.573157+0000","last_fullsized":"2024-04-26T20:08:15.573157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:04.747170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111751+0000","last_change":"2024-04-26T20:07:34.762383+0000","last_active":"2024-04-26T20:08:16.111751+0000","last_peered":"2024-04-26T20:08:16.111751+0000","last_clean":"2024-04-26T20:08:16.111751+0000","last_became_active":"2024-04-26T20:07:34.762012+0000","last_became_peered":"2024-04-26T20:07:34.762012+0000","last_unstale":"2024-04-26T20:08:16.111751+0000","last_undegraded":"2024-04-26T20:08:16.111751+0000","last_fullsized":"2024-04-26T20:08:16.111751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:27:16.715387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573094+0000","last_change":"2024-04-26T20:07:38.822443+0000","last_active":"2024-04-26T20:08:15.573094+0000","last_peered":"2024-04-26T20:08:15.573094+0000","last_clean":"2024-04-26T20:08:15.573094+0000","last_became_active":"2024-04-26T20:07:38.822046+0000","last_became_peered":"2024-04-26T20:07:38.822046+0000","last_unstale":"2024-04-26T20:08:15.573094+0000","last_undegraded":"2024-04-26T20:08:15.573094+0000","last_fullsized":"2024-04-26T20:08:15.573094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:26.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.961193+0000","last_change":"2024-04-26T20:07:31.715319+0000","last_active":"2024-04-26T20:08:49.961193+0000","last_peered":"2024-04-26T20:08:49.961193+0000","last_clean":"2024-04-26T20:08:49.961193+0000","last_became_active":"2024-04-26T20:07:31.714915+0000","last_became_peered":"2024-04-26T20:07:31.714915+0000","last_unstale":"2024-04-26T20:08:49.961193+0000","last_undegraded":"2024-04-26T20:08:49.961193+0000","last_fullsized":"2024-04-26T20:08:49.961193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:27.898652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374346+0000","last_change":"2024-04-26T20:07:34.753422+0000","last_active":"2024-04-26T20:08:16.374346+0000","last_peered":"2024-04-26T20:08:16.374346+0000","last_clean":"2024-04-26T20:08:16.374346+0000","last_became_active":"2024-04-26T20:07:34.753009+0000","last_became_peered":"2024-04-26T20:07:34.753009+0000","last_unstale":"2024-04-26T20:08:16.374346+0000","last_undegraded":"2024-04-26T20:08:16.374346+0000","last_fullsized":"2024-04-26T20:08:16.374346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.369715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573079+0000","last_change":"2024-04-26T20:07:38.830239+0000","last_active":"2024-04-26T20:08:15.573079+0000","last_peered":"2024-04-26T20:08:15.573079+0000","last_clean":"2024-04-26T20:08:15.573079+0000","last_became_active":"2024-04-26T20:07:38.829509+0000","last_became_peered":"2024-04-26T20:07:38.829509+0000","last_unstale":"2024-04-26T20:08:15.573079+0000","last_undegraded":"2024-04-26T20:08:15.573079+0000","last_fullsized":"2024-04-26T20:08:15.573079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:53.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571416+0000","last_change":"2024-04-26T20:08:14.576129+0000","last_active":"2024-04-26T20:08:15.571416+0000","last_peered":"2024-04-26T20:08:15.571416+0000","last_clean":"2024-04-26T20:08:15.571416+0000","last_became_active":"2024-04-26T20:08:14.573168+0000","last_became_peered":"2024-04-26T20:08:14.573168+0000","last_unstale":"2024-04-26T20:08:15.571416+0000","last_undegraded":"2024-04-26T20:08:15.571416+0000","last_fullsized":"2024-04-26T20:08:15.571416+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:38.234154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155226+0000","last_change":"2024-04-26T20:07:34.749270+0000","last_active":"2024-04-26T20:08:16.155226+0000","last_peered":"2024-04-26T20:08:16.155226+0000","last_clean":"2024-04-26T20:08:16.155226+0000","last_became_active":"2024-04-26T20:07:34.748585+0000","last_became_peered":"2024-04-26T20:07:34.748585+0000","last_unstale":"2024-04-26T20:08:16.155226+0000","last_undegraded":"2024-04-26T20:08:16.155226+0000","last_fullsized":"2024-04-26T20:08:16.155226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:03.091720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837290+0000","last_change":"2024-04-26T20:07:38.806247+0000","last_active":"2024-04-26T20:08:15.837290+0000","last_peered":"2024-04-26T20:08:15.837290+0000","last_clean":"2024-04-26T20:08:15.837290+0000","last_became_active":"2024-04-26T20:07:38.805748+0000","last_became_peered":"2024-04-26T20:07:38.805748+0000","last_unstale":"2024-04-26T20:08:15.837290+0000","last_undegraded":"2024-04-26T20:08:15.837290+0000","last_fullsized":"2024-04-26T20:08:15.837290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:16.962494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":65,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:11.287089+0000","last_change":"2024-04-26T20:07:31.713618+0000","last_active":"2024-04-26T20:09:11.287089+0000","last_peered":"2024-04-26T20:09:11.287089+0000","last_clean":"2024-04-26T20:09:11.287089+0000","last_became_active":"2024-04-26T20:07:31.713259+0000","last_became_peered":"2024-04-26T20:07:31.713259+0000","last_unstale":"2024-04-26T20:09:11.287089+0000","last_undegraded":"2024-04-26T20:09:11.287089+0000","last_fullsized":"2024-04-26T20:09:11.287089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:25.215087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568618+0000","last_change":"2024-04-26T20:08:14.585688+0000","last_active":"2024-04-26T20:08:15.568618+0000","last_peered":"2024-04-26T20:08:15.568618+0000","last_clean":"2024-04-26T20:08:15.568618+0000","last_became_active":"2024-04-26T20:08:14.585337+0000","last_became_peered":"2024-04-26T20:08:14.585337+0000","last_unstale":"2024-04-26T20:08:15.568618+0000","last_undegraded":"2024-04-26T20:08:15.568618+0000","last_fullsized":"2024-04-26T20:08:15.568618+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:03.515457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837120+0000","last_change":"2024-04-26T20:07:38.829772+0000","last_active":"2024-04-26T20:08:15.837120+0000","last_peered":"2024-04-26T20:08:15.837120+0000","last_clean":"2024-04-26T20:08:15.837120+0000","last_became_active":"2024-04-26T20:07:38.829327+0000","last_became_peered":"2024-04-26T20:07:38.829327+0000","last_unstale":"2024-04-26T20:08:15.837120+0000","last_undegraded":"2024-04-26T20:08:15.837120+0000","last_fullsized":"2024-04-26T20:08:15.837120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:37.252863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.965820+0000","last_change":"2024-04-26T20:07:31.728654+0000","last_active":"2024-04-26T20:08:49.965820+0000","last_peered":"2024-04-26T20:08:49.965820+0000","last_clean":"2024-04-26T20:08:49.965820+0000","last_became_active":"2024-04-26T20:07:31.728278+0000","last_became_peered":"2024-04-26T20:07:31.728278+0000","last_unstale":"2024-04-26T20:08:49.965820+0000","last_undegraded":"2024-04-26T20:08:49.965820+0000","last_fullsized":"2024-04-26T20:08:49.965820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155076+0000","last_change":"2024-04-26T20:07:34.739575+0000","last_active":"2024-04-26T20:08:16.155076+0000","last_peered":"2024-04-26T20:08:16.155076+0000","last_clean":"2024-04-26T20:08:16.155076+0000","last_became_active":"2024-04-26T20:07:34.739264+0000","last_became_peered":"2024-04-26T20:07:34.739264+0000","last_unstale":"2024-04-26T20:08:16.155076+0000","last_undegraded":"2024-04-26T20:08:16.155076+0000","last_fullsized":"2024-04-26T20:08:16.155076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:25.130064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571551+0000","last_change":"2024-04-26T20:07:38.822669+0000","last_active":"2024-04-26T20:08:15.571551+0000","last_peered":"2024-04-26T20:08:15.571551+0000","last_clean":"2024-04-26T20:08:15.571551+0000","last_became_active":"2024-04-26T20:07:38.822197+0000","last_became_peered":"2024-04-26T20:07:38.822197+0000","last_unstale":"2024-04-26T20:08:15.571551+0000","last_undegraded":"2024-04-26T20:08:15.571551+0000","last_fullsized":"2024-04-26T20:08:15.571551+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:56.782794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"95'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.967402+0000","last_change":"2024-04-26T20:07:31.723527+0000","last_active":"2024-04-26T20:08:49.967402+0000","last_peered":"2024-04-26T20:08:49.967402+0000","last_clean":"2024-04-26T20:08:49.967402+0000","last_became_active":"2024-04-26T20:07:31.722574+0000","last_became_peered":"2024-04-26T20:07:31.722574+0000","last_unstale":"2024-04-26T20:08:49.967402+0000","last_undegraded":"2024-04-26T20:08:49.967402+0000","last_fullsized":"2024-04-26T20:08:49.967402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.521613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571094+0000","last_change":"2024-04-26T20:07:34.771459+0000","last_active":"2024-04-26T20:08:15.571094+0000","last_peered":"2024-04-26T20:08:15.571094+0000","last_clean":"2024-04-26T20:08:15.571094+0000","last_became_active":"2024-04-26T20:07:34.771124+0000","last_became_peered":"2024-04-26T20:07:34.771124+0000","last_unstale":"2024-04-26T20:08:15.571094+0000","last_undegraded":"2024-04-26T20:08:15.571094+0000","last_fullsized":"2024-04-26T20:08:15.571094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.560440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111642+0000","last_change":"2024-04-26T20:07:38.838858+0000","last_active":"2024-04-26T20:08:16.111642+0000","last_peered":"2024-04-26T20:08:16.111642+0000","last_clean":"2024-04-26T20:08:16.111642+0000","last_became_active":"2024-04-26T20:07:38.838125+0000","last_became_peered":"2024-04-26T20:07:38.838125+0000","last_unstale":"2024-04-26T20:08:16.111642+0000","last_undegraded":"2024-04-26T20:08:16.111642+0000","last_fullsized":"2024-04-26T20:08:16.111642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:01.856633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"95'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154860+0000","last_change":"2024-04-26T20:07:31.710690+0000","last_active":"2024-04-26T20:08:16.154860+0000","last_peered":"2024-04-26T20:08:16.154860+0000","last_clean":"2024-04-26T20:08:16.154860+0000","last_became_active":"2024-04-26T20:07:31.710353+0000","last_became_peered":"2024-04-26T20:07:31.710353+0000","last_unstale":"2024-04-26T20:08:16.154860+0000","last_undegraded":"2024-04-26T20:08:16.154860+0000","last_fullsized":"2024-04-26T20:08:16.154860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:47.442673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.578985+0000","last_change":"2024-04-26T20:07:34.776023+0000","last_active":"2024-04-26T20:08:15.578985+0000","last_peered":"2024-04-26T20:08:15.578985+0000","last_clean":"2024-04-26T20:08:15.578985+0000","last_became_active":"2024-04-26T20:07:34.775571+0000","last_became_peered":"2024-04-26T20:07:34.775571+0000","last_unstale":"2024-04-26T20:08:15.578985+0000","last_undegraded":"2024-04-26T20:08:15.578985+0000","last_fullsized":"2024-04-26T20:08:15.578985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:40.573046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.570964+0000","last_change":"2024-04-26T20:07:38.820636+0000","last_active":"2024-04-26T20:08:15.570964+0000","last_peered":"2024-04-26T20:08:15.570964+0000","last_clean":"2024-04-26T20:08:15.570964+0000","last_became_active":"2024-04-26T20:07:38.819390+0000","last_became_peered":"2024-04-26T20:07:38.819390+0000","last_unstale":"2024-04-26T20:08:15.570964+0000","last_undegraded":"2024-04-26T20:08:15.570964+0000","last_fullsized":"2024-04-26T20:08:15.570964+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:58.793479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'3","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:11.286188+0000","last_change":"2024-04-26T20:07:31.724192+0000","last_active":"2024-04-26T20:09:11.286188+0000","last_peered":"2024-04-26T20:09:11.286188+0000","last_clean":"2024-04-26T20:09:11.286188+0000","last_became_active":"2024-04-26T20:07:31.723816+0000","last_became_peered":"2024-04-26T20:07:31.723816+0000","last_unstale":"2024-04-26T20:09:11.286188+0000","last_undegraded":"2024-04-26T20:09:11.286188+0000","last_fullsized":"2024-04-26T20:09:11.286188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:31.707282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374611+0000","last_change":"2024-04-26T20:07:34.759027+0000","last_active":"2024-04-26T20:08:16.374611+0000","last_peered":"2024-04-26T20:08:16.374611+0000","last_clean":"2024-04-26T20:08:16.374611+0000","last_became_active":"2024-04-26T20:07:34.758689+0000","last_became_peered":"2024-04-26T20:07:34.758689+0000","last_unstale":"2024-04-26T20:08:16.374611+0000","last_undegraded":"2024-04-26T20:08:16.374611+0000","last_fullsized":"2024-04-26T20:08:16.374611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:29.110222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.579328+0000","last_change":"2024-04-26T20:07:38.819000+0000","last_active":"2024-04-26T20:08:15.579328+0000","last_peered":"2024-04-26T20:08:15.579328+0000","last_clean":"2024-04-26T20:08:15.579328+0000","last_became_active":"2024-04-26T20:07:38.818526+0000","last_became_peered":"2024-04-26T20:07:38.818526+0000","last_unstale":"2024-04-26T20:08:15.579328+0000","last_undegraded":"2024-04-26T20:08:15.579328+0000","last_fullsized":"2024-04-26T20:08:15.579328+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:28.830096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940017+0000","last_change":"2024-04-26T20:07:31.712952+0000","last_active":"2024-04-26T20:08:48.940017+0000","last_peered":"2024-04-26T20:08:48.940017+0000","last_clean":"2024-04-26T20:08:48.940017+0000","last_became_active":"2024-04-26T20:07:31.712081+0000","last_became_peered":"2024-04-26T20:07:31.712081+0000","last_unstale":"2024-04-26T20:08:48.940017+0000","last_undegraded":"2024-04-26T20:08:48.940017+0000","last_fullsized":"2024-04-26T20:08:48.940017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:46.923536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111838+0000","last_change":"2024-04-26T20:08:14.585054+0000","last_active":"2024-04-26T20:08:16.111838+0000","last_peered":"2024-04-26T20:08:16.111838+0000","last_clean":"2024-04-26T20:08:16.111838+0000","last_became_active":"2024-04-26T20:08:14.584525+0000","last_became_peered":"2024-04-26T20:08:14.584525+0000","last_unstale":"2024-04-26T20:08:16.111838+0000","last_undegraded":"2024-04-26T20:08:16.111838+0000","last_fullsized":"2024-04-26T20:08:16.111838+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:58.518819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573478+0000","last_change":"2024-04-26T20:07:38.822520+0000","last_active":"2024-04-26T20:08:15.573478+0000","last_peered":"2024-04-26T20:08:15.573478+0000","last_clean":"2024-04-26T20:08:15.573478+0000","last_became_active":"2024-04-26T20:07:38.822056+0000","last_became_peered":"2024-04-26T20:07:38.822056+0000","last_unstale":"2024-04-26T20:08:15.573478+0000","last_undegraded":"2024-04-26T20:08:15.573478+0000","last_fullsized":"2024-04-26T20:08:15.573478+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.324996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"95'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940181+0000","last_change":"2024-04-26T20:07:31.724015+0000","last_active":"2024-04-26T20:08:48.940181+0000","last_peered":"2024-04-26T20:08:48.940181+0000","last_clean":"2024-04-26T20:08:48.940181+0000","last_became_active":"2024-04-26T20:07:31.723686+0000","last_became_peered":"2024-04-26T20:07:31.723686+0000","last_unstale":"2024-04-26T20:08:48.940181+0000","last_undegraded":"2024-04-26T20:08:48.940181+0000","last_fullsized":"2024-04-26T20:08:48.940181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:55.925280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374527+0000","last_change":"2024-04-26T20:07:34.750276+0000","last_active":"2024-04-26T20:08:16.374527+0000","last_peered":"2024-04-26T20:08:16.374527+0000","last_clean":"2024-04-26T20:08:16.374527+0000","last_became_active":"2024-04-26T20:07:34.749716+0000","last_became_peered":"2024-04-26T20:07:34.749716+0000","last_unstale":"2024-04-26T20:08:16.374527+0000","last_undegraded":"2024-04-26T20:08:16.374527+0000","last_fullsized":"2024-04-26T20:08:16.374527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:22.429780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.579105+0000","last_change":"2024-04-26T20:07:38.815636+0000","last_active":"2024-04-26T20:08:15.579105+0000","last_peered":"2024-04-26T20:08:15.579105+0000","last_clean":"2024-04-26T20:08:15.579105+0000","last_became_active":"2024-04-26T20:07:38.815278+0000","last_became_peered":"2024-04-26T20:07:38.815278+0000","last_unstale":"2024-04-26T20:08:15.579105+0000","last_undegraded":"2024-04-26T20:08:15.579105+0000","last_fullsized":"2024-04-26T20:08:15.579105+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:27.522132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'30","reported_seq":112,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:12.289544+0000","last_change":"2024-04-26T20:07:31.711393+0000","last_active":"2024-04-26T20:09:12.289544+0000","last_peered":"2024-04-26T20:09:12.289544+0000","last_clean":"2024-04-26T20:09:12.289544+0000","last_became_active":"2024-04-26T20:07:31.710998+0000","last_became_peered":"2024-04-26T20:07:31.710998+0000","last_unstale":"2024-04-26T20:09:12.289544+0000","last_undegraded":"2024-04-26T20:09:12.289544+0000","last_fullsized":"2024-04-26T20:09:12.289544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:00.674950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52480,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":47,"num_read_kb":74,"num_write":31,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568876+0000","last_change":"2024-04-26T20:07:34.752774+0000","last_active":"2024-04-26T20:08:15.568876+0000","last_peered":"2024-04-26T20:08:15.568876+0000","last_clean":"2024-04-26T20:08:15.568876+0000","last_became_active":"2024-04-26T20:07:34.752471+0000","last_became_peered":"2024-04-26T20:07:34.752471+0000","last_unstale":"2024-04-26T20:08:15.568876+0000","last_undegraded":"2024-04-26T20:08:15.568876+0000","last_fullsized":"2024-04-26T20:08:15.568876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.646231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155471+0000","last_change":"2024-04-26T20:07:38.830322+0000","last_active":"2024-04-26T20:08:16.155471+0000","last_peered":"2024-04-26T20:08:16.155471+0000","last_clean":"2024-04-26T20:08:16.155471+0000","last_became_active":"2024-04-26T20:07:38.829910+0000","last_became_peered":"2024-04-26T20:07:38.829910+0000","last_unstale":"2024-04-26T20:08:16.155471+0000","last_undegraded":"2024-04-26T20:08:16.155471+0000","last_fullsized":"2024-04-26T20:08:16.155471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:01.979793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.965190+0000","last_change":"2024-04-26T20:07:31.712054+0000","last_active":"2024-04-26T20:08:49.965190+0000","last_peered":"2024-04-26T20:08:49.965190+0000","last_clean":"2024-04-26T20:08:49.965190+0000","last_became_active":"2024-04-26T20:07:31.711618+0000","last_became_peered":"2024-04-26T20:07:31.711618+0000","last_unstale":"2024-04-26T20:08:49.965190+0000","last_undegraded":"2024-04-26T20:08:49.965190+0000","last_fullsized":"2024-04-26T20:08:49.965190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:46.465777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584167+0000","last_change":"2024-04-26T20:07:34.772993+0000","last_active":"2024-04-26T20:08:15.584167+0000","last_peered":"2024-04-26T20:08:15.584167+0000","last_clean":"2024-04-26T20:08:15.584167+0000","last_became_active":"2024-04-26T20:07:34.772587+0000","last_became_peered":"2024-04-26T20:07:34.772587+0000","last_unstale":"2024-04-26T20:08:15.584167+0000","last_undegraded":"2024-04-26T20:08:15.584167+0000","last_fullsized":"2024-04-26T20:08:15.584167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:38.459869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.588307+0000","last_change":"2024-04-26T20:08:15.588307+0000","last_active":"2024-04-26T20:08:15.588307+0000","last_peered":"2024-04-26T20:08:15.588307+0000","last_clean":"2024-04-26T20:08:15.588307+0000","last_became_active":"2024-04-26T20:08:15.587978+0000","last_became_peered":"2024-04-26T20:08:15.587978+0000","last_unstale":"2024-04-26T20:08:15.588307+0000","last_undegraded":"2024-04-26T20:08:15.588307+0000","last_fullsized":"2024-04-26T20:08:15.588307+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:36.491892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"95'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940179+0000","last_change":"2024-04-26T20:07:31.720938+0000","last_active":"2024-04-26T20:08:48.940179+0000","last_peered":"2024-04-26T20:08:48.940179+0000","last_clean":"2024-04-26T20:08:48.940179+0000","last_became_active":"2024-04-26T20:07:31.720579+0000","last_became_peered":"2024-04-26T20:07:31.720579+0000","last_unstale":"2024-04-26T20:08:48.940179+0000","last_undegraded":"2024-04-26T20:08:48.940179+0000","last_fullsized":"2024-04-26T20:08:48.940179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:55.737976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569252+0000","last_change":"2024-04-26T20:07:34.764802+0000","last_active":"2024-04-26T20:08:15.569252+0000","last_peered":"2024-04-26T20:08:15.569252+0000","last_clean":"2024-04-26T20:08:15.569252+0000","last_became_active":"2024-04-26T20:07:34.763468+0000","last_became_peered":"2024-04-26T20:07:34.763468+0000","last_unstale":"2024-04-26T20:08:15.569252+0000","last_undegraded":"2024-04-26T20:08:15.569252+0000","last_fullsized":"2024-04-26T20:08:15.569252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:23.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583848+0000","last_change":"2024-04-26T20:07:38.830392+0000","last_active":"2024-04-26T20:08:15.583848+0000","last_peered":"2024-04-26T20:08:15.583848+0000","last_clean":"2024-04-26T20:08:15.583848+0000","last_became_active":"2024-04-26T20:07:38.822964+0000","last_became_peered":"2024-04-26T20:07:38.822964+0000","last_unstale":"2024-04-26T20:08:15.583848+0000","last_undegraded":"2024-04-26T20:08:15.583848+0000","last_fullsized":"2024-04-26T20:08:15.583848+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:17.992368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'3","reported_seq":59,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:10.296275+0000","last_change":"2024-04-26T20:07:31.710567+0000","last_active":"2024-04-26T20:09:10.296275+0000","last_peered":"2024-04-26T20:09:10.296275+0000","last_clean":"2024-04-26T20:09:10.296275+0000","last_became_active":"2024-04-26T20:07:31.710197+0000","last_became_peered":"2024-04-26T20:07:31.710197+0000","last_unstale":"2024-04-26T20:09:10.296275+0000","last_undegraded":"2024-04-26T20:09:10.296275+0000","last_fullsized":"2024-04-26T20:09:10.296275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:10.648320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569128+0000","last_change":"2024-04-26T20:07:34.743237+0000","last_active":"2024-04-26T20:08:15.569128+0000","last_peered":"2024-04-26T20:08:15.569128+0000","last_clean":"2024-04-26T20:08:15.569128+0000","last_became_active":"2024-04-26T20:07:34.742146+0000","last_became_peered":"2024-04-26T20:07:34.742146+0000","last_unstale":"2024-04-26T20:08:15.569128+0000","last_undegraded":"2024-04-26T20:08:15.569128+0000","last_fullsized":"2024-04-26T20:08:15.569128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:15.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.591942+0000","last_change":"2024-04-26T20:08:15.591942+0000","last_active":"2024-04-26T20:08:15.591942+0000","last_peered":"2024-04-26T20:08:15.591942+0000","last_clean":"2024-04-26T20:08:15.591942+0000","last_became_active":"2024-04-26T20:08:15.590842+0000","last_became_peered":"2024-04-26T20:08:15.590842+0000","last_unstale":"2024-04-26T20:08:15.591942+0000","last_undegraded":"2024-04-26T20:08:15.591942+0000","last_fullsized":"2024-04-26T20:08:15.591942+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:44.344152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"95'15","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.964519+0000","last_change":"2024-04-26T20:07:31.716877+0000","last_active":"2024-04-26T20:08:49.964519+0000","last_peered":"2024-04-26T20:08:49.964519+0000","last_clean":"2024-04-26T20:08:49.964519+0000","last_became_active":"2024-04-26T20:07:31.716560+0000","last_became_peered":"2024-04-26T20:07:31.716560+0000","last_unstale":"2024-04-26T20:08:49.964519+0000","last_undegraded":"2024-04-26T20:08:49.964519+0000","last_fullsized":"2024-04-26T20:08:49.964519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:08:47.680991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154746+0000","last_change":"2024-04-26T20:07:34.749893+0000","last_active":"2024-04-26T20:08:16.154746+0000","last_peered":"2024-04-26T20:08:16.154746+0000","last_clean":"2024-04-26T20:08:16.154746+0000","last_became_active":"2024-04-26T20:07:34.748535+0000","last_became_peered":"2024-04-26T20:07:34.748535+0000","last_unstale":"2024-04-26T20:08:16.154746+0000","last_undegraded":"2024-04-26T20:08:16.154746+0000","last_fullsized":"2024-04-26T20:08:16.154746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:32.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154813+0000","last_change":"2024-04-26T20:07:38.824046+0000","last_active":"2024-04-26T20:08:16.154813+0000","last_peered":"2024-04-26T20:08:16.154813+0000","last_clean":"2024-04-26T20:08:16.154813+0000","last_became_active":"2024-04-26T20:07:38.823549+0000","last_became_peered":"2024-04-26T20:07:38.823549+0000","last_unstale":"2024-04-26T20:08:16.154813+0000","last_undegraded":"2024-04-26T20:08:16.154813+0000","last_fullsized":"2024-04-26T20:08:16.154813+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:26.218201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:50.955777+0000","last_change":"2024-04-26T20:07:31.711968+0000","last_active":"2024-04-26T20:08:50.955777+0000","last_peered":"2024-04-26T20:08:50.955777+0000","last_clean":"2024-04-26T20:08:50.955777+0000","last_became_active":"2024-04-26T20:07:31.710982+0000","last_became_peered":"2024-04-26T20:07:31.710982+0000","last_unstale":"2024-04-26T20:08:50.955777+0000","last_undegraded":"2024-04-26T20:08:50.955777+0000","last_fullsized":"2024-04-26T20:08:50.955777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:29.526536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571515+0000","last_change":"2024-04-26T20:07:34.758101+0000","last_active":"2024-04-26T20:08:15.571515+0000","last_peered":"2024-04-26T20:08:15.571515+0000","last_clean":"2024-04-26T20:08:15.571515+0000","last_became_active":"2024-04-26T20:07:34.757747+0000","last_became_peered":"2024-04-26T20:07:34.757747+0000","last_unstale":"2024-04-26T20:08:15.571515+0000","last_undegraded":"2024-04-26T20:08:15.571515+0000","last_fullsized":"2024-04-26T20:08:15.571515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.337040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568753+0000","last_change":"2024-04-26T20:07:39.312976+0000","last_active":"2024-04-26T20:08:15.568753+0000","last_peered":"2024-04-26T20:08:15.568753+0000","last_clean":"2024-04-26T20:08:15.568753+0000","last_became_active":"2024-04-26T20:07:39.312443+0000","last_became_peered":"2024-04-26T20:07:39.312443+0000","last_unstale":"2024-04-26T20:08:15.568753+0000","last_undegraded":"2024-04-26T20:08:15.568753+0000","last_fullsized":"2024-04-26T20:08:15.568753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:23.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.581507+0000","last_change":"2024-04-26T20:08:14.587592+0000","last_active":"2024-04-26T20:08:15.581507+0000","last_peered":"2024-04-26T20:08:15.581507+0000","last_clean":"2024-04-26T20:08:15.581507+0000","last_became_active":"2024-04-26T20:08:14.587157+0000","last_became_peered":"2024-04-26T20:08:14.587157+0000","last_unstale":"2024-04-26T20:08:15.581507+0000","last_undegraded":"2024-04-26T20:08:15.581507+0000","last_fullsized":"2024-04-26T20:08:15.581507+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:22.184416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.575875+0000","last_change":"2024-04-26T20:07:34.764376+0000","last_active":"2024-04-26T20:08:15.575875+0000","last_peered":"2024-04-26T20:08:15.575875+0000","last_clean":"2024-04-26T20:08:15.575875+0000","last_became_active":"2024-04-26T20:07:34.764040+0000","last_became_peered":"2024-04-26T20:07:34.764040+0000","last_unstale":"2024-04-26T20:08:15.575875+0000","last_undegraded":"2024-04-26T20:08:15.575875+0000","last_fullsized":"2024-04-26T20:08:15.575875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:43.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583488+0000","last_change":"2024-04-26T20:07:38.837075+0000","last_active":"2024-04-26T20:08:15.583488+0000","last_peered":"2024-04-26T20:08:15.583488+0000","last_clean":"2024-04-26T20:08:15.583488+0000","last_became_active":"2024-04-26T20:07:38.835912+0000","last_became_peered":"2024-04-26T20:07:38.835912+0000","last_unstale":"2024-04-26T20:08:15.583488+0000","last_undegraded":"2024-04-26T20:08:15.583488+0000","last_fullsized":"2024-04-26T20:08:15.583488+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.853262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569209+0000","last_change":"2024-04-26T20:07:31.728087+0000","last_active":"2024-04-26T20:08:15.569209+0000","last_peered":"2024-04-26T20:08:15.569209+0000","last_clean":"2024-04-26T20:08:15.569209+0000","last_became_active":"2024-04-26T20:07:31.727688+0000","last_became_peered":"2024-04-26T20:07:31.727688+0000","last_unstale":"2024-04-26T20:08:15.569209+0000","last_undegraded":"2024-04-26T20:08:15.569209+0000","last_fullsized":"2024-04-26T20:08:15.569209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:16.220049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.575695+0000","last_change":"2024-04-26T20:07:34.765161+0000","last_active":"2024-04-26T20:08:15.575695+0000","last_peered":"2024-04-26T20:08:15.575695+0000","last_clean":"2024-04-26T20:08:15.575695+0000","last_became_active":"2024-04-26T20:07:34.764789+0000","last_became_peered":"2024-04-26T20:07:34.764789+0000","last_unstale":"2024-04-26T20:08:15.575695+0000","last_undegraded":"2024-04-26T20:08:15.575695+0000","last_fullsized":"2024-04-26T20:08:15.575695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:52.417157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584352+0000","last_change":"2024-04-26T20:07:39.313159+0000","last_active":"2024-04-26T20:08:15.584352+0000","last_peered":"2024-04-26T20:08:15.584352+0000","last_clean":"2024-04-26T20:08:15.584352+0000","last_became_active":"2024-04-26T20:07:39.312637+0000","last_became_peered":"2024-04-26T20:07:39.312637+0000","last_unstale":"2024-04-26T20:08:15.584352+0000","last_undegraded":"2024-04-26T20:08:15.584352+0000","last_fullsized":"2024-04-26T20:08:15.584352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:10.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155127+0000","last_change":"2024-04-26T20:08:14.586352+0000","last_active":"2024-04-26T20:08:16.155127+0000","last_peered":"2024-04-26T20:08:16.155127+0000","last_clean":"2024-04-26T20:08:16.155127+0000","last_became_active":"2024-04-26T20:08:14.585975+0000","last_became_peered":"2024-04-26T20:08:14.585975+0000","last_unstale":"2024-04-26T20:08:16.155127+0000","last_undegraded":"2024-04-26T20:08:16.155127+0000","last_fullsized":"2024-04-26T20:08:16.155127+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:27.676683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'4","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:11.297935+0000","last_change":"2024-04-26T20:07:31.726272+0000","last_active":"2024-04-26T20:09:11.297935+0000","last_peered":"2024-04-26T20:09:11.297935+0000","last_clean":"2024-04-26T20:09:11.297935+0000","last_became_active":"2024-04-26T20:07:31.725889+0000","last_became_peered":"2024-04-26T20:07:31.725889+0000","last_unstale":"2024-04-26T20:09:11.297935+0000","last_undegraded":"2024-04-26T20:09:11.297935+0000","last_fullsized":"2024-04-26T20:09:11.297935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:03.091138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569447+0000","last_change":"2024-04-26T20:07:38.815171+0000","last_active":"2024-04-26T20:08:15.569447+0000","last_peered":"2024-04-26T20:08:15.569447+0000","last_clean":"2024-04-26T20:08:15.569447+0000","last_became_active":"2024-04-26T20:07:38.814769+0000","last_became_peered":"2024-04-26T20:07:38.814769+0000","last_unstale":"2024-04-26T20:08:15.569447+0000","last_undegraded":"2024-04-26T20:08:15.569447+0000","last_fullsized":"2024-04-26T20:08:15.569447+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571131+0000","last_change":"2024-04-26T20:07:34.766538+0000","last_active":"2024-04-26T20:08:15.571131+0000","last_peered":"2024-04-26T20:08:15.571131+0000","last_clean":"2024-04-26T20:08:15.571131+0000","last_became_active":"2024-04-26T20:07:34.761651+0000","last_became_peered":"2024-04-26T20:07:34.761651+0000","last_unstale":"2024-04-26T20:08:15.571131+0000","last_undegraded":"2024-04-26T20:08:15.571131+0000","last_fullsized":"2024-04-26T20:08:15.571131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:11.213046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'3","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:53.256297+0000","last_change":"2024-04-26T20:07:31.720351+0000","last_active":"2024-04-26T20:08:53.256297+0000","last_peered":"2024-04-26T20:08:53.256297+0000","last_clean":"2024-04-26T20:08:53.256297+0000","last_became_active":"2024-04-26T20:07:31.719966+0000","last_became_peered":"2024-04-26T20:07:31.719966+0000","last_unstale":"2024-04-26T20:08:53.256297+0000","last_undegraded":"2024-04-26T20:08:53.256297+0000","last_fullsized":"2024-04-26T20:08:53.256297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:17.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569035+0000","last_change":"2024-04-26T20:05:56.337783+0000","last_active":"2024-04-26T20:08:15.569035+0000","last_peered":"2024-04-26T20:08:15.569035+0000","last_clean":"2024-04-26T20:08:15.569035+0000","last_became_active":"2024-04-26T20:05:56.337301+0000","last_became_peered":"2024-04-26T20:05:56.337301+0000","last_unstale":"2024-04-26T20:08:15.569035+0000","last_undegraded":"2024-04-26T20:08:15.569035+0000","last_fullsized":"2024-04-26T20:08:15.569035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_clean_scrub_stamp":"2024-04-26T20:01:58.332403+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:45.808606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155395+0000","last_change":"2024-04-26T20:07:39.313210+0000","last_active":"2024-04-26T20:08:16.155395+0000","last_peered":"2024-04-26T20:08:16.155395+0000","last_clean":"2024-04-26T20:08:16.155395+0000","last_became_active":"2024-04-26T20:07:39.312575+0000","last_became_peered":"2024-04-26T20:07:39.312575+0000","last_unstale":"2024-04-26T20:08:16.155395+0000","last_undegraded":"2024-04-26T20:08:16.155395+0000","last_fullsized":"2024-04-26T20:08:16.155395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:02.877951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.582422+0000","last_change":"2024-04-26T20:07:34.759474+0000","last_active":"2024-04-26T20:08:15.582422+0000","last_peered":"2024-04-26T20:08:15.582422+0000","last_clean":"2024-04-26T20:08:15.582422+0000","last_became_active":"2024-04-26T20:07:34.759127+0000","last_became_peered":"2024-04-26T20:07:34.759127+0000","last_unstale":"2024-04-26T20:08:15.582422+0000","last_undegraded":"2024-04-26T20:08:15.582422+0000","last_fullsized":"2024-04-26T20:08:15.582422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.874866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.957623+0000","last_change":"2024-04-26T20:07:31.710633+0000","last_active":"2024-04-26T20:08:49.957623+0000","last_peered":"2024-04-26T20:08:49.957623+0000","last_clean":"2024-04-26T20:08:49.957623+0000","last_became_active":"2024-04-26T20:07:31.710290+0000","last_became_peered":"2024-04-26T20:07:31.710290+0000","last_unstale":"2024-04-26T20:08:49.957623+0000","last_undegraded":"2024-04-26T20:08:49.957623+0000","last_fullsized":"2024-04-26T20:08:49.957623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:18.114770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571337+0000","last_change":"2024-04-26T20:07:38.829768+0000","last_active":"2024-04-26T20:08:15.571337+0000","last_peered":"2024-04-26T20:08:15.571337+0000","last_clean":"2024-04-26T20:08:15.571337+0000","last_became_active":"2024-04-26T20:07:38.829364+0000","last_became_peered":"2024-04-26T20:07:38.829364+0000","last_unstale":"2024-04-26T20:08:15.571337+0000","last_undegraded":"2024-04-26T20:08:15.571337+0000","last_fullsized":"2024-04-26T20:08:15.571337+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:56.887062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"95'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.937916+0000","last_change":"2024-04-26T20:07:31.720296+0000","last_active":"2024-04-26T20:08:48.937916+0000","last_peered":"2024-04-26T20:08:48.937916+0000","last_clean":"2024-04-26T20:08:48.937916+0000","last_became_active":"2024-04-26T20:07:31.719918+0000","last_became_peered":"2024-04-26T20:07:31.719918+0000","last_unstale":"2024-04-26T20:08:48.937916+0000","last_undegraded":"2024-04-26T20:08:48.937916+0000","last_fullsized":"2024-04-26T20:08:48.937916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:43.783104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.583862+0000","last_change":"2024-04-26T20:07:34.770634+0000","last_active":"2024-04-26T20:08:15.583862+0000","last_peered":"2024-04-26T20:08:15.583862+0000","last_clean":"2024-04-26T20:08:15.583862+0000","last_became_active":"2024-04-26T20:07:34.770170+0000","last_became_peered":"2024-04-26T20:07:34.770170+0000","last_unstale":"2024-04-26T20:08:15.583862+0000","last_undegraded":"2024-04-26T20:08:15.583862+0000","last_fullsized":"2024-04-26T20:08:15.583862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:21.528735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571088+0000","last_change":"2024-04-26T20:07:39.317345+0000","last_active":"2024-04-26T20:08:15.571088+0000","last_peered":"2024-04-26T20:08:15.571088+0000","last_clean":"2024-04-26T20:08:15.571088+0000","last_became_active":"2024-04-26T20:07:39.315998+0000","last_became_peered":"2024-04-26T20:07:39.315998+0000","last_unstale":"2024-04-26T20:08:15.571088+0000","last_undegraded":"2024-04-26T20:08:15.571088+0000","last_fullsized":"2024-04-26T20:08:15.571088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:18.715213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:09:07.746531+0000","last_change":"2024-04-26T20:07:31.729195+0000","last_active":"2024-04-26T20:09:07.746531+0000","last_peered":"2024-04-26T20:09:07.746531+0000","last_clean":"2024-04-26T20:09:07.746531+0000","last_became_active":"2024-04-26T20:07:31.728746+0000","last_became_peered":"2024-04-26T20:07:31.728746+0000","last_unstale":"2024-04-26T20:09:07.746531+0000","last_undegraded":"2024-04-26T20:09:07.746531+0000","last_fullsized":"2024-04-26T20:09:07.746531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:34.443091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.572210+0000","last_change":"2024-04-26T20:07:34.765013+0000","last_active":"2024-04-26T20:08:15.572210+0000","last_peered":"2024-04-26T20:08:15.572210+0000","last_clean":"2024-04-26T20:08:15.572210+0000","last_became_active":"2024-04-26T20:07:34.764597+0000","last_became_peered":"2024-04-26T20:07:34.764597+0000","last_unstale":"2024-04-26T20:08:15.572210+0000","last_undegraded":"2024-04-26T20:08:15.572210+0000","last_fullsized":"2024-04-26T20:08:15.572210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:51.513026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.836755+0000","last_change":"2024-04-26T20:07:38.805910+0000","last_active":"2024-04-26T20:08:15.836755+0000","last_peered":"2024-04-26T20:08:15.836755+0000","last_clean":"2024-04-26T20:08:15.836755+0000","last_became_active":"2024-04-26T20:07:38.804580+0000","last_became_peered":"2024-04-26T20:07:38.804580+0000","last_unstale":"2024-04-26T20:08:15.836755+0000","last_undegraded":"2024-04-26T20:08:15.836755+0000","last_fullsized":"2024-04-26T20:08:15.836755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:41.321084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.942068+0000","last_change":"2024-04-26T20:07:31.915771+0000","last_active":"2024-04-26T20:08:48.942068+0000","last_peered":"2024-04-26T20:08:48.942068+0000","last_clean":"2024-04-26T20:08:48.942068+0000","last_became_active":"2024-04-26T20:07:31.915422+0000","last_became_peered":"2024-04-26T20:07:31.915422+0000","last_unstale":"2024-04-26T20:08:48.942068+0000","last_undegraded":"2024-04-26T20:08:48.942068+0000","last_fullsized":"2024-04-26T20:08:48.942068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:33.309270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111862+0000","last_change":"2024-04-26T20:07:34.775090+0000","last_active":"2024-04-26T20:08:16.111862+0000","last_peered":"2024-04-26T20:08:16.111862+0000","last_clean":"2024-04-26T20:08:16.111862+0000","last_became_active":"2024-04-26T20:07:34.774431+0000","last_became_peered":"2024-04-26T20:07:34.774431+0000","last_unstale":"2024-04-26T20:08:16.111862+0000","last_undegraded":"2024-04-26T20:08:16.111862+0000","last_fullsized":"2024-04-26T20:08:16.111862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:47.554047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573424+0000","last_change":"2024-04-26T20:07:39.313079+0000","last_active":"2024-04-26T20:08:15.573424+0000","last_peered":"2024-04-26T20:08:15.573424+0000","last_clean":"2024-04-26T20:08:15.573424+0000","last_became_active":"2024-04-26T20:07:39.312603+0000","last_became_peered":"2024-04-26T20:07:39.312603+0000","last_unstale":"2024-04-26T20:08:15.573424+0000","last_undegraded":"2024-04-26T20:08:15.573424+0000","last_fullsized":"2024-04-26T20:08:15.573424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:45.717375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.940438+0000","last_change":"2024-04-26T20:07:31.715373+0000","last_active":"2024-04-26T20:08:48.940438+0000","last_peered":"2024-04-26T20:08:48.940438+0000","last_clean":"2024-04-26T20:08:48.940438+0000","last_became_active":"2024-04-26T20:07:31.714973+0000","last_became_peered":"2024-04-26T20:07:31.714973+0000","last_unstale":"2024-04-26T20:08:48.940438+0000","last_undegraded":"2024-04-26T20:08:48.940438+0000","last_fullsized":"2024-04-26T20:08:48.940438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.772671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.568431+0000","last_change":"2024-04-26T20:07:34.753619+0000","last_active":"2024-04-26T20:08:15.568431+0000","last_peered":"2024-04-26T20:08:15.568431+0000","last_clean":"2024-04-26T20:08:15.568431+0000","last_became_active":"2024-04-26T20:07:34.753263+0000","last_became_peered":"2024-04-26T20:07:34.753263+0000","last_unstale":"2024-04-26T20:08:15.568431+0000","last_undegraded":"2024-04-26T20:08:15.568431+0000","last_fullsized":"2024-04-26T20:08:15.568431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.358153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.840243+0000","last_change":"2024-04-26T20:08:15.840243+0000","last_active":"2024-04-26T20:08:15.840243+0000","last_peered":"2024-04-26T20:08:15.840243+0000","last_clean":"2024-04-26T20:08:15.840243+0000","last_became_active":"2024-04-26T20:08:15.838891+0000","last_became_peered":"2024-04-26T20:08:15.838891+0000","last_unstale":"2024-04-26T20:08:15.840243+0000","last_undegraded":"2024-04-26T20:08:15.840243+0000","last_fullsized":"2024-04-26T20:08:15.840243+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:15.018199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"95'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.941460+0000","last_change":"2024-04-26T20:07:31.722516+0000","last_active":"2024-04-26T20:08:48.941460+0000","last_peered":"2024-04-26T20:08:48.941460+0000","last_clean":"2024-04-26T20:08:48.941460+0000","last_became_active":"2024-04-26T20:07:31.722096+0000","last_became_peered":"2024-04-26T20:07:31.722096+0000","last_unstale":"2024-04-26T20:08:48.941460+0000","last_undegraded":"2024-04-26T20:08:48.941460+0000","last_fullsized":"2024-04-26T20:08:48.941460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:57.693947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571173+0000","last_change":"2024-04-26T20:07:34.768754+0000","last_active":"2024-04-26T20:08:15.571173+0000","last_peered":"2024-04-26T20:08:15.571173+0000","last_clean":"2024-04-26T20:08:15.571173+0000","last_became_active":"2024-04-26T20:07:34.768405+0000","last_became_peered":"2024-04-26T20:07:34.768405+0000","last_unstale":"2024-04-26T20:08:15.571173+0000","last_undegraded":"2024-04-26T20:08:15.571173+0000","last_fullsized":"2024-04-26T20:08:15.571173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:08.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573877+0000","last_change":"2024-04-26T20:07:38.828804+0000","last_active":"2024-04-26T20:08:15.573877+0000","last_peered":"2024-04-26T20:08:15.573877+0000","last_clean":"2024-04-26T20:08:15.573877+0000","last_became_active":"2024-04-26T20:07:38.828452+0000","last_became_peered":"2024-04-26T20:07:38.828452+0000","last_unstale":"2024-04-26T20:08:15.573877+0000","last_undegraded":"2024-04-26T20:08:15.573877+0000","last_fullsized":"2024-04-26T20:08:15.573877+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:29.905102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837033+0000","last_change":"2024-04-26T20:07:31.710696+0000","last_active":"2024-04-26T20:08:15.837033+0000","last_peered":"2024-04-26T20:08:15.837033+0000","last_clean":"2024-04-26T20:08:15.837033+0000","last_became_active":"2024-04-26T20:07:31.710039+0000","last_became_peered":"2024-04-26T20:07:31.710039+0000","last_unstale":"2024-04-26T20:08:15.837033+0000","last_undegraded":"2024-04-26T20:08:15.837033+0000","last_fullsized":"2024-04-26T20:08:15.837033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:50.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571825+0000","last_change":"2024-04-26T20:07:34.767482+0000","last_active":"2024-04-26T20:08:15.571825+0000","last_peered":"2024-04-26T20:08:15.571825+0000","last_clean":"2024-04-26T20:08:15.571825+0000","last_became_active":"2024-04-26T20:07:34.767169+0000","last_became_peered":"2024-04-26T20:07:34.767169+0000","last_unstale":"2024-04-26T20:08:15.571825+0000","last_undegraded":"2024-04-26T20:08:15.571825+0000","last_fullsized":"2024-04-26T20:08:15.571825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:02.117983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.573346+0000","last_change":"2024-04-26T20:07:39.312800+0000","last_active":"2024-04-26T20:08:15.573346+0000","last_peered":"2024-04-26T20:08:15.573346+0000","last_clean":"2024-04-26T20:08:15.573346+0000","last_became_active":"2024-04-26T20:07:39.312309+0000","last_became_peered":"2024-04-26T20:07:39.312309+0000","last_unstale":"2024-04-26T20:08:15.573346+0000","last_undegraded":"2024-04-26T20:08:15.573346+0000","last_fullsized":"2024-04-26T20:08:15.573346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:02.013468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:48.939380+0000","last_change":"2024-04-26T20:07:31.715509+0000","last_active":"2024-04-26T20:08:48.939380+0000","last_peered":"2024-04-26T20:08:48.939380+0000","last_clean":"2024-04-26T20:08:48.939380+0000","last_became_active":"2024-04-26T20:07:31.714606+0000","last_became_peered":"2024-04-26T20:07:31.714606+0000","last_unstale":"2024-04-26T20:08:48.939380+0000","last_undegraded":"2024-04-26T20:08:48.939380+0000","last_fullsized":"2024-04-26T20:08:48.939380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:03.979681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569206+0000","last_change":"2024-04-26T20:07:34.763771+0000","last_active":"2024-04-26T20:08:15.569206+0000","last_peered":"2024-04-26T20:08:15.569206+0000","last_clean":"2024-04-26T20:08:15.569206+0000","last_became_active":"2024-04-26T20:07:34.763422+0000","last_became_peered":"2024-04-26T20:07:34.763422+0000","last_unstale":"2024-04-26T20:08:15.569206+0000","last_undegraded":"2024-04-26T20:08:15.569206+0000","last_fullsized":"2024-04-26T20:08:15.569206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:27.224024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374185+0000","last_change":"2024-04-26T20:07:38.830156+0000","last_active":"2024-04-26T20:08:16.374185+0000","last_peered":"2024-04-26T20:08:16.374185+0000","last_clean":"2024-04-26T20:08:16.374185+0000","last_became_active":"2024-04-26T20:07:38.829530+0000","last_became_peered":"2024-04-26T20:07:38.829530+0000","last_unstale":"2024-04-26T20:08:16.374185+0000","last_undegraded":"2024-04-26T20:08:16.374185+0000","last_fullsized":"2024-04-26T20:08:16.374185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:21.720974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:49.965661+0000","last_change":"2024-04-26T20:07:31.915638+0000","last_active":"2024-04-26T20:08:49.965661+0000","last_peered":"2024-04-26T20:08:49.965661+0000","last_clean":"2024-04-26T20:08:49.965661+0000","last_became_active":"2024-04-26T20:07:31.915266+0000","last_became_peered":"2024-04-26T20:07:31.915266+0000","last_unstale":"2024-04-26T20:08:49.965661+0000","last_undegraded":"2024-04-26T20:08:49.965661+0000","last_fullsized":"2024-04-26T20:08:49.965661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:51.452828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569604+0000","last_change":"2024-04-26T20:07:34.754476+0000","last_active":"2024-04-26T20:08:15.569604+0000","last_peered":"2024-04-26T20:08:15.569604+0000","last_clean":"2024-04-26T20:08:15.569604+0000","last_became_active":"2024-04-26T20:07:34.754054+0000","last_became_peered":"2024-04-26T20:07:34.754054+0000","last_unstale":"2024-04-26T20:08:15.569604+0000","last_undegraded":"2024-04-26T20:08:15.569604+0000","last_fullsized":"2024-04-26T20:08:15.569604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.497083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.110783+0000","last_change":"2024-04-26T20:07:38.835265+0000","last_active":"2024-04-26T20:08:16.110783+0000","last_peered":"2024-04-26T20:08:16.110783+0000","last_clean":"2024-04-26T20:08:16.110783+0000","last_became_active":"2024-04-26T20:07:38.833873+0000","last_became_peered":"2024-04-26T20:07:38.833873+0000","last_unstale":"2024-04-26T20:08:16.110783+0000","last_undegraded":"2024-04-26T20:08:16.110783+0000","last_fullsized":"2024-04-26T20:08:16.110783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:17.850236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.155313+0000","last_change":"2024-04-26T20:07:31.725576+0000","last_active":"2024-04-26T20:08:16.155313+0000","last_peered":"2024-04-26T20:08:16.155313+0000","last_clean":"2024-04-26T20:08:16.155313+0000","last_became_active":"2024-04-26T20:07:31.725166+0000","last_became_peered":"2024-04-26T20:07:31.725166+0000","last_unstale":"2024-04-26T20:08:16.155313+0000","last_undegraded":"2024-04-26T20:08:16.155313+0000","last_fullsized":"2024-04-26T20:08:16.155313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:04.539215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569433+0000","last_change":"2024-04-26T20:07:34.748263+0000","last_active":"2024-04-26T20:08:15.569433+0000","last_peered":"2024-04-26T20:08:15.569433+0000","last_clean":"2024-04-26T20:08:15.569433+0000","last_became_active":"2024-04-26T20:07:34.747889+0000","last_became_peered":"2024-04-26T20:07:34.747889+0000","last_unstale":"2024-04-26T20:08:15.569433+0000","last_undegraded":"2024-04-26T20:08:15.569433+0000","last_fullsized":"2024-04-26T20:08:15.569433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.257405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154483+0000","last_change":"2024-04-26T20:07:38.803999+0000","last_active":"2024-04-26T20:08:16.154483+0000","last_peered":"2024-04-26T20:08:16.154483+0000","last_clean":"2024-04-26T20:08:16.154483+0000","last_became_active":"2024-04-26T20:07:38.803585+0000","last_became_peered":"2024-04-26T20:07:38.803585+0000","last_unstale":"2024-04-26T20:08:16.154483+0000","last_undegraded":"2024-04-26T20:08:16.154483+0000","last_fullsized":"2024-04-26T20:08:16.154483+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:58.863511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.587505+0000","last_change":"2024-04-26T20:07:31.730445+0000","last_active":"2024-04-26T20:08:15.587505+0000","last_peered":"2024-04-26T20:08:15.587505+0000","last_clean":"2024-04-26T20:08:15.587505+0000","last_became_active":"2024-04-26T20:07:31.729944+0000","last_became_peered":"2024-04-26T20:07:31.729944+0000","last_unstale":"2024-04-26T20:08:15.587505+0000","last_undegraded":"2024-04-26T20:08:15.587505+0000","last_fullsized":"2024-04-26T20:08:15.587505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:06.274097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.569790+0000","last_change":"2024-04-26T20:07:34.755449+0000","last_active":"2024-04-26T20:08:15.569790+0000","last_peered":"2024-04-26T20:08:15.569790+0000","last_clean":"2024-04-26T20:08:15.569790+0000","last_became_active":"2024-04-26T20:07:34.755100+0000","last_became_peered":"2024-04-26T20:07:34.755100+0000","last_unstale":"2024-04-26T20:08:15.569790+0000","last_undegraded":"2024-04-26T20:08:15.569790+0000","last_fullsized":"2024-04-26T20:08:15.569790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.774751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571204+0000","last_change":"2024-04-26T20:07:34.760161+0000","last_active":"2024-04-26T20:08:15.571204+0000","last_peered":"2024-04-26T20:08:15.571204+0000","last_clean":"2024-04-26T20:08:15.571204+0000","last_became_active":"2024-04-26T20:07:34.759709+0000","last_became_peered":"2024-04-26T20:07:34.759709+0000","last_unstale":"2024-04-26T20:08:15.571204+0000","last_undegraded":"2024-04-26T20:08:15.571204+0000","last_fullsized":"2024-04-26T20:08:15.571204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:54.468818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837080+0000","last_change":"2024-04-26T20:07:34.747723+0000","last_active":"2024-04-26T20:08:15.837080+0000","last_peered":"2024-04-26T20:08:15.837080+0000","last_clean":"2024-04-26T20:08:15.837080+0000","last_became_active":"2024-04-26T20:07:34.747411+0000","last_became_peered":"2024-04-26T20:07:34.747411+0000","last_unstale":"2024-04-26T20:08:15.837080+0000","last_undegraded":"2024-04-26T20:08:15.837080+0000","last_fullsized":"2024-04-26T20:08:15.837080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.836846+0000","last_change":"2024-04-26T20:07:34.773012+0000","last_active":"2024-04-26T20:08:15.836846+0000","last_peered":"2024-04-26T20:08:15.836846+0000","last_clean":"2024-04-26T20:08:15.836846+0000","last_became_active":"2024-04-26T20:07:34.772735+0000","last_became_peered":"2024-04-26T20:07:34.772735+0000","last_unstale":"2024-04-26T20:08:15.836846+0000","last_undegraded":"2024-04-26T20:08:15.836846+0000","last_fullsized":"2024-04-26T20:08:15.836846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:08.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.111234+0000","last_change":"2024-04-26T20:07:34.762977+0000","last_active":"2024-04-26T20:08:16.111234+0000","last_peered":"2024-04-26T20:08:16.111234+0000","last_clean":"2024-04-26T20:08:16.111234+0000","last_became_active":"2024-04-26T20:07:34.762530+0000","last_became_peered":"2024-04-26T20:07:34.762530+0000","last_unstale":"2024-04-26T20:08:16.111234+0000","last_undegraded":"2024-04-26T20:08:16.111234+0000","last_fullsized":"2024-04-26T20:08:16.111234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:24.811825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584059+0000","last_change":"2024-04-26T20:07:34.769986+0000","last_active":"2024-04-26T20:08:15.584059+0000","last_peered":"2024-04-26T20:08:15.584059+0000","last_clean":"2024-04-26T20:08:15.584059+0000","last_became_active":"2024-04-26T20:07:34.769591+0000","last_became_peered":"2024-04-26T20:07:34.769591+0000","last_unstale":"2024-04-26T20:08:15.584059+0000","last_undegraded":"2024-04-26T20:08:15.584059+0000","last_fullsized":"2024-04-26T20:08:15.584059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:27.466675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571565+0000","last_change":"2024-04-26T20:07:34.754733+0000","last_active":"2024-04-26T20:08:15.571565+0000","last_peered":"2024-04-26T20:08:15.571565+0000","last_clean":"2024-04-26T20:08:15.571565+0000","last_became_active":"2024-04-26T20:07:34.754374+0000","last_became_peered":"2024-04-26T20:07:34.754374+0000","last_unstale":"2024-04-26T20:08:15.571565+0000","last_undegraded":"2024-04-26T20:08:15.571565+0000","last_fullsized":"2024-04-26T20:08:15.571565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:02.732025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.837375+0000","last_change":"2024-04-26T20:07:34.750329+0000","last_active":"2024-04-26T20:08:15.837375+0000","last_peered":"2024-04-26T20:08:15.837375+0000","last_clean":"2024-04-26T20:08:15.837375+0000","last_became_active":"2024-04-26T20:07:34.749970+0000","last_became_peered":"2024-04-26T20:07:34.749970+0000","last_unstale":"2024-04-26T20:08:15.837375+0000","last_undegraded":"2024-04-26T20:08:15.837375+0000","last_fullsized":"2024-04-26T20:08:15.837375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:50.375128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.572512+0000","last_change":"2024-04-26T20:07:34.750623+0000","last_active":"2024-04-26T20:08:15.572512+0000","last_peered":"2024-04-26T20:08:15.572512+0000","last_clean":"2024-04-26T20:08:15.572512+0000","last_became_active":"2024-04-26T20:07:34.750314+0000","last_became_peered":"2024-04-26T20:07:34.750314+0000","last_unstale":"2024-04-26T20:08:15.572512+0000","last_undegraded":"2024-04-26T20:08:15.572512+0000","last_fullsized":"2024-04-26T20:08:15.572512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:16.773986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.154828+0000","last_change":"2024-04-26T20:07:34.755474+0000","last_active":"2024-04-26T20:08:16.154828+0000","last_peered":"2024-04-26T20:08:16.154828+0000","last_clean":"2024-04-26T20:08:16.154828+0000","last_became_active":"2024-04-26T20:07:34.755133+0000","last_became_peered":"2024-04-26T20:07:34.755133+0000","last_unstale":"2024-04-26T20:08:16.154828+0000","last_undegraded":"2024-04-26T20:08:16.154828+0000","last_fullsized":"2024-04-26T20:08:16.154828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:02.861643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571466+0000","last_change":"2024-04-26T20:07:34.752727+0000","last_active":"2024-04-26T20:08:15.571466+0000","last_peered":"2024-04-26T20:08:15.571466+0000","last_clean":"2024-04-26T20:08:15.571466+0000","last_became_active":"2024-04-26T20:07:34.752420+0000","last_became_peered":"2024-04-26T20:07:34.752420+0000","last_unstale":"2024-04-26T20:08:15.571466+0000","last_undegraded":"2024-04-26T20:08:15.571466+0000","last_fullsized":"2024-04-26T20:08:15.571466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:45.076837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:16.374461+0000","last_change":"2024-04-26T20:07:34.775116+0000","last_active":"2024-04-26T20:08:16.374461+0000","last_peered":"2024-04-26T20:08:16.374461+0000","last_clean":"2024-04-26T20:08:16.374461+0000","last_became_active":"2024-04-26T20:07:34.774710+0000","last_became_peered":"2024-04-26T20:07:34.774710+0000","last_unstale":"2024-04-26T20:08:16.374461+0000","last_undegraded":"2024-04-26T20:08:16.374461+0000","last_fullsized":"2024-04-26T20:08:16.374461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:53.203918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.584777+0000","last_change":"2024-04-26T20:07:34.773245+0000","last_active":"2024-04-26T20:08:15.584777+0000","last_peered":"2024-04-26T20:08:15.584777+0000","last_clean":"2024-04-26T20:08:15.584777+0000","last_became_active":"2024-04-26T20:07:34.772879+0000","last_became_peered":"2024-04-26T20:07:34.772879+0000","last_unstale":"2024-04-26T20:08:15.584777+0000","last_undegraded":"2024-04-26T20:08:15.584777+0000","last_fullsized":"2024-04-26T20:08:15.584777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:54.450454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T20:08:15.571986+0000","last_change":"2024-04-26T20:07:34.753716+0000","last_active":"2024-04-26T20:08:15.571986+0000","last_peered":"2024-04-26T20:08:15.571986+0000","last_clean":"2024-04-26T20:08:15.571986+0000","last_became_active":"2024-04-26T20:07:34.753378+0000","last_became_peered":"2024-04-26T20:07:34.753378+0000","last_unstale":"2024-04-26T20:08:15.571986+0000","last_undegraded":"2024-04-26T20:08:15.571986+0000","last_fullsized":"2024-04-26T20:08:15.571986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:47.165305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56481,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":145,"num_read_kb":133,"num_write":157,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":169443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":535138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1174886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":539470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":535294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1131203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":530550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":582940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":531938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40620,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704532,"statfs":{"total":95995035648,"available":95953440768,"internally_reserved":0,"allocated":1150976,"data_stored":576680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":530744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":535200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1699840,"data_stored":1126791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:09:14.315 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:09:14.316 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T20:09:14.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T20:09:14.809 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: pgmap v385: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [1, 2]}]: dispatch 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [2, 9]}]: dispatch 2024-04-26T20:09:15.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:14 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-26T20:09:15.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: pgmap v385: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T20:09:15.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:09:15.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:09:15.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-04-26T20:09:15.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-26T20:09:15.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-04-26T20:09:15.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [1, 2]}]: dispatch 2024-04-26T20:09:15.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [2, 9]}]: dispatch 2024-04-26T20:09:15.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:14 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-26T20:09:15.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: pgmap v385: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T20:09:15.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:09:15.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:09:15.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-04-26T20:09:15.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-26T20:09:15.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-04-26T20:09:15.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [1, 2]}]: dispatch 2024-04-26T20:09:15.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [2, 9]}]: dispatch 2024-04-26T20:09:15.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:14 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-26T20:09:16.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-04-26T20:09:16.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-26T20:09:16.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-04-26T20:09:16.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [1, 2]}]': finished 2024-04-26T20:09:16.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [2, 9]}]': finished 2024-04-26T20:09:16.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-26T20:09:16.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:15 smithi148 ceph-mon[32281]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T20:09:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-04-26T20:09:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-26T20:09:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-04-26T20:09:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [1, 2]}]': finished 2024-04-26T20:09:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [2, 9]}]': finished 2024-04-26T20:09:16.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-26T20:09:16.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:15 smithi157 ceph-mon[32703]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T20:09:16.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-04-26T20:09:16.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-26T20:09:16.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-04-26T20:09:16.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [1, 2]}]': finished 2024-04-26T20:09:16.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [2, 9]}]': finished 2024-04-26T20:09:16.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-26T20:09:16.257 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:15 smithi131 ceph-mon[27919]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T20:09:17.004 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:16 smithi131 ceph-mon[27919]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-04-26T20:09:17.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:16 smithi131 ceph-mon[27919]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T20:09:17.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:16 smithi131 ceph-mon[27919]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T20:09:17.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:16 smithi148 ceph-mon[32281]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-04-26T20:09:17.072 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:16 smithi148 ceph-mon[32281]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T20:09:17.078 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:16 smithi148 ceph-mon[32281]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T20:09:17.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:16 smithi157 ceph-mon[32703]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-04-26T20:09:17.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:16 smithi157 ceph-mon[32703]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T20:09:17.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:16 smithi157 ceph-mon[32703]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T20:09:17.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 -- 172.21.15.131:0/784375926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 msgr2=0x7f78581074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:17.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 --2- 172.21.15.131:0/784375926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 0x7f78581074e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f785000b0a0 tx=0x7f785000b3b0 comp rx=0 tx=0).stop 2024-04-26T20:09:17.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 -- 172.21.15.131:0/784375926 shutdown_connections 2024-04-26T20:09:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 --2- 172.21.15.131:0/784375926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 0x7f78581074e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:17.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 --2- 172.21.15.131:0/784375926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7858101d20 0x7f7858104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:17.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 --2- 172.21.15.131:0/784375926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858104690 0x7f7858106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:17.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 -- 172.21.15.131:0/784375926 >> 172.21.15.131:0/784375926 conn(0x7f78580fd940 msgr2=0x7f78580ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:17.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 -- 172.21.15.131:0/784375926 shutdown_connections 2024-04-26T20:09:17.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.338+0000 7f7860600700 1 -- 172.21.15.131:0/784375926 wait complete. 2024-04-26T20:09:17.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 Processor -- start 2024-04-26T20:09:17.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 -- start start 2024-04-26T20:09:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858101d20 0x7f785819fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:17.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7858104690 0x7f785819ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:17.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 0x7f78581a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:17.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7858115500 con 0x7f7858101d20 2024-04-26T20:09:17.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7858115380 con 0x7f7858107060 2024-04-26T20:09:17.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.340+0000 7f7860600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7858115680 con 0x7f7858104690 2024-04-26T20:09:17.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858101d20 0x7f785819fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:17.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7858104690 0x7f785819ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:17.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858101d20 0x7f785819fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52632/0 (socket says 172.21.15.131:52632) 2024-04-26T20:09:17.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785dc00700 1 -- 172.21.15.131:0/2850124128 learned_addr learned my addr 172.21.15.131:0/2850124128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785e600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 0x7f78581a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785e600700 1 -- 172.21.15.131:0/2850124128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7858104690 msgr2=0x7f785819ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785e600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7858104690 0x7f785819ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785e600700 1 -- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858101d20 msgr2=0x7f785819fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785e600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858101d20 0x7f785819fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:17.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785e600700 1 -- 172.21.15.131:0/2850124128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78581a7690 con 0x7f7858107060 2024-04-26T20:09:17.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.341+0000 7f785dc00700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858101d20 0x7f785819fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:17.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.342+0000 7f785e600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 0x7f78581a6ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7850009fd0 tx=0x7f7850004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:17.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.342+0000 7f7856000700 1 -- 172.21.15.131:0/2850124128 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850021070 con 0x7f7858107060 2024-04-26T20:09:17.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.342+0000 7f7856000700 1 -- 172.21.15.131:0/2850124128 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7850007b80 con 0x7f7858107060 2024-04-26T20:09:17.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.342+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78581a7920 con 0x7f7858107060 2024-04-26T20:09:17.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.342+0000 7f7856000700 1 -- 172.21.15.131:0/2850124128 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78500052a0 con 0x7f7858107060 2024-04-26T20:09:17.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.342+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581a7e30 con 0x7f7858107060 2024-04-26T20:09:17.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.343+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7858051960 con 0x7f7858107060 2024-04-26T20:09:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.345+0000 7f7856000700 1 -- 172.21.15.131:0/2850124128 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7850024070 con 0x7f7858107060 2024-04-26T20:09:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.347+0000 7f7856000700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78400a3e00 0x7f78400a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:17.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.347+0000 7f7856000700 1 -- 172.21.15.131:0/2850124128 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f78500d18e0 con 0x7f7858107060 2024-04-26T20:09:17.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.347+0000 7f785dc00700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78400a3e00 0x7f78400a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:17.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.348+0000 7f785dc00700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78400a3e00 0x7f78400a62c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f784c011590 tx=0x7f784c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:17.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.351+0000 7f7856000700 1 -- 172.21.15.131:0/2850124128 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f785009dda0 con 0x7f7858107060 2024-04-26T20:09:17.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.671+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7858064030 con 0x7f7858107060 2024-04-26T20:09:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.682+0000 7f7856000700 1 -- 172.21.15.131:0/2850124128 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f785009d6a0 con 0x7f7858107060 2024-04-26T20:09:17.927 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-04-26T20:09:17.990 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDtCSxm/JY4KBAATGGEPWXzrkIg9lc1W3LkqQ== 2024-04-26T20:09:18.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78400a3e00 msgr2=0x7f78400a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:18.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78400a3e00 0x7f78400a62c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f784c011590 tx=0x7f784c009260 comp rx=0 tx=0).stop 2024-04-26T20:09:18.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 msgr2=0x7f78581a6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:18.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 0x7f78581a6ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7850009fd0 tx=0x7f7850004cf0 comp rx=0 tx=0).stop 2024-04-26T20:09:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 shutdown_connections 2024-04-26T20:09:18.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78400a3e00 0x7f78400a62c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:18.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858107060 0x7f78581a6ff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7858101d20 0x7f785819fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:18.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 --2- 172.21.15.131:0/2850124128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7858104690 0x7f785819ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 >> 172.21.15.131:0/2850124128 conn(0x7f78580fd940 msgr2=0x7f78581029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 shutdown_connections 2024-04-26T20:09:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:17.685+0000 7f7860600700 1 -- 172.21.15.131:0/2850124128 wait complete. 2024-04-26T20:09:18.104 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:17 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2850124128' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T20:09:18.110 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:17 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T20:09:18.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:17 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T20:09:18.161 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:17 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2850124128' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T20:09:18.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:17 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T20:09:18.303 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:17 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T20:09:18.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:17 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2850124128' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T20:09:18.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:17 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T20:09:18.333 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:17 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T20:09:18.600 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:09:18.618 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T20:09:18.624 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T20:09:18.735 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T20:09:18.781 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T20:09:18.786 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T20:09:19.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:18 smithi148 ceph-mon[32281]: pgmap v389: 129 pgs: 3 peering, 126 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-26T20:09:19.222 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:19.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:18 smithi157 ceph-mon[32703]: pgmap v389: 129 pgs: 3 peering, 126 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-26T20:09:19.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:18 smithi131 ceph-mon[27919]: pgmap v389: 129 pgs: 3 peering, 126 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-26T20:09:21.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:20 smithi131 ceph-mon[27919]: pgmap v390: 129 pgs: 3 peering, 126 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-26T20:09:21.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:20 smithi148 ceph-mon[32281]: pgmap v390: 129 pgs: 3 peering, 126 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-26T20:09:21.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:20 smithi157 ceph-mon[32703]: pgmap v390: 129 pgs: 3 peering, 126 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-26T20:09:21.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 -- 172.21.15.131:0/2415390924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c103470 msgr2=0x7f834c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:21.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 --2- 172.21.15.131:0/2415390924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c103470 0x7f834c1038f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8338009990 tx=0x7f8338009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:21.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 -- 172.21.15.131:0/2415390924 shutdown_connections 2024-04-26T20:09:21.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 --2- 172.21.15.131:0/2415390924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f834c103e30 0x7f834c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:21.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 --2- 172.21.15.131:0/2415390924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c1094c0 0x7f834c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:21.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 --2- 172.21.15.131:0/2415390924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c103470 0x7f834c1038f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:21.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 -- 172.21.15.131:0/2415390924 >> 172.21.15.131:0/2415390924 conn(0x7f834c07a580 msgr2=0x7f834c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:21.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 -- 172.21.15.131:0/2415390924 shutdown_connections 2024-04-26T20:09:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.694+0000 7f8352800700 1 -- 172.21.15.131:0/2415390924 wait complete. 2024-04-26T20:09:21.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.695+0000 7f8352800700 1 Processor -- start 2024-04-26T20:09:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.695+0000 7f8352800700 1 -- start start 2024-04-26T20:09:21.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f8352800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f834c103470 0x7f834c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f8352800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c103e30 0x7f834c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:21.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f8352800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c1094c0 0x7f834c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:21.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f8352800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f834c1135e0 con 0x7f834c1094c0 2024-04-26T20:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f8352800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f834c113460 con 0x7f834c103470 2024-04-26T20:09:21.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f8352800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f834c113760 con 0x7f834c103e30 2024-04-26T20:09:21.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f834aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c103e30 0x7f834c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:21.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f834aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c103e30 0x7f834c075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35618/0 (socket says 172.21.15.131:35618) 2024-04-26T20:09:21.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f834aa00700 1 -- 172.21.15.131:0/1814126264 learned_addr learned my addr 172.21.15.131:0/1814126264 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:22.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f834be00700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c1094c0 0x7f834c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:22.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.696+0000 7f834b400700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f834c103470 0x7f834c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:22.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.697+0000 7f834aa00700 1 -- 172.21.15.131:0/1814126264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f834c103470 msgr2=0x7f834c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:22.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.697+0000 7f834aa00700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f834c103470 0x7f834c075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.697+0000 7f834aa00700 1 -- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c1094c0 msgr2=0x7f834c0761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.697+0000 7f834aa00700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c1094c0 0x7f834c0761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:22.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.697+0000 7f834aa00700 1 -- 172.21.15.131:0/1814126264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c1a8120 con 0x7f834c103e30 2024-04-26T20:09:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.697+0000 7f834aa00700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c103e30 0x7f834c075c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83380057f0 tx=0x7f833800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:22.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.697+0000 7f8343e00700 1 -- 172.21.15.131:0/1814126264 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338026070 con 0x7f834c103e30 2024-04-26T20:09:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.698+0000 7f8343e00700 1 -- 172.21.15.131:0/1814126264 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8338021410 con 0x7f834c103e30 2024-04-26T20:09:22.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.698+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834c1a83b0 con 0x7f834c103e30 2024-04-26T20:09:22.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.698+0000 7f8343e00700 1 -- 172.21.15.131:0/1814126264 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338017650 con 0x7f834c103e30 2024-04-26T20:09:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.698+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834c1a88e0 con 0x7f834c103e30 2024-04-26T20:09:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.699+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f834c06a110 con 0x7f834c103e30 2024-04-26T20:09:22.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.701+0000 7f8343e00700 1 -- 172.21.15.131:0/1814126264 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8338029070 con 0x7f834c103e30 2024-04-26T20:09:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.702+0000 7f8343e00700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83300a3d50 0x7f83300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.703+0000 7f8343e00700 1 -- 172.21.15.131:0/1814126264 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f83380d1730 con 0x7f834c103e30 2024-04-26T20:09:22.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.703+0000 7f834b400700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83300a3d50 0x7f83300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:22.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.703+0000 7f834b400700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83300a3d50 0x7f83300a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f833c009be0 tx=0x7f833c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.705+0000 7f8343e00700 1 -- 172.21.15.131:0/1814126264 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8338013070 con 0x7f834c103e30 2024-04-26T20:09:22.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:21 smithi148 ceph-mon[32281]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-04-26T20:09:22.347 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:21 smithi148 ceph-mon[32281]: Cluster is now healthy 2024-04-26T20:09:22.361 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:21 smithi157 ceph-mon[32703]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-04-26T20:09:22.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:21 smithi157 ceph-mon[32703]: Cluster is now healthy 2024-04-26T20:09:22.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:21.996+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f834c076c50 con 0x7f834c103e30 2024-04-26T20:09:22.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:21 smithi131 ceph-mon[27919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-04-26T20:09:22.438 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:21 smithi131 ceph-mon[27919]: Cluster is now healthy 2024-04-26T20:09:22.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.002+0000 7f8343e00700 1 -- 172.21.15.131:0/1814126264 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265005 (secure 0 0 0) 0x7f833809dbf0 con 0x7f834c103e30 2024-04-26T20:09:22.459 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:09:22.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83300a3d50 msgr2=0x7f83300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:22.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83300a3d50 0x7f83300a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f833c009be0 tx=0x7f833c009260 comp rx=0 tx=0).stop 2024-04-26T20:09:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c103e30 msgr2=0x7f834c075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c103e30 0x7f834c075c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83380057f0 tx=0x7f833800bfd0 comp rx=0 tx=0).stop 2024-04-26T20:09:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 shutdown_connections 2024-04-26T20:09:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f834c1094c0 0x7f834c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83300a3d50 0x7f83300a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f834c103470 0x7f834c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 --2- 172.21.15.131:0/1814126264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834c103e30 0x7f834c075c60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:22.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 >> 172.21.15.131:0/1814126264 conn(0x7f834c07a580 msgr2=0x7f834c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:22.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 shutdown_connections 2024-04-26T20:09:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:22.009+0000 7f8352800700 1 -- 172.21.15.131:0/1814126264 wait complete. 2024-04-26T20:09:22.917 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2537375548},{"type":"v1","addr":"172.21.15.131:6801","nonce":2537375548}]},"active_addr":"172.21.15.131:6801/2537375548","active_change":"2024-04-26T19:59:13.443306+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.131:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3818652562}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":245853723}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":4133589329}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":731526281}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":4099508339}]}]} 2024-04-26T20:09:22.984 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T20:09:22.993 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T20:09:23.002 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T20:09:23.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:22 smithi131 ceph-mon[27919]: pgmap v391: 129 pgs: 129 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-26T20:09:23.135 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:22 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1814126264' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T20:09:23.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:22 smithi157 ceph-mon[32703]: pgmap v391: 129 pgs: 129 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-26T20:09:23.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:22 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1814126264' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T20:09:23.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:22 smithi148 ceph-mon[32281]: pgmap v391: 129 pgs: 129 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-26T20:09:23.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:22 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1814126264' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T20:09:23.456 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:24.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:09:24.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:09:24.268 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:09:25.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:25 smithi131 ceph-mon[27919]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 809 B/s wr, 7 op/s 2024-04-26T20:09:25.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.293 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.360 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.375 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:25 smithi157 ceph-mon[32703]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 809 B/s wr, 7 op/s 2024-04-26T20:09:25.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.615 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.634 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.657 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.671 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.696 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:25 smithi148 ceph-mon[32281]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 809 B/s wr, 7 op/s 2024-04-26T20:09:25.710 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.722 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.745 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.919+0000 7fa382400700 1 -- 172.21.15.131:0/2756131029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075f70 msgr2=0x7fa37c10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:25.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.919+0000 7fa382400700 1 --2- 172.21.15.131:0/2756131029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075f70 0x7fa37c10f8e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa364009990 tx=0x7fa364009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:25.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.920+0000 7fa382400700 1 -- 172.21.15.131:0/2756131029 shutdown_connections 2024-04-26T20:09:25.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.920+0000 7fa382400700 1 --2- 172.21.15.131:0/2756131029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa37c10fe20 0x7fa37c112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.920+0000 7fa382400700 1 --2- 172.21.15.131:0/2756131029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075650 0x7fa37c075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.920+0000 7fa382400700 1 --2- 172.21.15.131:0/2756131029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075f70 0x7fa37c10f8e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.920+0000 7fa382400700 1 -- 172.21.15.131:0/2756131029 >> 172.21.15.131:0/2756131029 conn(0x7fa37c07a580 msgr2=0x7fa37c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:26.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.920+0000 7fa382400700 1 -- 172.21.15.131:0/2756131029 shutdown_connections 2024-04-26T20:09:26.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.920+0000 7fa382400700 1 -- 172.21.15.131:0/2756131029 wait complete. 2024-04-26T20:09:26.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 Processor -- start 2024-04-26T20:09:26.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 -- start start 2024-04-26T20:09:26.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075650 0x7fa37c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075f70 0x7fa37c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:26.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa37c10fe20 0x7fa37c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:26.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c115850 con 0x7fa37c075f70 2024-04-26T20:09:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1156d0 con 0x7fa37c075650 2024-04-26T20:09:26.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.921+0000 7fa382400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1159d0 con 0x7fa37c10fe20 2024-04-26T20:09:26.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075f70 0x7fa37c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:26.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa37b400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075650 0x7fa37c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:26.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075f70 0x7fa37c1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59374/0 (socket says 172.21.15.131:59374) 2024-04-26T20:09:26.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 -- 172.21.15.131:0/3582137649 learned_addr learned my addr 172.21.15.131:0/3582137649 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:26.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa37be00700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa37c10fe20 0x7fa37c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 -- 172.21.15.131:0/3582137649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa37c10fe20 msgr2=0x7fa37c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:26.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa37c10fe20 0x7fa37c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 -- 172.21.15.131:0/3582137649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075650 msgr2=0x7fa37c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075650 0x7fa37c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 -- 172.21.15.131:0/3582137649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c19c450 con 0x7fa37c075f70 2024-04-26T20:09:26.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa37b400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075650 0x7fa37c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:26.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.922+0000 7fa372a00700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075f70 0x7fa37c1a1f90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa364000c00 tx=0x7fa3640057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:26.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.923+0000 7fa378c00700 1 -- 172.21.15.131:0/3582137649 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364027070 con 0x7fa37c075f70 2024-04-26T20:09:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.923+0000 7fa378c00700 1 -- 172.21.15.131:0/3582137649 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa364022470 con 0x7fa37c075f70 2024-04-26T20:09:26.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.923+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c19c6e0 con 0x7fa37c075f70 2024-04-26T20:09:26.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.923+0000 7fa378c00700 1 -- 172.21.15.131:0/3582137649 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364017650 con 0x7fa37c075f70 2024-04-26T20:09:26.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.923+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c1a8800 con 0x7fa37c075f70 2024-04-26T20:09:26.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.924+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c076d40 con 0x7fa37c075f70 2024-04-26T20:09:26.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.926+0000 7fa378c00700 1 -- 172.21.15.131:0/3582137649 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa36402a070 con 0x7fa37c075f70 2024-04-26T20:09:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.928+0000 7fa378c00700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3600a3cd0 0x7fa3600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.928+0000 7fa378c00700 1 -- 172.21.15.131:0/3582137649 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa3640d1690 con 0x7fa37c075f70 2024-04-26T20:09:26.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.928+0000 7fa37b400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3600a3cd0 0x7fa3600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:26.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.929+0000 7fa37b400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3600a3cd0 0x7fa3600a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa36c011490 tx=0x7fa36c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:26.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:25.931+0000 7fa378c00700 1 -- 172.21.15.131:0/3582137649 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa364013070 con 0x7fa37c075f70 2024-04-26T20:09:26.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.157+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa37c02e490 con 0x7fa37c075f70 2024-04-26T20:09:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.159+0000 7fa378c00700 1 -- 172.21.15.131:0/3582137649 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7fa36409db50 con 0x7fa37c075f70 2024-04-26T20:09:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:09:26.712 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:09:26.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.162+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3600a3cd0 msgr2=0x7fa3600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:26.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3600a3cd0 0x7fa3600a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa36c011490 tx=0x7fa36c009260 comp rx=0 tx=0).stop 2024-04-26T20:09:26.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075f70 msgr2=0x7fa37c1a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:26.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075f70 0x7fa37c1a1f90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa364000c00 tx=0x7fa3640057c0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 shutdown_connections 2024-04-26T20:09:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa37c10fe20 0x7fa37c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa3600a3cd0 0x7fa3600a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c075650 0x7fa37c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 --2- 172.21.15.131:0/3582137649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa37c075f70 0x7fa37c1a1f90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 >> 172.21.15.131:0/3582137649 conn(0x7fa37c07a580 msgr2=0x7fa37c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:26.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.163+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 shutdown_connections 2024-04-26T20:09:26.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:26.164+0000 7fa382400700 1 -- 172.21.15.131:0/3582137649 wait complete. 2024-04-26T20:09:27.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:27 smithi157 ceph-mon[32703]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2024-04-26T20:09:27.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:27 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3582137649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:27.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:27 smithi131 ceph-mon[27919]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2024-04-26T20:09:27.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:27 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3582137649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:27.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:27 smithi148 ceph-mon[32281]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2024-04-26T20:09:27.574 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:27 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3582137649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:27.685 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T20:09:27.712 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T20:09:28.162 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:29.429 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:29 smithi131 ceph-mon[27919]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 518 B/s wr, 6 op/s 2024-04-26T20:09:29.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:29 smithi157 ceph-mon[32703]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 518 B/s wr, 6 op/s 2024-04-26T20:09:29.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:29 smithi148 ceph-mon[32281]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 518 B/s wr, 6 op/s 2024-04-26T20:09:30.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 -- 172.21.15.131:0/3669009319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103470 msgr2=0x7f75181038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:30.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 --2- 172.21.15.131:0/3669009319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103470 0x7f75181038f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7504009990 tx=0x7f7504009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:30.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 -- 172.21.15.131:0/3669009319 shutdown_connections 2024-04-26T20:09:30.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 --2- 172.21.15.131:0/3669009319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7518103e30 0x7f7518110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:30.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 --2- 172.21.15.131:0/3669009319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181094c0 0x7f75181098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:30.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 --2- 172.21.15.131:0/3669009319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103470 0x7f75181038f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:30.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 -- 172.21.15.131:0/3669009319 >> 172.21.15.131:0/3669009319 conn(0x7f751807a580 msgr2=0x7f751807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 -- 172.21.15.131:0/3669009319 shutdown_connections 2024-04-26T20:09:30.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.388+0000 7f751da00700 1 -- 172.21.15.131:0/3669009319 wait complete. 2024-04-26T20:09:30.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.389+0000 7f751da00700 1 Processor -- start 2024-04-26T20:09:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.389+0000 7f751da00700 1 -- start start 2024-04-26T20:09:30.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f751da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7518103470 0x7f751819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f751da00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103e30 0x7f751819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f751da00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181094c0 0x7f75181a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:30.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f751da00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f75181135e0 con 0x7f7518103470 2024-04-26T20:09:30.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f751da00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7518113460 con 0x7f75181094c0 2024-04-26T20:09:30.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f751da00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7518113760 con 0x7f7518103e30 2024-04-26T20:09:30.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f7516000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103e30 0x7f751819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f7517400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181094c0 0x7f75181a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:30.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f7517400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181094c0 0x7f75181a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42130/0 (socket says 172.21.15.131:42130) 2024-04-26T20:09:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.390+0000 7f7517400700 1 -- 172.21.15.131:0/1841681641 learned_addr learned my addr 172.21.15.131:0/1841681641 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:30.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.391+0000 7f7516000700 1 -- 172.21.15.131:0/1841681641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181094c0 msgr2=0x7f75181a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:30.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.391+0000 7f7516000700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181094c0 0x7f75181a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:30.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.391+0000 7f7516000700 1 -- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7518103470 msgr2=0x7f751819bdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:09:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.391+0000 7f7516a00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7518103470 0x7f751819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:30.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.391+0000 7f7516000700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7518103470 0x7f751819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.391+0000 7f7516000700 1 -- 172.21.15.131:0/1841681641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75181a0e00 con 0x7f7518103e30 2024-04-26T20:09:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.391+0000 7f7516a00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7518103470 0x7f751819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:09:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.392+0000 7f7516000700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103e30 0x7f751819c330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75040059d0 tx=0x7f75040047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.392+0000 7f750f400700 1 -- 172.21.15.131:0/1841681641 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504027070 con 0x7f7518103e30 2024-04-26T20:09:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.392+0000 7f750f400700 1 -- 172.21.15.131:0/1841681641 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7504022470 con 0x7f7518103e30 2024-04-26T20:09:30.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.392+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75181a1090 con 0x7f7518103e30 2024-04-26T20:09:30.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.392+0000 7f750f400700 1 -- 172.21.15.131:0/1841681641 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504017650 con 0x7f7518103e30 2024-04-26T20:09:30.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.392+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181a8970 con 0x7f7518103e30 2024-04-26T20:09:30.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.393+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751806a110 con 0x7f7518103e30 2024-04-26T20:09:30.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.395+0000 7f750f400700 1 -- 172.21.15.131:0/1841681641 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f750402a070 con 0x7f7518103e30 2024-04-26T20:09:30.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.396+0000 7f750f400700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74fc0a3cd0 0x7f74fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:30.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.397+0000 7f750f400700 1 -- 172.21.15.131:0/1841681641 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f75040d1f10 con 0x7f7518103e30 2024-04-26T20:09:30.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.397+0000 7f7516a00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74fc0a3cd0 0x7f74fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:30.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.399+0000 7f7516a00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74fc0a3cd0 0x7f74fc0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7508009bc0 tx=0x7f7508009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.399+0000 7f750f400700 1 -- 172.21.15.131:0/1841681641 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7504031070 con 0x7f7518103e30 2024-04-26T20:09:31.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.052 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.059 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.648+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f751802e490 con 0x7f7518103e30 2024-04-26T20:09:31.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.650+0000 7f750f400700 1 -- 172.21.15.131:0/1841681641 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7f7504013070 con 0x7f7518103e30 2024-04-26T20:09:31.095 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:09:31.123 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:09:31.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74fc0a3cd0 msgr2=0x7f74fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:31.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74fc0a3cd0 0x7f74fc0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7508009bc0 tx=0x7f7508009260 comp rx=0 tx=0).stop 2024-04-26T20:09:31.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103e30 msgr2=0x7f751819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:31.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103e30 0x7f751819c330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75040059d0 tx=0x7f75040047b0 comp rx=0 tx=0).stop 2024-04-26T20:09:31.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 shutdown_connections 2024-04-26T20:09:31.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74fc0a3cd0 0x7f74fc0a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:31.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181094c0 0x7f75181a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:31.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7518103470 0x7f751819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:31.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 --2- 172.21.15.131:0/1841681641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7518103e30 0x7f751819c330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:31.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 >> 172.21.15.131:0/1841681641 conn(0x7f751807a580 msgr2=0x7f751810e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:31.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 shutdown_connections 2024-04-26T20:09:31.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:30.653+0000 7f751da00700 1 -- 172.21.15.131:0/1841681641 wait complete. 2024-04-26T20:09:31.577 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T20:09:31.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T20:09:31.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T20:09:31.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T20:09:31.612 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T20:09:31.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T20:09:31.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T20:09:31.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T20:09:31.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T20:09:31.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T20:09:31.750 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T20:09:31.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T20:09:31.815 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:31 smithi157 ceph-mon[32703]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:09:31.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:09:31.859 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:09:31.867 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.875 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:31 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1841681641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:31.881 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:31 smithi131 ceph-mon[27919]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:09:31.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:09:31.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:09:31.902 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:31.910 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:31 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1841681641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:31.920 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:31 smithi148 ceph-mon[32281]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:09:32.008 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:09:32.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:09:32.024 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:09:32.032 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:31 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1841681641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:09:32.856 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:32.901 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:32.956 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:32.986 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:33.330 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:33.673 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:33 smithi131 ceph-mon[27919]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:09:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:33 smithi157 ceph-mon[32703]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:09:33.762 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:33.770 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:33.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:33 smithi148 ceph-mon[32281]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:09:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:35.571 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:35.592 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:35.599 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:09:35.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:35 smithi157 ceph-mon[32703]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:09:35.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:35 smithi131 ceph-mon[27919]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:09:35.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:35 smithi148 ceph-mon[32281]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:09:37.657 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:37 smithi131 ceph-mon[27919]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:37 smithi157 ceph-mon[32703]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:37.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:37 smithi148 ceph-mon[32281]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:39.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:39 smithi157 ceph-mon[32703]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:09:39.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:39 smithi131 ceph-mon[27919]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:09:39.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:39 smithi148 ceph-mon[32281]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:09:39.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 -- 172.21.15.131:0/344289585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 msgr2=0x7f5aa8109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:40.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 --2- 172.21.15.131:0/344289585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 0x7f5aa8109880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009990 tx=0x7f5a9c009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 -- 172.21.15.131:0/344289585 shutdown_connections 2024-04-26T20:09:40.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 --2- 172.21.15.131:0/344289585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa806bad0 0x7f5aa81102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 --2- 172.21.15.131:0/344289585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 0x7f5aa8109880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 --2- 172.21.15.131:0/344289585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa806b130 0x7f5aa806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 -- 172.21.15.131:0/344289585 >> 172.21.15.131:0/344289585 conn(0x7f5aa80792d0 msgr2=0x7f5aa80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:40.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 -- 172.21.15.131:0/344289585 shutdown_connections 2024-04-26T20:09:40.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.477+0000 7f5ab0400700 1 -- 172.21.15.131:0/344289585 wait complete. 2024-04-26T20:09:40.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.478+0000 7f5ab0400700 1 Processor -- start 2024-04-26T20:09:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.478+0000 7f5ab0400700 1 -- start start 2024-04-26T20:09:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.478+0000 7f5ab0400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa806b130 0x7f5aa819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:40.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5ab0400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa806bad0 0x7f5aa819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:40.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5ab0400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 0x7f5aa81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:40.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5ab0400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8113600 con 0x7f5aa806b130 2024-04-26T20:09:40.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5ab0400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8113480 con 0x7f5aa81094a0 2024-04-26T20:09:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5ab0400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8113780 con 0x7f5aa806bad0 2024-04-26T20:09:40.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 0x7f5aa81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aada00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa806b130 0x7f5aa819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 0x7f5aa81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:54100/0 (socket says 172.21.15.131:54100) 2024-04-26T20:09:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 -- 172.21.15.131:0/3153020215 learned_addr learned my addr 172.21.15.131:0/3153020215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:40.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aad000700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa806bad0 0x7f5aa819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:40.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 -- 172.21.15.131:0/3153020215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa806bad0 msgr2=0x7f5aa819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:40.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa806bad0 0x7f5aa819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 -- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa806b130 msgr2=0x7f5aa819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa806b130 0x7f5aa819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aae400700 1 -- 172.21.15.131:0/3153020215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa81a0d60 con 0x7f5aa81094a0 2024-04-26T20:09:40.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.479+0000 7f5aada00700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa806b130 0x7f5aa819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:40.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.480+0000 7f5aae400700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 0x7f5aa81a0620 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000ef30 tx=0x7f5aa000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.480+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00092e0 con 0x7f5aa81094a0 2024-04-26T20:09:40.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.480+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa0004bb0 con 0x7f5aa81094a0 2024-04-26T20:09:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.480+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa81a0ff0 con 0x7f5aa81094a0 2024-04-26T20:09:40.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.480+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0004020 con 0x7f5aa81094a0 2024-04-26T20:09:40.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.480+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa81a88d0 con 0x7f5aa81094a0 2024-04-26T20:09:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.481+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5aa8100bf0 con 0x7f5aa81094a0 2024-04-26T20:09:40.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.483+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5aa001b030 con 0x7f5aa81094a0 2024-04-26T20:09:40.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.484+0000 7f5aa6000700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a900a3d50 0x7f5a900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:40.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.485+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5aa00d0310 con 0x7f5aa81094a0 2024-04-26T20:09:40.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.485+0000 7f5aada00700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a900a3d50 0x7f5a900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.485+0000 7f5aa6000700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] conn(0x7f5a900af130 0x7f5a900b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.485+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 --> [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a900b1ca0 con 0x7f5a900af130 2024-04-26T20:09:40.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.485+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f5aa00d7050 con 0x7f5aa81094a0 2024-04-26T20:09:40.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.485+0000 7f5aad000700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] conn(0x7f5a900af130 0x7f5a900b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.486+0000 7f5aada00700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a900a3d50 0x7f5a900a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006010 tx=0x7f5a9c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:40.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.486+0000 7f5aad000700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] conn(0x7f5a900af130 0x7f5a900b15d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:40.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.487+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== osd.3 v2:172.21.15.131:6810/1043436815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a900b1ca0 con 0x7f5a900af130 2024-04-26T20:09:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.505+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 --> [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5aa8104b50 con 0x7f5a900af130 2024-04-26T20:09:40.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.505+0000 7f5aa6000700 1 -- 172.21.15.131:0/3153020215 <== osd.3 v2:172.21.15.131:6810/1043436815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5aa8104b50 con 0x7f5a900af130 2024-04-26T20:09:40.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] conn(0x7f5a900af130 msgr2=0x7f5a900b15d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:40.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] conn(0x7f5a900af130 0x7f5a900b15d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a900a3d50 msgr2=0x7f5a900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:40.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a900a3d50 0x7f5a900a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006010 tx=0x7f5a9c00b260 comp rx=0 tx=0).stop 2024-04-26T20:09:40.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 msgr2=0x7f5aa81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:40.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 --2- 172.21.15.131:0/3153020215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aa81094a0 0x7f5aa81a0620 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000ef30 tx=0x7f5aa000c3a0 comp rx=0 tx=0).stop 2024-04-26T20:09:40.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5aae400700 1 -- 172.21.15.131:0/3153020215 reap_dead start 2024-04-26T20:09:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 shutdown_connections 2024-04-26T20:09:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 >> 172.21.15.131:0/3153020215 conn(0x7f5aa80792d0 msgr2=0x7f5aa8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 shutdown_connections 2024-04-26T20:09:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:38.506+0000 7f5ab0400700 1 -- 172.21.15.131:0/3153020215 wait complete. 2024-04-26T20:09:41.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:41 smithi157 ceph-mon[32703]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:41.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:41 smithi131 ceph-mon[27919]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:41.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:41 smithi148 ceph-mon[32281]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:43.621 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:43 smithi131 ceph-mon[27919]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:43.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:43 smithi157 ceph-mon[32703]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:43.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:43 smithi148 ceph-mon[32281]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:45.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:45 smithi157 ceph-mon[32703]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:45.798 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:45 smithi131 ceph-mon[27919]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:45.826 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:45 smithi148 ceph-mon[32281]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:47.089 INFO:teuthology.orchestra.run.smithi131.stdout:107374182488 2024-04-26T20:09:47.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T20:09:47.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:47 smithi157 ceph-mon[32703]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:47.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:47 smithi131 ceph-mon[27919]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:47.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:47 smithi148 ceph-mon[32281]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:09:49.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:49 smithi131 ceph-mon[27919]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:09:49.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:49 smithi157 ceph-mon[32703]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:09:49.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.745+0000 7f3925e00700 1 -- 172.21.15.131:0/315470505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39201034d0 msgr2=0x7f3920103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:49.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.745+0000 7f3925e00700 1 --2- 172.21.15.131:0/315470505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39201034d0 0x7f3920103950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f390c009990 tx=0x7f390c009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:49.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.746+0000 7f3925e00700 1 -- 172.21.15.131:0/315470505 shutdown_connections 2024-04-26T20:09:49.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.746+0000 7f3925e00700 1 --2- 172.21.15.131:0/315470505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3920103e90 0x7f3920110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:49.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.746+0000 7f3925e00700 1 --2- 172.21.15.131:0/315470505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39201083b0 0x7f3920108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:49.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.746+0000 7f3925e00700 1 --2- 172.21.15.131:0/315470505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39201034d0 0x7f3920103950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.746+0000 7f3925e00700 1 -- 172.21.15.131:0/315470505 >> 172.21.15.131:0/315470505 conn(0x7f392007a580 msgr2=0x7f392007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:49.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.746+0000 7f3925e00700 1 -- 172.21.15.131:0/315470505 shutdown_connections 2024-04-26T20:09:49.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.746+0000 7f3925e00700 1 -- 172.21.15.131:0/315470505 wait complete. 2024-04-26T20:09:49.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 Processor -- start 2024-04-26T20:09:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 -- start start 2024-04-26T20:09:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39201034d0 0x7f392019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:50.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920103e90 0x7f392019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:50.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39201083b0 0x7f3920199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:50.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3920113670 con 0x7f39201083b0 2024-04-26T20:09:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39201134f0 con 0x7f39201034d0 2024-04-26T20:09:50.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.747+0000 7f3925e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39201137f0 con 0x7f3920103e90 2024-04-26T20:09:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39201083b0 0x7f3920199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:50.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920103e90 0x7f392019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39201083b0 0x7f3920199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42554/0 (socket says 172.21.15.131:42554) 2024-04-26T20:09:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39201034d0 0x7f392019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:50.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391f400700 1 -- 172.21.15.131:0/1708383867 learned_addr learned my addr 172.21.15.131:0/1708383867 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:50.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391ea00700 1 -- 172.21.15.131:0/1708383867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920103e90 msgr2=0x7f392019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:50.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391ea00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920103e90 0x7f392019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:50.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391ea00700 1 -- 172.21.15.131:0/1708383867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39201083b0 msgr2=0x7f3920199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:50.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391ea00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39201083b0 0x7f3920199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:50.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391ea00700 1 -- 172.21.15.131:0/1708383867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392019a0b0 con 0x7f39201034d0 2024-04-26T20:09:50.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.748+0000 7f391ea00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39201034d0 0x7f392019f790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f391000e9f0 tx=0x7f391000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:50.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.749+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391000cb70 con 0x7f39201034d0 2024-04-26T20:09:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.749+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392019a3a0 con 0x7f39201034d0 2024-04-26T20:09:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.749+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f391000ee90 con 0x7f39201034d0 2024-04-26T20:09:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.749+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910003d40 con 0x7f39201034d0 2024-04-26T20:09:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.749+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39201a6660 con 0x7f39201034d0 2024-04-26T20:09:50.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.750+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3920063bb0 con 0x7f39201034d0 2024-04-26T20:09:50.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:49 smithi148 ceph-mon[32281]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:09:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.752+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3910003ee0 con 0x7f39201034d0 2024-04-26T20:09:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.753+0000 7f3924a00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39040a3cd0 0x7f39040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.754+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f39100d0a70 con 0x7f39201034d0 2024-04-26T20:09:50.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.754+0000 7f391e000700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39040a3cd0 0x7f39040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:50.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.754+0000 7f3924a00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] conn(0x7f39040af0b0 0x7f39040b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:50.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.754+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 --> [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f391000ce00 con 0x7f39040af0b0 2024-04-26T20:09:50.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.754+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f39100d52a0 con 0x7f39201034d0 2024-04-26T20:09:50.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.754+0000 7f391f400700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] conn(0x7f39040af0b0 0x7f39040b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:50.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.755+0000 7f391e000700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39040a3cd0 0x7f39040a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b790 tx=0x7f390c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.755+0000 7f391f400700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] conn(0x7f39040af0b0 0x7f39040b1550 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:50.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.756+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== osd.2 v2:172.21.15.157:6800/527328621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f391000ce00 con 0x7f39040af0b0 2024-04-26T20:09:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.774+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 --> [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3920063fe0 con 0x7f39040af0b0 2024-04-26T20:09:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.774+0000 7f3924a00700 1 -- 172.21.15.131:0/1708383867 <== osd.2 v2:172.21.15.157:6800/527328621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3920063fe0 con 0x7f39040af0b0 2024-04-26T20:09:50.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 >> [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] conn(0x7f39040af0b0 msgr2=0x7f39040b1550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:50.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] conn(0x7f39040af0b0 0x7f39040b1550 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:50.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39040a3cd0 msgr2=0x7f39040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f39040a3cd0 0x7f39040a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b790 tx=0x7f390c00b260 comp rx=0 tx=0).stop 2024-04-26T20:09:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39201034d0 msgr2=0x7f392019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:51.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 --2- 172.21.15.131:0/1708383867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39201034d0 0x7f392019f790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f391000e9f0 tx=0x7f391000ed00 comp rx=0 tx=0).stop 2024-04-26T20:09:51.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f391f400700 1 -- 172.21.15.131:0/1708383867 reap_dead start 2024-04-26T20:09:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 shutdown_connections 2024-04-26T20:09:51.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 >> 172.21.15.131:0/1708383867 conn(0x7f392007a580 msgr2=0x7f3920101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:51.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 shutdown_connections 2024-04-26T20:09:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.775+0000 7f3925e00700 1 -- 172.21.15.131:0/1708383867 wait complete. 2024-04-26T20:09:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.835+0000 7f2fee000700 1 -- 172.21.15.131:0/1055309431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe810fe20 msgr2=0x7f2fe8112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.835+0000 7f2fee000700 1 --2- 172.21.15.131:0/1055309431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe810fe20 0x7f2fe8112210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000b0a0 tx=0x7f2fe000b3b0 comp rx=0 tx=0).stop 2024-04-26T20:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.835+0000 7f2fee000700 1 -- 172.21.15.131:0/1055309431 shutdown_connections 2024-04-26T20:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.836+0000 7f2fee000700 1 --2- 172.21.15.131:0/1055309431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe810fe20 0x7f2fe8112210 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:51.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.836+0000 7f2fee000700 1 --2- 172.21.15.131:0/1055309431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe8075650 0x7f2fe8075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.836+0000 7f2fee000700 1 --2- 172.21.15.131:0/1055309431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fe8075f70 0x7f2fe810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:51.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.836+0000 7f2fee000700 1 -- 172.21.15.131:0/1055309431 >> 172.21.15.131:0/1055309431 conn(0x7f2fe807a580 msgr2=0x7f2fe807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.836+0000 7f2fee000700 1 -- 172.21.15.131:0/1055309431 shutdown_connections 2024-04-26T20:09:51.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.836+0000 7f2fee000700 1 -- 172.21.15.131:0/1055309431 wait complete. 2024-04-26T20:09:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 Processor -- start 2024-04-26T20:09:51.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 -- start start 2024-04-26T20:09:51.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fe8075650 0x7f2fe81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:51.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe8075f70 0x7f2fe81a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:51.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe810fe20 0x7f2fe819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81157c0 con 0x7f2fe8075f70 2024-04-26T20:09:51.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8115640 con 0x7f2fe8075650 2024-04-26T20:09:51.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.837+0000 7f2fee000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8115940 con 0x7f2fe810fe20 2024-04-26T20:09:51.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fe8075650 0x7f2fe81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:51.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fe8075650 0x7f2fe81a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:34412/0 (socket says 172.21.15.131:34412) 2024-04-26T20:09:51.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 -- 172.21.15.131:0/1725165965 learned_addr learned my addr 172.21.15.131:0/1725165965 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:51.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe6a00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe8075f70 0x7f2fe81a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:51.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7e00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe810fe20 0x7f2fe819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:51.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 -- 172.21.15.131:0/1725165965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe810fe20 msgr2=0x7f2fe819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:51.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe810fe20 0x7f2fe819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:51.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 -- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe8075f70 msgr2=0x7f2fe81a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:51.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe8075f70 0x7f2fe81a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:51.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe7400700 1 -- 172.21.15.131:0/1725165965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe819c1a0 con 0x7f2fe8075650 2024-04-26T20:09:51.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.838+0000 7f2fe6a00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fe8075f70 0x7f2fe81a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:09:51.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.839+0000 7f2fe7400700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fe8075650 0x7f2fe81a1880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800e9a0 tx=0x7f2fd800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:51.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.839+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800cbd0 con 0x7f2fe8075650 2024-04-26T20:09:51.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.839+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd800cd70 con 0x7f2fe8075650 2024-04-26T20:09:51.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.839+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe819c490 con 0x7f2fe8075650 2024-04-26T20:09:51.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:51 smithi157 ceph-mon[32703]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:09:51.989 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:51 smithi148 ceph-mon[32281]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:09:51.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.839+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8018680 con 0x7f2fe8075650 2024-04-26T20:09:52.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.839+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe81a8750 con 0x7f2fe8075650 2024-04-26T20:09:52.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.840+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fe8063bb0 con 0x7f2fe8075650 2024-04-26T20:09:52.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.842+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2fd8010970 con 0x7f2fe8075650 2024-04-26T20:09:52.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.843+0000 7f2fdfe00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2fcc0a3d50 0x7f2fcc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.843+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2fd80d1460 con 0x7f2fe8075650 2024-04-26T20:09:52.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.844+0000 7f2fe6a00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2fcc0a3d50 0x7f2fcc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:52.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.844+0000 7f2fdfe00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] conn(0x7f2fcc0af130 0x7f2fcc0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:52.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.844+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 --> [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fcc0b1ca0 con 0x7f2fcc0af130 2024-04-26T20:09:52.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.844+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f2fd80d18b0 con 0x7f2fe8075650 2024-04-26T20:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.844+0000 7f2fe7e00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] conn(0x7f2fcc0af130 0x7f2fcc0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:52.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.844+0000 7f2fe6a00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2fcc0a3d50 0x7f2fcc0a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8111ae0 tx=0x7f2fd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:52.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.845+0000 7f2fe7e00700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] conn(0x7f2fcc0af130 0x7f2fcc0b15d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.846+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== osd.6 v2:172.21.15.131:6818/1518784346 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fcc0b1ca0 con 0x7f2fcc0af130 2024-04-26T20:09:52.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.864+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 --> [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fe8063fb0 con 0x7f2fcc0af130 2024-04-26T20:09:52.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.864+0000 7f2fdfe00700 1 -- 172.21.15.131:0/1725165965 <== osd.6 v2:172.21.15.131:6818/1518784346 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2fe8063fb0 con 0x7f2fcc0af130 2024-04-26T20:09:52.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] conn(0x7f2fcc0af130 msgr2=0x7f2fcc0b15d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:52.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] conn(0x7f2fcc0af130 0x7f2fcc0b15d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:52.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2fcc0a3d50 msgr2=0x7f2fcc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:52.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2fcc0a3d50 0x7f2fcc0a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8111ae0 tx=0x7f2fd4009260 comp rx=0 tx=0).stop 2024-04-26T20:09:52.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fe8075650 msgr2=0x7f2fe81a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:52.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 --2- 172.21.15.131:0/1725165965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fe8075650 0x7f2fe81a1880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800e9a0 tx=0x7f2fd800ed60 comp rx=0 tx=0).stop 2024-04-26T20:09:52.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fe7e00700 1 -- 172.21.15.131:0/1725165965 reap_dead start 2024-04-26T20:09:52.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 shutdown_connections 2024-04-26T20:09:52.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 >> 172.21.15.131:0/1725165965 conn(0x7f2fe807a580 msgr2=0x7f2fe81005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:52.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 shutdown_connections 2024-04-26T20:09:52.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:49.865+0000 7f2fee000700 1 -- 172.21.15.131:0/1725165965 wait complete. 2024-04-26T20:09:52.538 INFO:teuthology.orchestra.run.smithi131.stdout:197568495689 2024-04-26T20:09:52.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T20:09:52.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:51 smithi131 ceph-mon[27919]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:09:52.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 -- 172.21.15.131:0/2084794016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103e30 msgr2=0x7f09b8110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:52.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 --2- 172.21.15.131:0/2084794016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103e30 0x7f09b8110290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f09b000b0a0 tx=0x7f09b000b3b0 comp rx=0 tx=0).stop 2024-04-26T20:09:52.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 -- 172.21.15.131:0/2084794016 shutdown_connections 2024-04-26T20:09:52.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 --2- 172.21.15.131:0/2084794016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103e30 0x7f09b8110290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:52.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 --2- 172.21.15.131:0/2084794016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b81094c0 0x7f09b81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:52.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 --2- 172.21.15.131:0/2084794016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b8103470 0x7f09b81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:52.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 -- 172.21.15.131:0/2084794016 >> 172.21.15.131:0/2084794016 conn(0x7f09b807a580 msgr2=0x7f09b807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:52.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.658+0000 7f09bf000700 1 -- 172.21.15.131:0/2084794016 shutdown_connections 2024-04-26T20:09:52.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.659+0000 7f09bf000700 1 -- 172.21.15.131:0/2084794016 wait complete. 2024-04-26T20:09:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.659+0000 7f09bf000700 1 Processor -- start 2024-04-26T20:09:52.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.660+0000 7f09bf000700 1 -- start start 2024-04-26T20:09:52.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.660+0000 7f09bf000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103470 0x7f09b81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:52.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.660+0000 7f09bf000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b8103e30 0x7f09b81a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:52.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.660+0000 7f09bf000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b81094c0 0x7f09b819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:52.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.660+0000 7f09bf000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8113630 con 0x7f09b81094c0 2024-04-26T20:09:52.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.660+0000 7f09bf000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81134b0 con 0x7f09b8103e30 2024-04-26T20:09:52.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.660+0000 7f09bf000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81137b0 con 0x7f09b8103470 2024-04-26T20:09:52.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103470 0x7f09b81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:52.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09bd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b81094c0 0x7f09b819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:52.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103470 0x7f09b81a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50562/0 (socket says 172.21.15.131:50562) 2024-04-26T20:09:52.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 -- 172.21.15.131:0/3417683184 learned_addr learned my addr 172.21.15.131:0/3417683184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:52.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7400700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b8103e30 0x7f09b81a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:52.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 -- 172.21.15.131:0/3417683184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b8103e30 msgr2=0x7f09b81a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:52.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b8103e30 0x7f09b81a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:52.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 -- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b81094c0 msgr2=0x7f09b819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:53.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b81094c0 0x7f09b819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:53.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 -- 172.21.15.131:0/3417683184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b819c240 con 0x7f09b8103470 2024-04-26T20:09:53.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09bd000700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b81094c0 0x7f09b819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:53.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7400700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b8103e30 0x7f09b81a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.661+0000 7f09b7e00700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103470 0x7f09b81a1920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e9c0 tx=0x7f09a800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.662+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a800cbd0 con 0x7f09b8103470 2024-04-26T20:09:53.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.662+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a800ee60 con 0x7f09b8103470 2024-04-26T20:09:53.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.662+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80055c0 con 0x7f09b8103470 2024-04-26T20:09:53.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.662+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b819c530 con 0x7f09b8103470 2024-04-26T20:09:53.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.662+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b81a87f0 con 0x7f09b8103470 2024-04-26T20:09:53.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.663+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09b8102940 con 0x7f09b8103470 2024-04-26T20:09:53.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.665+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f09a8004750 con 0x7f09b8103470 2024-04-26T20:09:53.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.666+0000 7f09b4c00700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f099c0a3d50 0x7f099c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:53.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.666+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f09a80d0f70 con 0x7f09b8103470 2024-04-26T20:09:53.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.666+0000 7f09b7400700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f099c0a3d50 0x7f099c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:53.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.667+0000 7f09b4c00700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] conn(0x7f099c0af130 0x7f099c0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:53.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.667+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 --> [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f099c0b1ca0 con 0x7f099c0af130 2024-04-26T20:09:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.667+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f09a80d1320 con 0x7f09b8103470 2024-04-26T20:09:53.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.667+0000 7f09bd000700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] conn(0x7f099c0af130 0x7f099c0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.667+0000 7f09b7400700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f099c0a3d50 0x7f099c0a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09b81106d0 tx=0x7f09a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:53.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.668+0000 7f09bd000700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] conn(0x7f099c0af130 0x7f099c0b15d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:53.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.669+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== osd.11 v2:172.21.15.157:6824/1583012638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f099c0b1ca0 con 0x7f099c0af130 2024-04-26T20:09:53.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.687+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 --> [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09b8104b60 con 0x7f099c0af130 2024-04-26T20:09:53.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09b4c00700 1 -- 172.21.15.131:0/3417683184 <== osd.11 v2:172.21.15.157:6824/1583012638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09b8104b60 con 0x7f099c0af130 2024-04-26T20:09:53.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] conn(0x7f099c0af130 msgr2=0x7f099c0b15d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:53.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bf000700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] conn(0x7f099c0af130 0x7f099c0b15d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:53.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f099c0a3d50 msgr2=0x7f099c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:53.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bf000700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f099c0a3d50 0x7f099c0a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09b81106d0 tx=0x7f09a4009260 comp rx=0 tx=0).stop 2024-04-26T20:09:53.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103470 msgr2=0x7f09b81a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bf000700 1 --2- 172.21.15.131:0/3417683184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8103470 0x7f09b81a1920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e9c0 tx=0x7f09a800ecd0 comp rx=0 tx=0).stop 2024-04-26T20:09:53.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bd000700 1 -- 172.21.15.131:0/3417683184 reap_dead start 2024-04-26T20:09:53.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.688+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 shutdown_connections 2024-04-26T20:09:53.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.689+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 >> 172.21.15.131:0/3417683184 conn(0x7f09b807a580 msgr2=0x7f09b8101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:53.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.689+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 shutdown_connections 2024-04-26T20:09:53.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:48.689+0000 7f09bf000700 1 -- 172.21.15.131:0/3417683184 wait complete. 2024-04-26T20:09:53.741 INFO:teuthology.orchestra.run.smithi131.stdout:77309411425 2024-04-26T20:09:53.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T20:09:53.756 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:53 smithi157 ceph-mon[32703]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T20:09:53.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 -- 172.21.15.131:0/3361554242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075650 msgr2=0x7fda58075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:53.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 --2- 172.21.15.131:0/3361554242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075650 0x7fda58075a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009990 tx=0x7fda4c009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:53.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 -- 172.21.15.131:0/3361554242 shutdown_connections 2024-04-26T20:09:53.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 --2- 172.21.15.131:0/3361554242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda581145f0 0x7fda581169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:53.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 --2- 172.21.15.131:0/3361554242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075650 0x7fda58075a30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:53.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 --2- 172.21.15.131:0/3361554242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda58075f70 0x7fda581140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:53.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 -- 172.21.15.131:0/3361554242 >> 172.21.15.131:0/3361554242 conn(0x7fda5807a580 msgr2=0x7fda5807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:53.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 -- 172.21.15.131:0/3361554242 shutdown_connections 2024-04-26T20:09:53.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.426+0000 7fda60800700 1 -- 172.21.15.131:0/3361554242 wait complete. 2024-04-26T20:09:53.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.427+0000 7fda60800700 1 Processor -- start 2024-04-26T20:09:53.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda60800700 1 -- start start 2024-04-26T20:09:53.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda60800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda58075650 0x7fda581a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:53.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda60800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075f70 0x7fda581a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda60800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda581145f0 0x7fda581a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda60800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda58119c50 con 0x7fda58075f70 2024-04-26T20:09:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda60800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda58119ad0 con 0x7fda581145f0 2024-04-26T20:09:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda60800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda58119dd0 con 0x7fda58075650 2024-04-26T20:09:54.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda5d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075f70 0x7fda581a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:54.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda5d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075f70 0x7fda581a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42584/0 (socket says 172.21.15.131:42584) 2024-04-26T20:09:54.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda5d400700 1 -- 172.21.15.131:0/1173465555 learned_addr learned my addr 172.21.15.131:0/1173465555 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:54.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda5de00700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda58075650 0x7fda581a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:54.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.428+0000 7fda5e800700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda581145f0 0x7fda581a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:54.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda5d400700 1 -- 172.21.15.131:0/1173465555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda58075650 msgr2=0x7fda581a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:54.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda5d400700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda58075650 0x7fda581a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda5d400700 1 -- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda581145f0 msgr2=0x7fda581a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda5d400700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda581145f0 0x7fda581a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:54.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda5d400700 1 -- 172.21.15.131:0/1173465555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda581a5150 con 0x7fda58075f70 2024-04-26T20:09:54.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda5d400700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075f70 0x7fda581a0680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fda4800e9a0 tx=0x7fda4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:54.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4800ca10 con 0x7fda58075f70 2024-04-26T20:09:54.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda4800cbb0 con 0x7fda58075f70 2024-04-26T20:09:54.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda581a5440 con 0x7fda58075f70 2024-04-26T20:09:54.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48018680 con 0x7fda58075f70 2024-04-26T20:09:54.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.429+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda581accc0 con 0x7fda58075f70 2024-04-26T20:09:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.430+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda58113ef0 con 0x7fda58075f70 2024-04-26T20:09:54.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.434+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fda48010430 con 0x7fda58075f70 2024-04-26T20:09:54.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.436+0000 7fda52a00700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda400a3cd0 0x7fda400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:54.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.436+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fda480d1100 con 0x7fda58075f70 2024-04-26T20:09:54.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.436+0000 7fda5de00700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda400a3cd0 0x7fda400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:54.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.437+0000 7fda52a00700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] conn(0x7fda400af0b0 0x7fda400b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:54.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.437+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 --> [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda48017e70 con 0x7fda400af0b0 2024-04-26T20:09:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.437+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fda480d14b0 con 0x7fda58075f70 2024-04-26T20:09:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.437+0000 7fda5e800700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] conn(0x7fda400af0b0 0x7fda400b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:54.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.437+0000 7fda5de00700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda400a3cd0 0x7fda400a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00b2e0 tx=0x7fda4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:54.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.438+0000 7fda5e800700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] conn(0x7fda400af0b0 0x7fda400b1550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:54.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.438+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== osd.1 v2:172.21.15.148:6800/2432608282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda48017e70 con 0x7fda400af0b0 2024-04-26T20:09:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.457+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 --> [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda58076d40 con 0x7fda400af0b0 2024-04-26T20:09:54.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.458+0000 7fda52a00700 1 -- 172.21.15.131:0/1173465555 <== osd.1 v2:172.21.15.148:6800/2432608282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fda58076d40 con 0x7fda400af0b0 2024-04-26T20:09:54.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.458+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] conn(0x7fda400af0b0 msgr2=0x7fda400b1550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:54.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.458+0000 7fda60800700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] conn(0x7fda400af0b0 0x7fda400b1550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:54.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda400a3cd0 msgr2=0x7fda400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:54.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda400a3cd0 0x7fda400a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00b2e0 tx=0x7fda4c005fd0 comp rx=0 tx=0).stop 2024-04-26T20:09:54.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075f70 msgr2=0x7fda581a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:54.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 --2- 172.21.15.131:0/1173465555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda58075f70 0x7fda581a0680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fda4800e9a0 tx=0x7fda4800ecb0 comp rx=0 tx=0).stop 2024-04-26T20:09:54.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda5e800700 1 -- 172.21.15.131:0/1173465555 reap_dead start 2024-04-26T20:09:54.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 shutdown_connections 2024-04-26T20:09:54.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 >> 172.21.15.131:0/1173465555 conn(0x7fda5807a580 msgr2=0x7fda58113490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:54.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 shutdown_connections 2024-04-26T20:09:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:53.459+0000 7fda60800700 1 -- 172.21.15.131:0/1173465555 wait complete. 2024-04-26T20:09:54.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:53 smithi131 ceph-mon[27919]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T20:09:54.840 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:53 smithi148 ceph-mon[32281]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T20:09:54.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.639+0000 7fc2a4a00700 1 -- 172.21.15.131:0/381540456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 msgr2=0x7fc2a0104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.639+0000 7fc2a4a00700 1 --2- 172.21.15.131:0/381540456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 0x7fc2a0104940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc290009990 tx=0x7fc290009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:54.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.640+0000 7fc2a4a00700 1 -- 172.21.15.131:0/381540456 shutdown_connections 2024-04-26T20:09:55.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.640+0000 7fc2a4a00700 1 --2- 172.21.15.131:0/381540456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a010d990 0x7fc2a010fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:55.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.640+0000 7fc2a4a00700 1 --2- 172.21.15.131:0/381540456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 0x7fc2a0104940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:55.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.640+0000 7fc2a4a00700 1 --2- 172.21.15.131:0/381540456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a0104e80 0x7fc2a010d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:55.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.640+0000 7fc2a4a00700 1 -- 172.21.15.131:0/381540456 >> 172.21.15.131:0/381540456 conn(0x7fc2a00fddb0 msgr2=0x7fc2a01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:55.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.641+0000 7fc2a4a00700 1 -- 172.21.15.131:0/381540456 shutdown_connections 2024-04-26T20:09:55.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.641+0000 7fc2a4a00700 1 -- 172.21.15.131:0/381540456 wait complete. 2024-04-26T20:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.642+0000 7fc2a4a00700 1 Processor -- start 2024-04-26T20:09:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.643+0000 7fc2a4a00700 1 -- start start 2024-04-26T20:09:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.643+0000 7fc2a4a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 0x7fc2a019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.643+0000 7fc2a4a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a0104e80 0x7fc2a019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:55.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.643+0000 7fc2a4a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a010d990 0x7fc2a01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc2a4a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0102b40 con 0x7fc2a0104560 2024-04-26T20:09:55.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc2a4a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01029c0 con 0x7fc2a0104e80 2024-04-26T20:09:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc2a4a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0102cc0 con 0x7fc2a010d990 2024-04-26T20:09:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc297e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a0104e80 0x7fc2a019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:55.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc297e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a0104e80 0x7fc2a019c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:34438/0 (socket says 172.21.15.131:34438) 2024-04-26T20:09:55.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc297e00700 1 -- 172.21.15.131:0/2515416474 learned_addr learned my addr 172.21.15.131:0/2515416474 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc29e800700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 0x7fc2a019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:55.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc29f200700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a010d990 0x7fc2a01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:55.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc297e00700 1 -- 172.21.15.131:0/2515416474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a010d990 msgr2=0x7fc2a01a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:55.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc297e00700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a010d990 0x7fc2a01a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:55.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.644+0000 7fc297e00700 1 -- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 msgr2=0x7fc2a019be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.645+0000 7fc297e00700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 0x7fc2a019be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:55.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.645+0000 7fc297e00700 1 -- 172.21.15.131:0/2515416474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a01a0cf0 con 0x7fc2a0104e80 2024-04-26T20:09:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.645+0000 7fc29f200700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a010d990 0x7fc2a01a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:55.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.645+0000 7fc29e800700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2a0104560 0x7fc2a019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:55.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.645+0000 7fc297e00700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a0104e80 0x7fc2a019c3b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc28800e980 tx=0x7fc28800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.646+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288004220 con 0x7fc2a0104e80 2024-04-26T20:09:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.646+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2880043c0 con 0x7fc2a0104e80 2024-04-26T20:09:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.646+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a01a0fe0 con 0x7fc2a0104e80 2024-04-26T20:09:55.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.646+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a0112b80 con 0x7fc2a0104e80 2024-04-26T20:09:55.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.647+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288003d60 con 0x7fc2a0104e80 2024-04-26T20:09:55.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.647+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc27c001030 con 0x7fc2a0104e80 2024-04-26T20:09:55.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.651+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc28800c810 con 0x7fc2a0104e80 2024-04-26T20:09:55.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.652+0000 7fc297400700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2840a3d50 0x7fc2840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:55.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.652+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc2880d0c60 con 0x7fc2a0104e80 2024-04-26T20:09:55.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.652+0000 7fc29e800700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2840a3d50 0x7fc2840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.652+0000 7fc297400700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] conn(0x7fc2840af130 0x7fc2840b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:56.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.652+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 --> [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc28800e1a0 con 0x7fc2840af130 2024-04-26T20:09:56.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.652+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fc2880d10b0 con 0x7fc2a0104e80 2024-04-26T20:09:56.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.653+0000 7fc29f200700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] conn(0x7fc2840af130 0x7fc2840b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:56.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.653+0000 7fc29e800700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2840a3d50 0x7fc2840a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc290006010 tx=0x7fc290022040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:56.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.653+0000 7fc29f200700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] conn(0x7fc2840af130 0x7fc2840b15d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:56.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.654+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== osd.0 v2:172.21.15.131:6802/1131821741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc28800e1a0 con 0x7fc2840af130 2024-04-26T20:09:56.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.674+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 --> [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc27c002e60 con 0x7fc2840af130 2024-04-26T20:09:56.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc297400700 1 -- 172.21.15.131:0/2515416474 <== osd.0 v2:172.21.15.131:6802/1131821741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc27c002e60 con 0x7fc2840af130 2024-04-26T20:09:56.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] conn(0x7fc2840af130 msgr2=0x7fc2840b15d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] conn(0x7fc2840af130 0x7fc2840b15d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:56.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2840a3d50 msgr2=0x7fc2840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:56.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2840a3d50 0x7fc2840a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc290006010 tx=0x7fc290022040 comp rx=0 tx=0).stop 2024-04-26T20:09:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a0104e80 msgr2=0x7fc2a019c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:56.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 --2- 172.21.15.131:0/2515416474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a0104e80 0x7fc2a019c3b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc28800e980 tx=0x7fc28800ec90 comp rx=0 tx=0).stop 2024-04-26T20:09:56.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc29f200700 1 -- 172.21.15.131:0/2515416474 reap_dead start 2024-04-26T20:09:56.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 shutdown_connections 2024-04-26T20:09:56.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 >> 172.21.15.131:0/2515416474 conn(0x7fc2a00fddb0 msgr2=0x7fc2a01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:56.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.675+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 shutdown_connections 2024-04-26T20:09:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:51.676+0000 7fc2a4a00700 1 -- 172.21.15.131:0/2515416474 wait complete. 2024-04-26T20:09:56.530 INFO:teuthology.orchestra.run.smithi131.stdout:343597383724 2024-04-26T20:09:56.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T20:09:56.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.049+0000 7fa758800700 1 -- 172.21.15.131:0/2218172522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa750109490 msgr2=0x7fa750109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:56.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.049+0000 7fa758800700 1 --2- 172.21.15.131:0/2218172522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa750109490 0x7fa750109870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa7440098f0 tx=0x7fa744009c00 comp rx=0 tx=0).stop 2024-04-26T20:09:56.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.050+0000 7fa758800700 1 -- 172.21.15.131:0/2218172522 shutdown_connections 2024-04-26T20:09:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.050+0000 7fa758800700 1 --2- 172.21.15.131:0/2218172522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa75006c360 0x7fa7501102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.050+0000 7fa758800700 1 --2- 172.21.15.131:0/2218172522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa750109490 0x7fa750109870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.050+0000 7fa758800700 1 --2- 172.21.15.131:0/2218172522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75006b9c0 0x7fa75006be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.050+0000 7fa758800700 1 -- 172.21.15.131:0/2218172522 >> 172.21.15.131:0/2218172522 conn(0x7fa750079290 msgr2=0x7fa7500796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.050+0000 7fa758800700 1 -- 172.21.15.131:0/2218172522 shutdown_connections 2024-04-26T20:09:56.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.051+0000 7fa758800700 1 -- 172.21.15.131:0/2218172522 wait complete. 2024-04-26T20:09:56.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 Processor -- start 2024-04-26T20:09:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 -- start start 2024-04-26T20:09:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75006b9c0 0x7fa75019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:56.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75006c360 0x7fa75019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750109490 0x7fa750199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:56.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501135a0 con 0x7fa75006b9c0 2024-04-26T20:09:56.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa750113420 con 0x7fa75006c360 2024-04-26T20:09:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.052+0000 7fa758800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa750113720 con 0x7fa750109490 2024-04-26T20:09:56.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75006b9c0 0x7fa75019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:56.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75006b9c0 0x7fa75019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40186/0 (socket says 172.21.15.131:40186) 2024-04-26T20:09:56.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 -- 172.21.15.131:0/1126489652 learned_addr learned my addr 172.21.15.131:0/1126489652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:56.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755400700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75006c360 0x7fa75019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:56.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa756800700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750109490 0x7fa750199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:56.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 -- 172.21.15.131:0/1126489652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750109490 msgr2=0x7fa750199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:56.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750109490 0x7fa750199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:56.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 -- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75006c360 msgr2=0x7fa75019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:56.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75006c360 0x7fa75019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 -- 172.21.15.131:0/1126489652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75019a0c0 con 0x7fa75006b9c0 2024-04-26T20:09:57.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa756800700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750109490 0x7fa750199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:09:57.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.053+0000 7fa755e00700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75006b9c0 0x7fa75019f7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa74400f990 tx=0x7fa74400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:57.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.054+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744005400 con 0x7fa75006b9c0 2024-04-26T20:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.054+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75019a350 con 0x7fa75006b9c0 2024-04-26T20:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.054+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7440055a0 con 0x7fa75006b9c0 2024-04-26T20:09:57.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.054+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744004dc0 con 0x7fa75006b9c0 2024-04-26T20:09:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.054+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7501a6670 con 0x7fa75006b9c0 2024-04-26T20:09:57.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.055+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa750063bb0 con 0x7fa75006b9c0 2024-04-26T20:09:57.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.058+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa74400b560 con 0x7fa75006b9c0 2024-04-26T20:09:57.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.060+0000 7fa74aa00700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa7380a3d50 0x7fa7380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:57.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.060+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa7440d2420 con 0x7fa75006b9c0 2024-04-26T20:09:57.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.060+0000 7fa755400700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa7380a3d50 0x7fa7380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:57.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.061+0000 7fa74aa00700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] conn(0x7fa7380af130 0x7fa7380b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.061+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 --> [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa744009080 con 0x7fa7380af130 2024-04-26T20:09:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.061+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fa7440d2870 con 0x7fa75006b9c0 2024-04-26T20:09:57.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.061+0000 7fa756800700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] conn(0x7fa7380af130 0x7fa7380b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:57.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.061+0000 7fa755400700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa7380a3d50 0x7fa7380a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7500798e0 tx=0x7fa740009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.062+0000 7fa756800700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] conn(0x7fa7380af130 0x7fa7380b15d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.063+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== osd.10 v2:172.21.15.148:6824/3795716153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa744009080 con 0x7fa7380af130 2024-04-26T20:09:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.094+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 --> [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa750063fb0 con 0x7fa7380af130 2024-04-26T20:09:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.095+0000 7fa74aa00700 1 -- 172.21.15.131:0/1126489652 <== osd.10 v2:172.21.15.148:6824/3795716153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa750063fb0 con 0x7fa7380af130 2024-04-26T20:09:57.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:55 smithi157 ceph-mon[32703]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:57.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:55 smithi148 ceph-mon[32281]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:57.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.095+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] conn(0x7fa7380af130 msgr2=0x7fa7380b15d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:57.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.095+0000 7fa758800700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] conn(0x7fa7380af130 0x7fa7380b15d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa7380a3d50 msgr2=0x7fa7380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:57.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa7380a3d50 0x7fa7380a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7500798e0 tx=0x7fa740009310 comp rx=0 tx=0).stop 2024-04-26T20:09:57.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75006b9c0 msgr2=0x7fa75019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:57.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 --2- 172.21.15.131:0/1126489652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75006b9c0 0x7fa75019f7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa74400f990 tx=0x7fa74400fa70 comp rx=0 tx=0).stop 2024-04-26T20:09:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa756800700 1 -- 172.21.15.131:0/1126489652 reap_dead start 2024-04-26T20:09:57.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 shutdown_connections 2024-04-26T20:09:57.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 >> 172.21.15.131:0/1126489652 conn(0x7fa750079290 msgr2=0x7fa750100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:57.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 shutdown_connections 2024-04-26T20:09:57.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.096+0000 7fa758800700 1 -- 172.21.15.131:0/1126489652 wait complete. 2024-04-26T20:09:57.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:55 smithi131 ceph-mon[27919]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:09:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 -- 172.21.15.131:0/980498030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 msgr2=0x7f09441102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 --2- 172.21.15.131:0/980498030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 0x7f09441102d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f093c00b0a0 tx=0x7f093c00b3b0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 -- 172.21.15.131:0/980498030 shutdown_connections 2024-04-26T20:09:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 --2- 172.21.15.131:0/980498030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 0x7f09441102d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 --2- 172.21.15.131:0/980498030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0944109490 0x7f0944109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 --2- 172.21.15.131:0/980498030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406b9c0 0x7f094406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 -- 172.21.15.131:0/980498030 >> 172.21.15.131:0/980498030 conn(0x7f0944079290 msgr2=0x7f09440796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:57.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 -- 172.21.15.131:0/980498030 shutdown_connections 2024-04-26T20:09:57.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.807+0000 7f094bc00700 1 -- 172.21.15.131:0/980498030 wait complete. 2024-04-26T20:09:57.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.808+0000 7f094bc00700 1 Processor -- start 2024-04-26T20:09:57.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.808+0000 7f094bc00700 1 -- start start 2024-04-26T20:09:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f094bc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406b9c0 0x7f09441a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f094bc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 0x7f09441a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:57.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f094bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0944109490 0x7f094419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f094bc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0944113590 con 0x7f094406c360 2024-04-26T20:09:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f094bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0944113410 con 0x7f0944109490 2024-04-26T20:09:57.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f094bc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0944113710 con 0x7f094406b9c0 2024-04-26T20:09:57.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406b9c0 0x7f09441a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:57.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0943e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 0x7f09441a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:57.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0944109490 0x7f094419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0943e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 0x7f09441a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40198/0 (socket says 172.21.15.131:40198) 2024-04-26T20:09:57.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0944109490 0x7f094419ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:60608/0 (socket says 172.21.15.131:60608) 2024-04-26T20:09:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0943e00700 1 -- 172.21.15.131:0/2392597508 learned_addr learned my addr 172.21.15.131:0/2392597508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949c00700 1 -- 172.21.15.131:0/2392597508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406b9c0 msgr2=0x7f09441a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949c00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406b9c0 0x7f09441a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949c00700 1 -- 172.21.15.131:0/2392597508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 msgr2=0x7f09441a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:57.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949c00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094406c360 0x7f09441a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:57.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.809+0000 7f0949c00700 1 -- 172.21.15.131:0/2392597508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094419c1f0 con 0x7f0944109490 2024-04-26T20:09:58.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.810+0000 7f0949c00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0944109490 0x7f094419ba90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f093c00b070 tx=0x7f093c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.810+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c021070 con 0x7f0944109490 2024-04-26T20:09:58.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.810+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f093c007e30 con 0x7f0944109490 2024-04-26T20:09:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.810+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c0045d0 con 0x7f0944109490 2024-04-26T20:09:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.810+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094419c480 con 0x7f0944109490 2024-04-26T20:09:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.810+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09441a87a0 con 0x7f0944109490 2024-04-26T20:09:58.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.811+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09441033d0 con 0x7f0944109490 2024-04-26T20:09:58.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.814+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f093c003c00 con 0x7f0944109490 2024-04-26T20:09:58.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.815+0000 7f0941600700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f092c0a3cd0 0x7f092c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.815+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f093c00f020 con 0x7f0944109490 2024-04-26T20:09:58.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.815+0000 7f0949200700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f092c0a3cd0 0x7f092c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:58.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.815+0000 7f0941600700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] conn(0x7f092c0af0b0 0x7f092c0b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:58.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.815+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 --> [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f092c0b1c20 con 0x7f092c0af0b0 2024-04-26T20:09:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.815+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f093c0d2220 con 0x7f0944109490 2024-04-26T20:09:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.816+0000 7f0943e00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] conn(0x7f092c0af0b0 0x7f092c0b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.816+0000 7f0949200700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f092c0a3cd0 0x7f092c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0938009680 tx=0x7f0938009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:58.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.816+0000 7f0943e00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] conn(0x7f092c0af0b0 0x7f092c0b1550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:58.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.817+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== osd.5 v2:172.21.15.157:6808/2019329968 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f092c0b1c20 con 0x7f092c0af0b0 2024-04-26T20:09:58.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.836+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 --> [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0944104b40 con 0x7f092c0af0b0 2024-04-26T20:09:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f0941600700 1 -- 172.21.15.131:0/2392597508 <== osd.5 v2:172.21.15.157:6808/2019329968 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0944104b40 con 0x7f092c0af0b0 2024-04-26T20:09:58.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 >> [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] conn(0x7f092c0af0b0 msgr2=0x7f092c0b1550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:58.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] conn(0x7f092c0af0b0 0x7f092c0b1550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f092c0a3cd0 msgr2=0x7f092c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f092c0a3cd0 0x7f092c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0938009680 tx=0x7f0938009260 comp rx=0 tx=0).stop 2024-04-26T20:09:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0944109490 msgr2=0x7f094419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:58.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 --2- 172.21.15.131:0/2392597508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0944109490 0x7f094419ba90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f093c00b070 tx=0x7f093c004d00 comp rx=0 tx=0).stop 2024-04-26T20:09:58.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f0949c00700 1 -- 172.21.15.131:0/2392597508 reap_dead start 2024-04-26T20:09:58.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 shutdown_connections 2024-04-26T20:09:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.837+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 >> 172.21.15.131:0/2392597508 conn(0x7f0944079290 msgr2=0x7f09441004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:58.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.838+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 shutdown_connections 2024-04-26T20:09:58.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.838+0000 7f094bc00700 1 -- 172.21.15.131:0/2392597508 wait complete. 2024-04-26T20:09:58.469 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:57 smithi148 ceph-mon[32281]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T20:09:58.477 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:57 smithi157 ceph-mon[32703]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T20:09:58.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 -- 172.21.15.131:0/523302648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f668010b660 msgr2=0x7f668010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 --2- 172.21.15.131:0/523302648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f668010b660 0x7f668010ba40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6674009990 tx=0x7f6674009ca0 comp rx=0 tx=0).stop 2024-04-26T20:09:58.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 -- 172.21.15.131:0/523302648 shutdown_connections 2024-04-26T20:09:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 --2- 172.21.15.131:0/523302648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6680106020 0x7f6680112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:58.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 --2- 172.21.15.131:0/523302648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f668010b660 0x7f668010ba40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:58.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 --2- 172.21.15.131:0/523302648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680105660 0x7f6680105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:58.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 -- 172.21.15.131:0/523302648 >> 172.21.15.131:0/523302648 conn(0x7f6680101150 msgr2=0x7f6680103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 -- 172.21.15.131:0/523302648 shutdown_connections 2024-04-26T20:09:58.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.402+0000 7f6687a00700 1 -- 172.21.15.131:0/523302648 wait complete. 2024-04-26T20:09:58.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.403+0000 7f6687a00700 1 Processor -- start 2024-04-26T20:09:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.403+0000 7f6687a00700 1 -- start start 2024-04-26T20:09:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6687a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680105660 0x7f668019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:58.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6687a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6680106020 0x7f668019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:58.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6687a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f668010b660 0x7f66801a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:58.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6687a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6680115860 con 0x7f668010b660 2024-04-26T20:09:58.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6687a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66801156e0 con 0x7f6680105660 2024-04-26T20:09:58.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6687a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66801159e0 con 0x7f6680106020 2024-04-26T20:09:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f667fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6680106020 0x7f668019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:58.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f667fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6680106020 0x7f668019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46080/0 (socket says 172.21.15.131:46080) 2024-04-26T20:09:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f667fe00700 1 -- 172.21.15.131:0/948321551 learned_addr learned my addr 172.21.15.131:0/948321551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6685000700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680105660 0x7f668019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:58.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.404+0000 7f6685a00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f668010b660 0x7f66801a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667fe00700 1 -- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680105660 msgr2=0x7f668019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:59.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667fe00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680105660 0x7f668019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:59.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667fe00700 1 -- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f668010b660 msgr2=0x7f66801a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:59.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667fe00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f668010b660 0x7f66801a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667fe00700 1 -- 172.21.15.131:0/948321551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66801a0d40 con 0x7f6680106020 2024-04-26T20:09:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f6685000700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680105660 0x7f668019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:09:59.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667fe00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6680106020 0x7f668019c270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f666c00e9c0 tx=0x7f666c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:59.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c00cbd0 con 0x7f6680106020 2024-04-26T20:09:59.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f666c00ee60 con 0x7f6680106020 2024-04-26T20:09:59.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.405+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66801a1030 con 0x7f6680106020 2024-04-26T20:09:59.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.406+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c0055c0 con 0x7f6680106020 2024-04-26T20:09:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.406+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801a88b0 con 0x7f6680106020 2024-04-26T20:09:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.406+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6680063bb0 con 0x7f6680106020 2024-04-26T20:09:59.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.409+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f666c004750 con 0x7f6680106020 2024-04-26T20:09:59.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.410+0000 7f667d600700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f66680a3d50 0x7f66680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:59.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.410+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f666c0d0fc0 con 0x7f6680106020 2024-04-26T20:09:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.411+0000 7f6685000700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f66680a3d50 0x7f66680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.411+0000 7f667d600700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] conn(0x7f66680af130 0x7f66680b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:59.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.411+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 --> [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66680b1ca0 con 0x7f66680af130 2024-04-26T20:09:59.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.411+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f666c0d1370 con 0x7f6680106020 2024-04-26T20:09:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.411+0000 7f6685a00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] conn(0x7f66680af130 0x7f66680b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:59.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.411+0000 7f6685000700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f66680a3d50 0x7f66680a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f667400b2e0 tx=0x7f667401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.412+0000 7f6685a00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] conn(0x7f66680af130 0x7f66680b15d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:09:59.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.413+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== osd.4 v2:172.21.15.148:6808/1242934013 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66680b1ca0 con 0x7f66680af130 2024-04-26T20:09:59.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.431+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 --> [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6680064030 con 0x7f66680af130 2024-04-26T20:09:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.431+0000 7f667d600700 1 -- 172.21.15.131:0/948321551 <== osd.4 v2:172.21.15.148:6808/1242934013 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6680064030 con 0x7f66680af130 2024-04-26T20:09:59.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] conn(0x7f66680af130 msgr2=0x7f66680b15d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:59.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] conn(0x7f66680af130 0x7f66680b15d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f66680a3d50 msgr2=0x7f66680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:59.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f66680a3d50 0x7f66680a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f667400b2e0 tx=0x7f667401a040 comp rx=0 tx=0).stop 2024-04-26T20:09:59.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6680106020 msgr2=0x7f668019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:59.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 --2- 172.21.15.131:0/948321551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6680106020 0x7f668019c270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f666c00e9c0 tx=0x7f666c00ecd0 comp rx=0 tx=0).stop 2024-04-26T20:09:59.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6685a00700 1 -- 172.21.15.131:0/948321551 reap_dead start 2024-04-26T20:09:59.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 shutdown_connections 2024-04-26T20:09:59.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 >> 172.21.15.131:0/948321551 conn(0x7f6680101150 msgr2=0x7f66801019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 shutdown_connections 2024-04-26T20:09:59.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:56.432+0000 7f6687a00700 1 -- 172.21.15.131:0/948321551 wait complete. 2024-04-26T20:09:59.554 INFO:teuthology.orchestra.run.smithi131.stdout:34359738475 2024-04-26T20:09:59.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T20:09:59.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:57 smithi131 ceph-mon[27919]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T20:09:59.575 INFO:teuthology.orchestra.run.smithi131.stdout:309237645362 2024-04-26T20:09:59.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T20:09:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 -- 172.21.15.131:0/3836239765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778103470 msgr2=0x7f97781038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:59.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 --2- 172.21.15.131:0/3836239765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778103470 0x7f97781038f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f97640098f0 tx=0x7f9764009c00 comp rx=0 tx=0).stop 2024-04-26T20:09:59.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 -- 172.21.15.131:0/3836239765 shutdown_connections 2024-04-26T20:09:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 --2- 172.21.15.131:0/3836239765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9778103e30 0x7f97781102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 --2- 172.21.15.131:0/3836239765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97781094f0 0x7f97781098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 --2- 172.21.15.131:0/3836239765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778103470 0x7f97781038f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:09:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 -- 172.21.15.131:0/3836239765 >> 172.21.15.131:0/3836239765 conn(0x7f977807a5d0 msgr2=0x7f977807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:09:59.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 -- 172.21.15.131:0/3836239765 shutdown_connections 2024-04-26T20:09:59.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.180+0000 7f977e800700 1 -- 172.21.15.131:0/3836239765 wait complete. 2024-04-26T20:09:59.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.181+0000 7f977e800700 1 Processor -- start 2024-04-26T20:09:59.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f977e800700 1 -- start start 2024-04-26T20:09:59.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f977e800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9778103470 0x7f977819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:59.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f977e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9778103e30 0x7f977819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:59.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f977e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97781094f0 0x7f97781999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:09:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f977e800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9778113640 con 0x7f9778103e30 2024-04-26T20:09:59.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f977e800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97781134c0 con 0x7f9778103470 2024-04-26T20:09:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f977e800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97781137c0 con 0x7f97781094f0 2024-04-26T20:09:59.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f9776a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9778103e30 0x7f977819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f9776a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9778103e30 0x7f977819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40190/0 (socket says 172.21.15.131:40190) 2024-04-26T20:09:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f9776a00700 1 -- 172.21.15.131:0/2970845176 learned_addr learned my addr 172.21.15.131:0/2970845176 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:09:59.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.182+0000 7f9777400700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9778103470 0x7f977819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:59.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.183+0000 7f9777e00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97781094f0 0x7f97781999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:09:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.183+0000 7f9776a00700 1 -- 172.21.15.131:0/2970845176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97781094f0 msgr2=0x7f97781999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:09:59.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.183+0000 7f9776a00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97781094f0 0x7f97781999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:00.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.183+0000 7f9776a00700 1 -- 172.21.15.131:0/2970845176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9778103470 msgr2=0x7f977819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:00.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.183+0000 7f9776a00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9778103470 0x7f977819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:00.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.183+0000 7f9776a00700 1 -- 172.21.15.131:0/2970845176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977819a120 con 0x7f9778103e30 2024-04-26T20:10:00.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.183+0000 7f9776a00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9778103e30 0x7f977819fd40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9764009230 tx=0x7f976400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:00.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.184+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640050c0 con 0x7f9778103e30 2024-04-26T20:10:00.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.184+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f976400b560 con 0x7f9778103e30 2024-04-26T20:10:00.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.184+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977819a3b0 con 0x7f9778103e30 2024-04-26T20:10:00.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.184+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764003a00 con 0x7f9778103e30 2024-04-26T20:10:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.184+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97781a66d0 con 0x7f9778103e30 2024-04-26T20:10:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.185+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9778063bb0 con 0x7f9778103e30 2024-04-26T20:10:00.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.187+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f976400b710 con 0x7f9778103e30 2024-04-26T20:10:00.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.188+0000 7f976fe00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f975c0a3cd0 0x7f975c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:00.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.189+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f97640d1f10 con 0x7f9778103e30 2024-04-26T20:10:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.189+0000 7f9777400700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f975c0a3cd0 0x7f975c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:00.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.189+0000 7f976fe00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] conn(0x7f975c0aef80 0x7f975c0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:00.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.189+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 --> [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f976402fe50 con 0x7f975c0aef80 2024-04-26T20:10:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.189+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f97640d22c0 con 0x7f9778103e30 2024-04-26T20:10:00.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.190+0000 7f9777400700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f975c0a3cd0 0x7f975c0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9768009750 tx=0x7f9768009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.190+0000 7f9777e00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] conn(0x7f975c0aef80 0x7f975c0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.190+0000 7f9777e00700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] conn(0x7f975c0aef80 0x7f975c0b1420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:00.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.191+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== osd.9 v2:172.21.15.131:6826/529934339 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f976402fe50 con 0x7f975c0aef80 2024-04-26T20:10:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.210+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 --> [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9778064030 con 0x7f975c0aef80 2024-04-26T20:10:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f976fe00700 1 -- 172.21.15.131:0/2970845176 <== osd.9 v2:172.21.15.131:6826/529934339 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9778064030 con 0x7f975c0aef80 2024-04-26T20:10:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] conn(0x7f975c0aef80 msgr2=0x7f975c0b1420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:00.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f977e800700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] conn(0x7f975c0aef80 0x7f975c0b1420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f975c0a3cd0 msgr2=0x7f975c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f977e800700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f975c0a3cd0 0x7f975c0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9768009750 tx=0x7f9768009310 comp rx=0 tx=0).stop 2024-04-26T20:10:00.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9778103e30 msgr2=0x7f977819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:00.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f977e800700 1 --2- 172.21.15.131:0/2970845176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9778103e30 0x7f977819fd40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9764009230 tx=0x7f976400fa70 comp rx=0 tx=0).stop 2024-04-26T20:10:00.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.211+0000 7f9777e00700 1 -- 172.21.15.131:0/2970845176 reap_dead start 2024-04-26T20:10:00.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.212+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 shutdown_connections 2024-04-26T20:10:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.212+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 >> 172.21.15.131:0/2970845176 conn(0x7f977807a5d0 msgr2=0x7f97781022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:00.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.212+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 shutdown_connections 2024-04-26T20:10:00.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:55.212+0000 7f977e800700 1 -- 172.21.15.131:0/2970845176 wait complete. 2024-04-26T20:10:00.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 -- 172.21.15.131:0/612424920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 msgr2=0x7f18c010ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:00.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 --2- 172.21.15.131:0/612424920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 0x7f18c010ba20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009990 tx=0x7f18b0009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:00.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 -- 172.21.15.131:0/612424920 shutdown_connections 2024-04-26T20:10:00.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 --2- 172.21.15.131:0/612424920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c0106000 0x7f18c0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:00.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 --2- 172.21.15.131:0/612424920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 0x7f18c010ba20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:00.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 --2- 172.21.15.131:0/612424920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c0105640 0x7f18c0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:00.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 -- 172.21.15.131:0/612424920 >> 172.21.15.131:0/612424920 conn(0x7f18c0101130 msgr2=0x7f18c0103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:00.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 -- 172.21.15.131:0/612424920 shutdown_connections 2024-04-26T20:10:00.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.645+0000 7f18c7200700 1 -- 172.21.15.131:0/612424920 wait complete. 2024-04-26T20:10:00.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.646+0000 7f18c7200700 1 Processor -- start 2024-04-26T20:10:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.646+0000 7f18c7200700 1 -- start start 2024-04-26T20:10:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18c7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c0105640 0x7f18c01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18c7200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c0106000 0x7f18c01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:00.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18c7200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 0x7f18c019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:00.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18c7200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0115890 con 0x7f18c0106000 2024-04-26T20:10:00.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18c7200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0115710 con 0x7f18c010b640 2024-04-26T20:10:01.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18c7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0115a10 con 0x7f18c0105640 2024-04-26T20:10:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c0106000 0x7f18c01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:01.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c0106000 0x7f18c01a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40252/0 (socket says 172.21.15.131:40252) 2024-04-26T20:10:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18bf400700 1 -- 172.21.15.131:0/946641352 learned_addr learned my addr 172.21.15.131:0/946641352 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:01.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18c5200700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 0x7f18c019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.647+0000 7f18bfe00700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c0105640 0x7f18c01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18c5200700 1 -- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c0105640 msgr2=0x7f18c01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:01.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18c5200700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c0105640 0x7f18c01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18c5200700 1 -- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c0106000 msgr2=0x7f18c01a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:01.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18c5200700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c0106000 0x7f18c01a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18c5200700 1 -- 172.21.15.131:0/946641352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c019c410 con 0x7f18c010b640 2024-04-26T20:10:01.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18bf400700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c0106000 0x7f18c01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:01.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18c5200700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 0x7f18c019bb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18b800efc0 tx=0x7f18b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b80092e0 con 0x7f18c010b640 2024-04-26T20:10:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b8004bb0 con 0x7f18c010b640 2024-04-26T20:10:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8004020 con 0x7f18c010b640 2024-04-26T20:10:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.648+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c019c6a0 con 0x7f18c010b640 2024-04-26T20:10:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.649+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01a8810 con 0x7f18c010b640 2024-04-26T20:10:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.649+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18c0063b70 con 0x7f18c010b640 2024-04-26T20:10:01.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.652+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f18b801b030 con 0x7f18c010b640 2024-04-26T20:10:01.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.653+0000 7f18bcc00700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18a40a3d50 0x7f18a40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:01.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.653+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f18b80d0d60 con 0x7f18c010b640 2024-04-26T20:10:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.653+0000 7f18bfe00700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18a40a3d50 0x7f18a40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:01.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.654+0000 7f18bcc00700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] conn(0x7f18a40af130 0x7f18a40b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:01.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.654+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 --> [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18b800fe50 con 0x7f18a40af130 2024-04-26T20:10:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.654+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f18b80d1110 con 0x7f18c010b640 2024-04-26T20:10:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.654+0000 7f18bfe00700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18a40a3d50 0x7f18a40a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18b0006010 tx=0x7f18b000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.654+0000 7f18bf400700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] conn(0x7f18a40af130 0x7f18a40b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:01.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.655+0000 7f18bf400700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] conn(0x7f18a40af130 0x7f18a40b15d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:01.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.656+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== osd.8 v2:172.21.15.157:6816/1452104897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18b800fe50 con 0x7f18a40af130 2024-04-26T20:10:01.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.674+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 --> [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18c0063ff0 con 0x7f18a40af130 2024-04-26T20:10:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.674+0000 7f18bcc00700 1 -- 172.21.15.131:0/946641352 <== osd.8 v2:172.21.15.157:6816/1452104897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f18c0063ff0 con 0x7f18a40af130 2024-04-26T20:10:01.564 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:09:59 smithi157 ceph-mon[32703]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T20:10:01.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:00 smithi157 ceph-mon[32703]: overall HEALTH_OK 2024-04-26T20:10:01.651 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:09:59 smithi148 ceph-mon[32281]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T20:10:01.685 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:00 smithi148 ceph-mon[32281]: overall HEALTH_OK 2024-04-26T20:10:01.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] conn(0x7f18a40af130 msgr2=0x7f18a40b15d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:01.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] conn(0x7f18a40af130 0x7f18a40b15d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:01.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18a40a3d50 msgr2=0x7f18a40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f18a40a3d50 0x7f18a40a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18b0006010 tx=0x7f18b000b260 comp rx=0 tx=0).stop 2024-04-26T20:10:01.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 msgr2=0x7f18c019bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:01.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 --2- 172.21.15.131:0/946641352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c010b640 0x7f18c019bb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18b800efc0 tx=0x7f18b800c3a0 comp rx=0 tx=0).stop 2024-04-26T20:10:01.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c5200700 1 -- 172.21.15.131:0/946641352 reap_dead start 2024-04-26T20:10:01.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 shutdown_connections 2024-04-26T20:10:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.675+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 >> 172.21.15.131:0/946641352 conn(0x7f18c0101130 msgr2=0x7f18c01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.676+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 shutdown_connections 2024-04-26T20:10:01.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:09:58.676+0000 7f18c7200700 1 -- 172.21.15.131:0/946641352 wait complete. 2024-04-26T20:10:01.895 INFO:teuthology.orchestra.run.smithi131.stdout:163208757328 2024-04-26T20:10:01.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T20:10:01.915 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:09:59 smithi131 ceph-mon[27919]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T20:10:01.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:01 smithi148 ceph-mon[32281]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:02.012 INFO:teuthology.orchestra.run.smithi131.stdout:253403070528 2024-04-26T20:10:02.033 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T20:10:02.042 INFO:teuthology.orchestra.run.smithi131.stdout:55834574950 2024-04-26T20:10:02.080 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T20:10:02.088 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:00 smithi131 ceph-mon[27919]: overall HEALTH_OK 2024-04-26T20:10:02.096 INFO:teuthology.orchestra.run.smithi131.stdout:287762808889 2024-04-26T20:10:02.105 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T20:10:02.113 INFO:teuthology.orchestra.run.smithi131.stdout:141733920854 2024-04-26T20:10:02.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T20:10:02.132 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:01 smithi157 ceph-mon[32703]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:02.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 -- 172.21.15.131:0/616992183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103e30 msgr2=0x7f0814110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 --2- 172.21.15.131:0/616992183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103e30 0x7f0814110290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f080c00b0a0 tx=0x7f080c00b3b0 comp rx=0 tx=0).stop 2024-04-26T20:10:02.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 -- 172.21.15.131:0/616992183 shutdown_connections 2024-04-26T20:10:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 --2- 172.21.15.131:0/616992183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103e30 0x7f0814110290 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:02.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 --2- 172.21.15.131:0/616992183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08141094c0 0x7f08141098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:02.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 --2- 172.21.15.131:0/616992183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0814103470 0x7f08141038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 -- 172.21.15.131:0/616992183 >> 172.21.15.131:0/616992183 conn(0x7f081407a580 msgr2=0x7f081407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:02.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 -- 172.21.15.131:0/616992183 shutdown_connections 2024-04-26T20:10:02.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.701+0000 7f081ba00700 1 -- 172.21.15.131:0/616992183 wait complete. 2024-04-26T20:10:02.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.703+0000 7f081ba00700 1 Processor -- start 2024-04-26T20:10:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.703+0000 7f081ba00700 1 -- start start 2024-04-26T20:10:02.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f081ba00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103470 0x7f081419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:02.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f081ba00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0814103e30 0x7f081419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:02.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f081ba00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08141094c0 0x7f08141a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:02.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f081ba00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0814113630 con 0x7f08141094c0 2024-04-26T20:10:02.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f081ba00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08141134b0 con 0x7f0814103470 2024-04-26T20:10:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f081ba00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08141137b0 con 0x7f0814103e30 2024-04-26T20:10:02.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103470 0x7f081419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:02.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08141094c0 0x7f08141a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103470 0x7f081419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:60668/0 (socket says 172.21.15.131:60668) 2024-04-26T20:10:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 -- 172.21.15.131:0/822641365 learned_addr learned my addr 172.21.15.131:0/822641365 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0813e00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0814103e30 0x7f081419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:02.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 -- 172.21.15.131:0/822641365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0814103e30 msgr2=0x7f081419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:02.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0814103e30 0x7f081419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:02.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 -- 172.21.15.131:0/822641365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08141094c0 msgr2=0x7f08141a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:02.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08141094c0 0x7f08141a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:02.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.704+0000 7f0819000700 1 -- 172.21.15.131:0/822641365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08141a0db0 con 0x7f0814103470 2024-04-26T20:10:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.705+0000 7f0819000700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103470 0x7f081419bda0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:02.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.705+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080800cbd0 con 0x7f0814103470 2024-04-26T20:10:02.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.705+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f080800ee60 con 0x7f0814103470 2024-04-26T20:10:02.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.705+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08141a10a0 con 0x7f0814103470 2024-04-26T20:10:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.705+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080055c0 con 0x7f0814103470 2024-04-26T20:10:02.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.705+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08141a8920 con 0x7f0814103470 2024-04-26T20:10:02.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.706+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08141102d0 con 0x7f0814103470 2024-04-26T20:10:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.708+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0808004750 con 0x7f0814103470 2024-04-26T20:10:02.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.709+0000 7f0811600700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f07fc0a3cd0 0x7f07fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:02.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.710+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f08080d0fe0 con 0x7f0814103470 2024-04-26T20:10:02.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.710+0000 7f0813e00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f07fc0a3cd0 0x7f07fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:02.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.710+0000 7f0811600700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] conn(0x7f07fc0af0b0 0x7f07fc0b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.710+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 --> [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f080801ee50 con 0x7f07fc0af0b0 2024-04-26T20:10:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.710+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f08080d1390 con 0x7f0814103470 2024-04-26T20:10:03.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.710+0000 7f0813e00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f07fc0a3cd0 0x7f07fc0a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0800009680 tx=0x7f0800009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:03.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.711+0000 7f0819a00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] conn(0x7f07fc0af0b0 0x7f07fc0b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:03.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.711+0000 7f0819a00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] conn(0x7f07fc0af0b0 0x7f07fc0b1550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.712+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== osd.7 v2:172.21.15.148:6816/367491900 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f080801ee50 con 0x7f07fc0af0b0 2024-04-26T20:10:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.730+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 --> [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0814104b60 con 0x7f07fc0af0b0 2024-04-26T20:10:03.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.730+0000 7f0811600700 1 -- 172.21.15.131:0/822641365 <== osd.7 v2:172.21.15.148:6816/367491900 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0814104b60 con 0x7f07fc0af0b0 2024-04-26T20:10:03.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] conn(0x7f07fc0af0b0 msgr2=0x7f07fc0b1550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] conn(0x7f07fc0af0b0 0x7f07fc0b1550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:03.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f07fc0a3cd0 msgr2=0x7f07fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:03.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f07fc0a3cd0 0x7f07fc0a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0800009680 tx=0x7f0800009260 comp rx=0 tx=0).stop 2024-04-26T20:10:03.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103470 msgr2=0x7f081419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 --2- 172.21.15.131:0/822641365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0814103470 0x7f081419bda0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ecd0 comp rx=0 tx=0).stop 2024-04-26T20:10:03.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f0819a00700 1 -- 172.21.15.131:0/822641365 reap_dead start 2024-04-26T20:10:03.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 shutdown_connections 2024-04-26T20:10:03.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 >> 172.21.15.131:0/822641365 conn(0x7f081407a580 msgr2=0x7f081410e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:03.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.731+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 shutdown_connections 2024-04-26T20:10:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:00.732+0000 7f081ba00700 1 -- 172.21.15.131:0/822641365 wait complete. 2024-04-26T20:10:03.451 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:01 smithi131 ceph-mon[27919]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:03.472 INFO:teuthology.orchestra.run.smithi131.stdout:219043332165 2024-04-26T20:10:03.485 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T20:10:03.530 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:03.553 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:03.562 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:03.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:03 smithi131 ceph-mon[27919]: pgmap v411: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:03.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:03 smithi148 ceph-mon[32281]: pgmap v411: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:03.923 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:03.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:03 smithi157 ceph-mon[32703]: pgmap v411: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:05.632 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:05 smithi131 ceph-mon[27919]: pgmap v412: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-26T20:10:05.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:05 smithi148 ceph-mon[32281]: pgmap v412: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-26T20:10:05.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:05 smithi157 ceph-mon[32703]: pgmap v412: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-26T20:10:06.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 -- 172.21.15.131:0/3184659738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 msgr2=0x7f8e78113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:06.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 --2- 172.21.15.131:0/3184659738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e78113e20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009990 tx=0x7f8e64009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:06.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 -- 172.21.15.131:0/3184659738 shutdown_connections 2024-04-26T20:10:06.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 --2- 172.21.15.131:0/3184659738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78114360 0x7f8e78116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:06.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 --2- 172.21.15.131:0/3184659738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e780757c0 0x7f8e78075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 --2- 172.21.15.131:0/3184659738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e78113e20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:06.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 -- 172.21.15.131:0/3184659738 >> 172.21.15.131:0/3184659738 conn(0x7f8e780feea0 msgr2=0x7f8e781012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 -- 172.21.15.131:0/3184659738 shutdown_connections 2024-04-26T20:10:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.675+0000 7f8e7f000700 1 -- 172.21.15.131:0/3184659738 wait complete. 2024-04-26T20:10:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.676+0000 7f8e7f000700 1 Processor -- start 2024-04-26T20:10:06.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.676+0000 7f8e7f000700 1 -- start start 2024-04-26T20:10:06.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7f000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e780757c0 0x7f8e781a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e781a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:06.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7f000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78114360 0x7f8e781a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7f000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78119ba0 con 0x7f8e780757c0 2024-04-26T20:10:06.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7f000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78119a20 con 0x7f8e78114360 2024-04-26T20:10:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78119d20 con 0x7f8e780760e0 2024-04-26T20:10:06.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e781a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e781a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45266/0 (socket says 172.21.15.131:45266) 2024-04-26T20:10:06.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7d200700 1 -- 172.21.15.131:0/4035998325 learned_addr learned my addr 172.21.15.131:0/4035998325 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7dc00700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e780757c0 0x7f8e781a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:06.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.677+0000 7f8e7e600700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78114360 0x7f8e781a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:06.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7d200700 1 -- 172.21.15.131:0/4035998325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78114360 msgr2=0x7f8e781a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7d200700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78114360 0x7f8e781a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:06.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7d200700 1 -- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e780757c0 msgr2=0x7f8e781a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7d200700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e780757c0 0x7f8e781a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7d200700 1 -- 172.21.15.131:0/4035998325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e781a5060 con 0x7f8e780760e0 2024-04-26T20:10:06.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7dc00700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e780757c0 0x7f8e781a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:10:06.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7d200700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e781a0590 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005910 tx=0x7f8e6400b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:07.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e76000700 1 -- 172.21.15.131:0/4035998325 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64027070 con 0x7f8e780760e0 2024-04-26T20:10:07.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e781a52f0 con 0x7f8e780760e0 2024-04-26T20:10:07.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e76000700 1 -- 172.21.15.131:0/4035998325 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e64022470 con 0x7f8e780760e0 2024-04-26T20:10:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e76000700 1 -- 172.21.15.131:0/4035998325 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64017650 con 0x7f8e780760e0 2024-04-26T20:10:07.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.678+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e781acbd0 con 0x7f8e780760e0 2024-04-26T20:10:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.680+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7806a110 con 0x7f8e780760e0 2024-04-26T20:10:07.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.682+0000 7f8e76000700 1 -- 172.21.15.131:0/4035998325 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8e6402a070 con 0x7f8e780760e0 2024-04-26T20:10:07.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.683+0000 7f8e76000700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e600a3d50 0x7f8e600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:07.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.683+0000 7f8e76000700 1 -- 172.21.15.131:0/4035998325 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8e640d1670 con 0x7f8e780760e0 2024-04-26T20:10:07.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.683+0000 7f8e7dc00700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e600a3d50 0x7f8e600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.684+0000 7f8e7dc00700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e600a3d50 0x7f8e600a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009cb0 tx=0x7f8e6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.687+0000 7f8e76000700 1 -- 172.21.15.131:0/4035998325 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8e64013070 con 0x7f8e780760e0 2024-04-26T20:10:07.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.922+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8e781ace00 con 0x7f8e780760e0 2024-04-26T20:10:07.266 INFO:teuthology.orchestra.run.smithi131.stdout:107374182493 2024-04-26T20:10:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.923+0000 7f8e76000700 1 -- 172.21.15.131:0/4035998325 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e6409db30 con 0x7f8e780760e0 2024-04-26T20:10:07.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e600a3d50 msgr2=0x7f8e600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:07.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e600a3d50 0x7f8e600a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009cb0 tx=0x7f8e6c009260 comp rx=0 tx=0).stop 2024-04-26T20:10:07.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 msgr2=0x7f8e781a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:07.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e781a0590 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005910 tx=0x7f8e6400b670 comp rx=0 tx=0).stop 2024-04-26T20:10:07.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 shutdown_connections 2024-04-26T20:10:07.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8e600a3d50 0x7f8e600a6210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:07.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78114360 0x7f8e781a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e780757c0 0x7f8e781a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:07.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 --2- 172.21.15.131:0/4035998325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e780760e0 0x7f8e781a0590 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.926+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 >> 172.21.15.131:0/4035998325 conn(0x7f8e780feea0 msgr2=0x7f8e781005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:07.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.927+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 shutdown_connections 2024-04-26T20:10:07.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:05.927+0000 7f8e7f000700 1 -- 172.21.15.131:0/4035998325 wait complete. 2024-04-26T20:10:07.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:06 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4035998325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T20:10:07.668 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182488 got 107374182493 for osd.3 2024-04-26T20:10:07.677 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:06 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4035998325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T20:10:07.807 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:06 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4035998325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T20:10:07.831 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:07 smithi131 ceph-mon[27919]: pgmap v413: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:07.843 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:07 smithi148 ceph-mon[32281]: pgmap v413: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:07.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:07 smithi157 ceph-mon[32703]: pgmap v413: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:08.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.006+0000 7effdf800700 1 -- 172.21.15.131:0/2875710175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8105640 msgr2=0x7effd8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:08.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.006+0000 7effdf800700 1 --2- 172.21.15.131:0/2875710175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8105640 0x7effd8105ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7effc8009990 tx=0x7effc8009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.007+0000 7effdf800700 1 -- 172.21.15.131:0/2875710175 shutdown_connections 2024-04-26T20:10:08.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.007+0000 7effdf800700 1 --2- 172.21.15.131:0/2875710175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd8106000 0x7effd8112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:08.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.007+0000 7effdf800700 1 --2- 172.21.15.131:0/2875710175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd810b640 0x7effd810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:08.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.007+0000 7effdf800700 1 --2- 172.21.15.131:0/2875710175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8105640 0x7effd8105ac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:08.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.007+0000 7effdf800700 1 -- 172.21.15.131:0/2875710175 >> 172.21.15.131:0/2875710175 conn(0x7effd8101130 msgr2=0x7effd8103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:08.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.007+0000 7effdf800700 1 -- 172.21.15.131:0/2875710175 shutdown_connections 2024-04-26T20:10:08.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.007+0000 7effdf800700 1 -- 172.21.15.131:0/2875710175 wait complete. 2024-04-26T20:10:08.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.008+0000 7effdf800700 1 Processor -- start 2024-04-26T20:10:08.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.008+0000 7effdf800700 1 -- start start 2024-04-26T20:10:08.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdf800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd8105640 0x7effd81a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:08.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdf800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8106000 0x7effd81a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:08.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdf800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd810b640 0x7effd819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:08.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdf800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effd8115820 con 0x7effd8106000 2024-04-26T20:10:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdf800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7effd81156a0 con 0x7effd8105640 2024-04-26T20:10:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdf800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effd81159a0 con 0x7effd810b640 2024-04-26T20:10:08.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8106000 0x7effd81a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:08.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8106000 0x7effd81a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33602/0 (socket says 172.21.15.131:33602) 2024-04-26T20:10:08.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 -- 172.21.15.131:0/2262576789 learned_addr learned my addr 172.21.15.131:0/2262576789 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:08.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd8105640 0x7effd81a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:08.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effdd800700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd810b640 0x7effd819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:08.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 -- 172.21.15.131:0/2262576789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd810b640 msgr2=0x7effd819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:08.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd810b640 0x7effd819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 -- 172.21.15.131:0/2262576789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd8105640 msgr2=0x7effd81a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:08.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd8105640 0x7effd81a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:08.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.009+0000 7effd7e00700 1 -- 172.21.15.131:0/2262576789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd819c280 con 0x7effd8106000 2024-04-26T20:10:08.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.010+0000 7effd7e00700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8106000 0x7effd81a1e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7effc80057f0 tx=0x7effc800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:08.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.010+0000 7effd5600700 1 -- 172.21.15.131:0/2262576789 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8026070 con 0x7effd8106000 2024-04-26T20:10:08.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.010+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd819c510 con 0x7effd8106000 2024-04-26T20:10:08.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.010+0000 7effd5600700 1 -- 172.21.15.131:0/2262576789 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc8021410 con 0x7effd8106000 2024-04-26T20:10:08.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.010+0000 7effd5600700 1 -- 172.21.15.131:0/2262576789 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8017650 con 0x7effd8106000 2024-04-26T20:10:08.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.010+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a87e0 con 0x7effd8106000 2024-04-26T20:10:09.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.012+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd806a110 con 0x7effd8106000 2024-04-26T20:10:09.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.013+0000 7effd5600700 1 -- 172.21.15.131:0/2262576789 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7effc8029070 con 0x7effd8106000 2024-04-26T20:10:09.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.015+0000 7effd5600700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effc00a3cd0 0x7effc00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.015+0000 7effd5600700 1 -- 172.21.15.131:0/2262576789 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7effc80d15e0 con 0x7effd8106000 2024-04-26T20:10:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.015+0000 7effdce00700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effc00a3cd0 0x7effc00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:09.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.016+0000 7effdce00700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effc00a3cd0 0x7effc00a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7effcc009680 tx=0x7effcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:09.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.020+0000 7effd5600700 1 -- 172.21.15.131:0/2262576789 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7effc805e060 con 0x7effd8106000 2024-04-26T20:10:09.131 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:08 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2262576789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T20:10:09.140 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:08 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2262576789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T20:10:09.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.253+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7effd802e490 con 0x7effd8106000 2024-04-26T20:10:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.254+0000 7effd5600700 1 -- 172.21.15.131:0/2262576789 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effc8013070 con 0x7effd8106000 2024-04-26T20:10:09.304 INFO:teuthology.orchestra.run.smithi131.stdout:77309411428 2024-04-26T20:10:09.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effc00a3cd0 msgr2=0x7effc00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effc00a3cd0 0x7effc00a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7effcc009680 tx=0x7effcc009260 comp rx=0 tx=0).stop 2024-04-26T20:10:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8106000 msgr2=0x7effd81a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:09.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8106000 0x7effd81a1e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7effc80057f0 tx=0x7effc800bfd0 comp rx=0 tx=0).stop 2024-04-26T20:10:09.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 shutdown_connections 2024-04-26T20:10:09.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd810b640 0x7effd819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effc00a3cd0 0x7effc00a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd8105640 0x7effd81a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 --2- 172.21.15.131:0/2262576789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8106000 0x7effd81a1e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 >> 172.21.15.131:0/2262576789 conn(0x7effd8101130 msgr2=0x7effd8101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 shutdown_connections 2024-04-26T20:10:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:08.257+0000 7effdf800700 1 -- 172.21.15.131:0/2262576789 wait complete. 2024-04-26T20:10:09.572 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:08 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2262576789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T20:10:09.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:09 smithi148 ceph-mon[32281]: pgmap v414: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T20:10:09.921 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:09 smithi131 ceph-mon[27919]: pgmap v414: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T20:10:09.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:09 smithi157 ceph-mon[32703]: pgmap v414: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T20:10:10.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 -- 172.21.15.131:0/637044080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810fa50 msgr2=0x7f7fc810fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:10.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 --2- 172.21.15.131:0/637044080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810fa50 0x7f7fc810fe30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009990 tx=0x7f7fb8009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:10.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 -- 172.21.15.131:0/637044080 shutdown_connections 2024-04-26T20:10:10.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 --2- 172.21.15.131:0/637044080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc810a410 0x7f7fc8116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 --2- 172.21.15.131:0/637044080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810fa50 0x7f7fc810fe30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:10.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 --2- 172.21.15.131:0/637044080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fc8109a50 0x7f7fc8109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 -- 172.21.15.131:0/637044080 >> 172.21.15.131:0/637044080 conn(0x7f7fc8076bf0 msgr2=0x7f7fc8108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:10.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 -- 172.21.15.131:0/637044080 shutdown_connections 2024-04-26T20:10:10.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.279+0000 7f7fce800700 1 -- 172.21.15.131:0/637044080 wait complete. 2024-04-26T20:10:10.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 Processor -- start 2024-04-26T20:10:10.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 -- start start 2024-04-26T20:10:10.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8109a50 0x7f7fc81a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:10.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810a410 0x7f7fc81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:10.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fc810fa50 0x7f7fc81a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:10.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8119ca0 con 0x7f7fc810fa50 2024-04-26T20:10:10.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8119b20 con 0x7f7fc810a410 2024-04-26T20:10:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.281+0000 7f7fce800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8119e20 con 0x7f7fc8109a50 2024-04-26T20:10:10.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8109a50 0x7f7fc81a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:10.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8109a50 0x7f7fc81a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45288/0 (socket says 172.21.15.131:45288) 2024-04-26T20:10:10.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 -- 172.21.15.131:0/3741698226 learned_addr learned my addr 172.21.15.131:0/3741698226 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:10.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7e00700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fc810fa50 0x7f7fc81a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:10.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc6a00700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810a410 0x7f7fc81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:10.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 -- 172.21.15.131:0/3741698226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810a410 msgr2=0x7f7fc81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:10.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810a410 0x7f7fc81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:10.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 -- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fc810fa50 msgr2=0x7f7fc81a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:10.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fc810fa50 0x7f7fc81a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:10.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.282+0000 7f7fc7400700 1 -- 172.21.15.131:0/3741698226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc81a5100 con 0x7f7fc8109a50 2024-04-26T20:10:10.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.283+0000 7f7fc7400700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8109a50 0x7f7fc81a00f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8005910 tx=0x7f7fb800b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:10.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.283+0000 7f7fbfe00700 1 -- 172.21.15.131:0/3741698226 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8027070 con 0x7f7fc8109a50 2024-04-26T20:10:10.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.283+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc81a5390 con 0x7f7fc8109a50 2024-04-26T20:10:10.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.283+0000 7f7fbfe00700 1 -- 172.21.15.131:0/3741698226 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb8022470 con 0x7f7fc8109a50 2024-04-26T20:10:10.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.283+0000 7f7fbfe00700 1 -- 172.21.15.131:0/3741698226 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8017650 con 0x7f7fc8109a50 2024-04-26T20:10:10.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.283+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81acc70 con 0x7f7fc8109a50 2024-04-26T20:10:10.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.284+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc8111fe0 con 0x7f7fc8109a50 2024-04-26T20:10:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.287+0000 7f7fbfe00700 1 -- 172.21.15.131:0/3741698226 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7fb802a070 con 0x7f7fc8109a50 2024-04-26T20:10:11.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.288+0000 7f7fbfe00700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7fac0a3d50 0x7f7fac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.288+0000 7f7fbfe00700 1 -- 172.21.15.131:0/3741698226 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7fb80d17a0 con 0x7f7fc8109a50 2024-04-26T20:10:11.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.288+0000 7f7fc6a00700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7fac0a3d50 0x7f7fac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:11.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.289+0000 7f7fc6a00700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7fac0a3d50 0x7f7fac0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009680 tx=0x7f7fb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.291+0000 7f7fbfe00700 1 -- 172.21.15.131:0/3741698226 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7fb8013070 con 0x7f7fc8109a50 2024-04-26T20:10:11.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:10 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3741698226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T20:10:11.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:10 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3741698226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T20:10:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.533+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7fc81a1450 con 0x7f7fc8109a50 2024-04-26T20:10:11.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.533+0000 7f7fbfe00700 1 -- 172.21.15.131:0/3741698226 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7fb809dc60 con 0x7f7fc8109a50 2024-04-26T20:10:11.321 INFO:teuthology.orchestra.run.smithi131.stdout:197568495693 2024-04-26T20:10:11.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.536+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7fac0a3d50 msgr2=0x7f7fac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.536+0000 7f7fce800700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7fac0a3d50 0x7f7fac0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009680 tx=0x7f7fb4009260 comp rx=0 tx=0).stop 2024-04-26T20:10:11.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.536+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8109a50 msgr2=0x7f7fc81a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.536+0000 7f7fce800700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8109a50 0x7f7fc81a00f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8005910 tx=0x7f7fb800b670 comp rx=0 tx=0).stop 2024-04-26T20:10:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 shutdown_connections 2024-04-26T20:10:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fc810fa50 0x7f7fc81a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:11.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8109a50 0x7f7fc81a00f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7fac0a3d50 0x7f7fac0a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:11.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 --2- 172.21.15.131:0/3741698226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fc810a410 0x7f7fc81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:11.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 >> 172.21.15.131:0/3741698226 conn(0x7f7fc8076bf0 msgr2=0x7f7fc8077210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 shutdown_connections 2024-04-26T20:10:11.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:10.537+0000 7f7fce800700 1 -- 172.21.15.131:0/3741698226 wait complete. 2024-04-26T20:10:11.594 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:10 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3741698226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T20:10:11.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411425 got 77309411428 for osd.2 2024-04-26T20:10:11.805 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:11.811 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:11 smithi131 ceph-mon[27919]: pgmap v415: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T20:10:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:11.827 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:11 smithi148 ceph-mon[32281]: pgmap v415: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T20:10:11.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:11 smithi157 ceph-mon[32703]: pgmap v415: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T20:10:13.688 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:13 smithi131 ceph-mon[27919]: pgmap v416: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:13.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:13 smithi148 ceph-mon[32281]: pgmap v416: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:13.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:13 smithi157 ceph-mon[32703]: pgmap v416: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:14.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 -- 172.21.15.131:0/2306154789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 msgr2=0x7f28901167c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:14.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 --2- 172.21.15.131:0/2306154789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 0x7f28901167c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f288800b0a0 tx=0x7f288800b3b0 comp rx=0 tx=0).stop 2024-04-26T20:10:14.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 -- 172.21.15.131:0/2306154789 shutdown_connections 2024-04-26T20:10:14.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 --2- 172.21.15.131:0/2306154789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 0x7f28901167c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:14.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 --2- 172.21.15.131:0/2306154789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289010e830 0x7f289010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:14.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 --2- 172.21.15.131:0/2306154789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 0x7f2890109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:14.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 -- 172.21.15.131:0/2306154789 >> 172.21.15.131:0/2306154789 conn(0x7f28900791b0 msgr2=0x7f2890108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 -- 172.21.15.131:0/2306154789 shutdown_connections 2024-04-26T20:10:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.197+0000 7f2896200700 1 -- 172.21.15.131:0/2306154789 wait complete. 2024-04-26T20:10:14.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.198+0000 7f2896200700 1 Processor -- start 2024-04-26T20:10:14.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.198+0000 7f2896200700 1 -- start start 2024-04-26T20:10:14.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2896200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 0x7f289019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:14.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2896200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 0x7f28901a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:14.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2896200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289010e830 0x7f28901a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:14.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2896200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2890119b80 con 0x7f289010a320 2024-04-26T20:10:14.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2896200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2890119a00 con 0x7f289010e830 2024-04-26T20:10:14.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2896200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2890119d00 con 0x7f2890109960 2024-04-26T20:10:14.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f288fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 0x7f28901a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 0x7f289019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:14.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 0x7f289019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41612/0 (socket says 172.21.15.131:41612) 2024-04-26T20:10:14.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 -- 172.21.15.131:0/85650324 learned_addr learned my addr 172.21.15.131:0/85650324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2895800700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289010e830 0x7f28901a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:14.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 -- 172.21.15.131:0/85650324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289010e830 msgr2=0x7f28901a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:14.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289010e830 0x7f28901a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:14.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 -- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 msgr2=0x7f28901a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 0x7f28901a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:14.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.199+0000 7f2894e00700 1 -- 172.21.15.131:0/85650324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28901a5000 con 0x7f2890109960 2024-04-26T20:10:14.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.200+0000 7f2894e00700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 0x7f289019fff0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f288400e9a0 tx=0x7f288400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:14.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.200+0000 7f288d600700 1 -- 172.21.15.131:0/85650324 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288400ca10 con 0x7f2890109960 2024-04-26T20:10:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.200+0000 7f288d600700 1 -- 172.21.15.131:0/85650324 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288400cbb0 con 0x7f2890109960 2024-04-26T20:10:14.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.200+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28901a52f0 con 0x7f2890109960 2024-04-26T20:10:14.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.200+0000 7f288d600700 1 -- 172.21.15.131:0/85650324 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884018680 con 0x7f2890109960 2024-04-26T20:10:14.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.200+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28901acb70 con 0x7f2890109960 2024-04-26T20:10:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.201+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289006a110 con 0x7f2890109960 2024-04-26T20:10:14.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.203+0000 7f288d600700 1 -- 172.21.15.131:0/85650324 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2884010430 con 0x7f2890109960 2024-04-26T20:10:14.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.205+0000 7f288d600700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28780a3cd0 0x7f28780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.205+0000 7f288d600700 1 -- 172.21.15.131:0/85650324 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f28840d0ac0 con 0x7f2890109960 2024-04-26T20:10:14.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.205+0000 7f288fe00700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28780a3cd0 0x7f28780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:14.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.206+0000 7f288fe00700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28780a3cd0 0x7f28780a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2890109440 tx=0x7f287c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:14.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.209+0000 7f288d600700 1 -- 172.21.15.131:0/85650324 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f288409cef0 con 0x7f2890109960 2024-04-26T20:10:14.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:14 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/85650324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T20:10:14.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.437+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f289002e490 con 0x7f2890109960 2024-04-26T20:10:15.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.437+0000 7f288d600700 1 -- 172.21.15.131:0/85650324 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f288409c7f0 con 0x7f2890109960 2024-04-26T20:10:15.090 INFO:teuthology.orchestra.run.smithi131.stdout:343597383729 2024-04-26T20:10:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28780a3cd0 msgr2=0x7f28780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28780a3cd0 0x7f28780a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2890109440 tx=0x7f287c009260 comp rx=0 tx=0).stop 2024-04-26T20:10:15.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 msgr2=0x7f289019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:15.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 0x7f289019fff0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f288400e9a0 tx=0x7f288400ecb0 comp rx=0 tx=0).stop 2024-04-26T20:10:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 shutdown_connections 2024-04-26T20:10:15.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28780a3cd0 0x7f28780a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:15.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289010e830 0x7f28901a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:15.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2890109960 0x7f289019fff0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:15.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 --2- 172.21.15.131:0/85650324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f289010a320 0x7f28901a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:15.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 >> 172.21.15.131:0/85650324 conn(0x7f28900791b0 msgr2=0x7f2890079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 shutdown_connections 2024-04-26T20:10:15.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:14.441+0000 7f2896200700 1 -- 172.21.15.131:0/85650324 wait complete. 2024-04-26T20:10:15.446 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:14 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/85650324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T20:10:15.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:14 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/85650324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T20:10:15.489 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495689 got 197568495693 for osd.6 2024-04-26T20:10:15.625 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:15.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:15 smithi131 ceph-mon[27919]: pgmap v417: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:15.828 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:15 smithi148 ceph-mon[32281]: pgmap v417: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:15.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383724 got 343597383729 for osd.11 2024-04-26T20:10:15.935 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:15.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:15 smithi157 ceph-mon[32703]: pgmap v417: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:16.928 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:17.011 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:17.019 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:17.093 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:17.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.956+0000 7fdfe7e00700 1 -- 172.21.15.131:0/423793193 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc017400 con 0x7fdfe8108350 2024-04-26T20:10:17.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.956+0000 7fdff0000700 1 -- 172.21.15.131:0/423793193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfe8108350 msgr2=0x7fdfe806b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:17.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.956+0000 7fdff0000700 1 --2- 172.21.15.131:0/423793193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfe8108350 0x7fdfe806b130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0098f0 tx=0x7fdfdc009c00 comp rx=0 tx=0).stop 2024-04-26T20:10:17.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.958+0000 7fdff0000700 1 -- 172.21.15.131:0/423793193 shutdown_connections 2024-04-26T20:10:17.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.958+0000 7fdff0000700 1 --2- 172.21.15.131:0/423793193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe8105e60 0x7fdfe81062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:17.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.958+0000 7fdff0000700 1 --2- 172.21.15.131:0/423793193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfe8108350 0x7fdfe806b130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:17.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.958+0000 7fdff0000700 1 --2- 172.21.15.131:0/423793193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe806b670 0x7fdfe806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.958+0000 7fdff0000700 1 -- 172.21.15.131:0/423793193 >> 172.21.15.131:0/423793193 conn(0x7fdfe80792d0 msgr2=0x7fdfe80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.958+0000 7fdff0000700 1 -- 172.21.15.131:0/423793193 shutdown_connections 2024-04-26T20:10:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.958+0000 7fdff0000700 1 -- 172.21.15.131:0/423793193 wait complete. 2024-04-26T20:10:17.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.960+0000 7fdff0000700 1 Processor -- start 2024-04-26T20:10:17.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.960+0000 7fdff0000700 1 -- start start 2024-04-26T20:10:17.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.961+0000 7fdff0000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe806b670 0x7fdfe81165d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:17.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.961+0000 7fdff0000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe8105e60 0x7fdfe8116b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:17.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.961+0000 7fdff0000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfe8108350 0x7fdfe810f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:17.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.962+0000 7fdff0000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8119c30 con 0x7fdfe8105e60 2024-04-26T20:10:17.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.962+0000 7fdff0000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8119ab0 con 0x7fdfe806b670 2024-04-26T20:10:17.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.962+0000 7fdff0000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8119db0 con 0x7fdfe8108350 2024-04-26T20:10:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.962+0000 7fdfed600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe806b670 0x7fdfe81165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.962+0000 7fdfed600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe806b670 0x7fdfe81165d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:33318/0 (socket says 172.21.15.131:33318) 2024-04-26T20:10:17.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.962+0000 7fdfed600700 1 -- 172.21.15.131:0/1303941272 learned_addr learned my addr 172.21.15.131:0/1303941272 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:17.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.964+0000 7fdfecc00700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe8105e60 0x7fdfe8116b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:17.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.965+0000 7fdfecc00700 1 -- 172.21.15.131:0/1303941272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfe8108350 msgr2=0x7fdfe810f7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:10:17.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.965+0000 7fdfecc00700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfe8108350 0x7fdfe810f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:17.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.965+0000 7fdfecc00700 1 -- 172.21.15.131:0/1303941272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe806b670 msgr2=0x7fdfe81165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:17.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.965+0000 7fdfecc00700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe806b670 0x7fdfe81165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:17.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.965+0000 7fdfecc00700 1 -- 172.21.15.131:0/1303941272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81100d0 con 0x7fdfe8105e60 2024-04-26T20:10:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.965+0000 7fdfecc00700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe8105e60 0x7fdfe8116b10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400d750 tx=0x7fdfd400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.967+0000 7fdfe6000700 1 -- 172.21.15.131:0/1303941272 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4016070 con 0x7fdfe8105e60 2024-04-26T20:10:17.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.967+0000 7fdfe6000700 1 -- 172.21.15.131:0/1303941272 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd400b6f0 con 0x7fdfe8105e60 2024-04-26T20:10:17.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.967+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81103c0 con 0x7fdfe8105e60 2024-04-26T20:10:17.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.967+0000 7fdfe6000700 1 -- 172.21.15.131:0/1303941272 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4016070 con 0x7fdfe8105e60 2024-04-26T20:10:17.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.967+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe8114320 con 0x7fdfe8105e60 2024-04-26T20:10:17.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.969+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe8102940 con 0x7fdfe8105e60 2024-04-26T20:10:17.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.976+0000 7fdfe6000700 1 -- 172.21.15.131:0/1303941272 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdfd4004220 con 0x7fdfe8105e60 2024-04-26T20:10:17.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.979+0000 7fdfe6000700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdfd00a3ed0 0x7fdfd00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:17.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.980+0000 7fdfe6000700 1 -- 172.21.15.131:0/1303941272 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fdfd40d03e0 con 0x7fdfe8105e60 2024-04-26T20:10:17.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.982+0000 7fdfed600700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdfd00a3ed0 0x7fdfd00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:17.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.985+0000 7fdfe6000700 1 -- 172.21.15.131:0/1303941272 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdfd401f030 con 0x7fdfe8105e60 2024-04-26T20:10:17.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:16.986+0000 7fdfed600700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdfd00a3ed0 0x7fdfd00a6390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc005bf0 tx=0x7fdfdc005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:17.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.230+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fdfe8110c60 con 0x7fdfe8105e60 2024-04-26T20:10:17.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.231+0000 7fdfe6000700 1 -- 172.21.15.131:0/1303941272 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdfe8110c60 con 0x7fdfe8105e60 2024-04-26T20:10:17.878 INFO:teuthology.orchestra.run.smithi131.stdout:309237645366 2024-04-26T20:10:17.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdfd00a3ed0 msgr2=0x7fdfd00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:17.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdfd00a3ed0 0x7fdfd00a6390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc005bf0 tx=0x7fdfdc005b40 comp rx=0 tx=0).stop 2024-04-26T20:10:17.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe8105e60 msgr2=0x7fdfe8116b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:17.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe8105e60 0x7fdfe8116b10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400d750 tx=0x7fdfd400db10 comp rx=0 tx=0).stop 2024-04-26T20:10:18.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 shutdown_connections 2024-04-26T20:10:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfe8108350 0x7fdfe810f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:18.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdfd00a3ed0 0x7fdfd00a6390 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:18.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe806b670 0x7fdfe81165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:18.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 --2- 172.21.15.131:0/1303941272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe8105e60 0x7fdfe8116b10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:18.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 >> 172.21.15.131:0/1303941272 conn(0x7fdfe80792d0 msgr2=0x7fdfe8104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.234+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 shutdown_connections 2024-04-26T20:10:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:17.235+0000 7fdff0000700 1 -- 172.21.15.131:0/1303941272 wait complete. 2024-04-26T20:10:18.154 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:17 smithi131 ceph-mon[27919]: pgmap v418: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:18.161 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:17 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1303941272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T20:10:18.167 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:17 smithi148 ceph-mon[32281]: pgmap v418: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:18.174 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:17 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1303941272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T20:10:18.248 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:17 smithi157 ceph-mon[32703]: pgmap v418: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:18.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:17 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1303941272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T20:10:18.533 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645362 got 309237645366 for osd.10 2024-04-26T20:10:18.551 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:19.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:19 smithi131 ceph-mon[27919]: pgmap v419: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:19.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:19 smithi157 ceph-mon[32703]: pgmap v419: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:20.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:19 smithi148 ceph-mon[32281]: pgmap v419: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:21.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 -- 172.21.15.131:0/1871941258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b80109a50 msgr2=0x7f5b80109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:21.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 --2- 172.21.15.131:0/1871941258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b80109a50 0x7f5b80109ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009990 tx=0x7f5b6c009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:21.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 -- 172.21.15.131:0/1871941258 shutdown_connections 2024-04-26T20:10:21.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 --2- 172.21.15.131:0/1871941258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b8010a410 0x7f5b80116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:21.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 --2- 172.21.15.131:0/1871941258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b8010fa50 0x7f5b8010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:21.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 --2- 172.21.15.131:0/1871941258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b80109a50 0x7f5b80109ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:21.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 -- 172.21.15.131:0/1871941258 >> 172.21.15.131:0/1871941258 conn(0x7f5b80076bf0 msgr2=0x7f5b80108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:21.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 -- 172.21.15.131:0/1871941258 shutdown_connections 2024-04-26T20:10:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.503+0000 7f5b88400700 1 -- 172.21.15.131:0/1871941258 wait complete. 2024-04-26T20:10:21.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.504+0000 7f5b88400700 1 Processor -- start 2024-04-26T20:10:21.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.504+0000 7f5b88400700 1 -- start start 2024-04-26T20:10:21.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b88400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b80109a50 0x7f5b801a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:21.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b88400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b8010a410 0x7f5b801a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:21.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b88400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b8010fa50 0x7f5b801a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:21.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b88400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80119c90 con 0x7f5b8010a410 2024-04-26T20:10:21.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b88400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80119b10 con 0x7f5b8010fa50 2024-04-26T20:10:22.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b88400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80119e10 con 0x7f5b80109a50 2024-04-26T20:10:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b85a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b80109a50 0x7f5b801a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:22.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b85000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b8010a410 0x7f5b801a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:22.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b85a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b80109a50 0x7f5b801a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41650/0 (socket says 172.21.15.131:41650) 2024-04-26T20:10:22.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b85a00700 1 -- 172.21.15.131:0/1343595699 learned_addr learned my addr 172.21.15.131:0/1343595699 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.505+0000 7f5b86400700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b8010fa50 0x7f5b801a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b85a00700 1 -- 172.21.15.131:0/1343595699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b8010fa50 msgr2=0x7f5b801a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:22.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b85a00700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b8010fa50 0x7f5b801a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:22.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b85a00700 1 -- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b8010a410 msgr2=0x7f5b801a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:22.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b85a00700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b8010a410 0x7f5b801a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:22.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b85a00700 1 -- 172.21.15.131:0/1343595699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b801a5130 con 0x7f5b80109a50 2024-04-26T20:10:22.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b85a00700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b80109a50 0x7f5b801a0120 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e9a0 tx=0x7f5b7400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:22.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b7e000700 1 -- 172.21.15.131:0/1343595699 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400ca10 con 0x7f5b80109a50 2024-04-26T20:10:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b7e000700 1 -- 172.21.15.131:0/1343595699 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b7400cbb0 con 0x7f5b80109a50 2024-04-26T20:10:22.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b801a5420 con 0x7f5b80109a50 2024-04-26T20:10:22.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b7e000700 1 -- 172.21.15.131:0/1343595699 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74018680 con 0x7f5b80109a50 2024-04-26T20:10:22.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.506+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b801acca0 con 0x7f5b80109a50 2024-04-26T20:10:22.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.507+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8006a110 con 0x7f5b80109a50 2024-04-26T20:10:22.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.510+0000 7f5b7e000700 1 -- 172.21.15.131:0/1343595699 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b74010430 con 0x7f5b80109a50 2024-04-26T20:10:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.511+0000 7f5b7e000700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b680a3cd0 0x7f5b680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:22.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.511+0000 7f5b7e000700 1 -- 172.21.15.131:0/1343595699 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5b740d0b80 con 0x7f5b80109a50 2024-04-26T20:10:22.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.511+0000 7f5b85000700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b680a3cd0 0x7f5b680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.512+0000 7f5b85000700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b680a3cd0 0x7f5b680a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b80078b40 tx=0x7f5b6c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:22.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.514+0000 7f5b7e000700 1 -- 172.21.15.131:0/1343595699 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5b7409cfb0 con 0x7f5b80109a50 2024-04-26T20:10:22.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:21 smithi148 ceph-mon[32281]: pgmap v420: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:22.606 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:21 smithi157 ceph-mon[32703]: pgmap v420: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:22.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.742+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5b8002e490 con 0x7f5b80109a50 2024-04-26T20:10:22.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.743+0000 7f5b7e000700 1 -- 172.21.15.131:0/1343595699 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b7409c8b0 con 0x7f5b80109a50 2024-04-26T20:10:22.623 INFO:teuthology.orchestra.run.smithi131.stdout:34359738481 2024-04-26T20:10:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.745+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b680a3cd0 msgr2=0x7f5b680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.745+0000 7f5b88400700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b680a3cd0 0x7f5b680a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b80078b40 tx=0x7f5b6c00b260 comp rx=0 tx=0).stop 2024-04-26T20:10:22.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.745+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b80109a50 msgr2=0x7f5b801a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.745+0000 7f5b88400700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b80109a50 0x7f5b801a0120 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e9a0 tx=0x7f5b7400ecb0 comp rx=0 tx=0).stop 2024-04-26T20:10:22.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 shutdown_connections 2024-04-26T20:10:22.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b680a3cd0 0x7f5b680a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:22.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b8010fa50 0x7f5b801a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:22.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b80109a50 0x7f5b801a0120 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 --2- 172.21.15.131:0/1343595699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b8010a410 0x7f5b801a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:22.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 >> 172.21.15.131:0/1343595699 conn(0x7f5b80076bf0 msgr2=0x7f5b800771d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:22.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 shutdown_connections 2024-04-26T20:10:22.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:21.746+0000 7f5b88400700 1 -- 172.21.15.131:0/1343595699 wait complete. 2024-04-26T20:10:22.931 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:21 smithi131 ceph-mon[27919]: pgmap v420: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:22.939 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:22 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1343595699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T20:10:22.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:22 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1343595699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T20:10:23.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:22 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1343595699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T20:10:23.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:23 smithi131 ceph-mon[27919]: pgmap v421: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:10:23.763 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:23 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3489676634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T20:10:23.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:23 smithi157 ceph-mon[32703]: pgmap v421: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:10:24.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:23 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3489676634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T20:10:24.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:23 smithi148 ceph-mon[32281]: pgmap v421: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:10:24.078 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:23 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3489676634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T20:10:24.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738475 got 34359738481 for osd.0 2024-04-26T20:10:24.906 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:25.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 -- 172.21.15.131:0/2413338282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c0105640 msgr2=0x7fa2c0105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:25.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 --2- 172.21.15.131:0/2413338282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c0105640 0x7fa2c0105ac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009990 tx=0x7fa2ac009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 -- 172.21.15.131:0/2413338282 shutdown_connections 2024-04-26T20:10:25.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 --2- 172.21.15.131:0/2413338282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0106000 0x7fa2c0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 --2- 172.21.15.131:0/2413338282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c010b640 0x7fa2c010ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 --2- 172.21.15.131:0/2413338282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c0105640 0x7fa2c0105ac0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:25.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 -- 172.21.15.131:0/2413338282 >> 172.21.15.131:0/2413338282 conn(0x7fa2c0101130 msgr2=0x7fa2c0103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:25.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.752+0000 7fa2c7600700 1 -- 172.21.15.131:0/2413338282 shutdown_connections 2024-04-26T20:10:25.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.753+0000 7fa2c7600700 1 -- 172.21.15.131:0/2413338282 wait complete. 2024-04-26T20:10:25.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.754+0000 7fa2c7600700 1 Processor -- start 2024-04-26T20:10:25.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.754+0000 7fa2c7600700 1 -- start start 2024-04-26T20:10:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c7600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0105640 0x7fa2c019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c7600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c0106000 0x7fa2c019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c7600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c010b640 0x7fa2c01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:25.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c7600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01158d0 con 0x7fa2c010b640 2024-04-26T20:10:25.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c7600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0115750 con 0x7fa2c0106000 2024-04-26T20:10:26.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c7600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0115a50 con 0x7fa2c0105640 2024-04-26T20:10:26.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0105640 0x7fa2c019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:26.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c5600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c010b640 0x7fa2c01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:26.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0105640 0x7fa2c019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41678/0 (socket says 172.21.15.131:41678) 2024-04-26T20:10:26.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 -- 172.21.15.131:0/3489676634 learned_addr learned my addr 172.21.15.131:0/3489676634 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:26.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2bfe00700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c0106000 0x7fa2c019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 -- 172.21.15.131:0/3489676634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c0106000 msgr2=0x7fa2c019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:26.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c0106000 0x7fa2c019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 -- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c010b640 msgr2=0x7fa2c01a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:26.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c010b640 0x7fa2c01a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c4c00700 1 -- 172.21.15.131:0/3489676634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c01a0d30 con 0x7fa2c0105640 2024-04-26T20:10:26.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.755+0000 7fa2c5600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c010b640 0x7fa2c01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:26.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.756+0000 7fa2c4c00700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0105640 0x7fa2c019bd20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400e9a0 tx=0x7fa2b400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:26.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.756+0000 7fa2bd600700 1 -- 172.21.15.131:0/3489676634 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400ca10 con 0x7fa2c0105640 2024-04-26T20:10:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.756+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c01a1020 con 0x7fa2c0105640 2024-04-26T20:10:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.756+0000 7fa2bd600700 1 -- 172.21.15.131:0/3489676634 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2b400cbb0 con 0x7fa2c0105640 2024-04-26T20:10:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.756+0000 7fa2bd600700 1 -- 172.21.15.131:0/3489676634 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4018680 con 0x7fa2c0105640 2024-04-26T20:10:26.339 INFO:teuthology.orchestra.run.smithi131.stdout:287762808895 2024-04-26T20:10:26.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.756+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c01a88a0 con 0x7fa2c0105640 2024-04-26T20:10:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.757+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c006a110 con 0x7fa2c0105640 2024-04-26T20:10:26.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.760+0000 7fa2bd600700 1 -- 172.21.15.131:0/3489676634 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa2b4010430 con 0x7fa2c0105640 2024-04-26T20:10:26.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.761+0000 7fa2bd600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2a80a3cd0 0x7fa2a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:26.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.762+0000 7fa2bd600700 1 -- 172.21.15.131:0/3489676634 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa2b40d0e10 con 0x7fa2c0105640 2024-04-26T20:10:26.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.762+0000 7fa2bfe00700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2a80a3cd0 0x7fa2a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:26.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.762+0000 7fa2bfe00700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2a80a3cd0 0x7fa2a80a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0103770 tx=0x7fa2ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:26.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:22.764+0000 7fa2bd600700 1 -- 172.21.15.131:0/3489676634 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa2b409d240 con 0x7fa2c0105640 2024-04-26T20:10:26.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.000+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa2c002e490 con 0x7fa2c0105640 2024-04-26T20:10:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.001+0000 7fa2bd600700 1 -- 172.21.15.131:0/3489676634 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa2b409cb40 con 0x7fa2c0105640 2024-04-26T20:10:26.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.004+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2a80a3cd0 msgr2=0x7fa2a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.004+0000 7fa2c7600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2a80a3cd0 0x7fa2a80a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0103770 tx=0x7fa2ac005fd0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0105640 msgr2=0x7fa2c019bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:26.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0105640 0x7fa2c019bd20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400e9a0 tx=0x7fa2b400ecb0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 shutdown_connections 2024-04-26T20:10:26.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2c010b640 0x7fa2c01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0105640 0x7fa2c019bd20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2a80a3cd0 0x7fa2a80a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 --2- 172.21.15.131:0/3489676634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c0106000 0x7fa2c019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:26.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.005+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 >> 172.21.15.131:0/3489676634 conn(0x7fa2c0101130 msgr2=0x7fa2c0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:26.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.006+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 shutdown_connections 2024-04-26T20:10:26.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:23.006+0000 7fa2c7600700 1 -- 172.21.15.131:0/3489676634 wait complete. 2024-04-26T20:10:26.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:25 smithi131 ceph-mon[27919]: pgmap v422: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T20:10:26.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:25 smithi148 ceph-mon[32281]: pgmap v422: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T20:10:26.875 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:25 smithi157 ceph-mon[32703]: pgmap v422: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T20:10:27.647 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:27 smithi131 ceph-mon[27919]: pgmap v423: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:10:27.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:27 smithi157 ceph-mon[32703]: pgmap v423: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:10:28.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:27 smithi148 ceph-mon[32281]: pgmap v423: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:10:28.650 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808889 got 287762808895 for osd.9 2024-04-26T20:10:28.724 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.068+0000 7eff23200700 1 -- 172.21.15.131:0/1653840849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24104130 msgr2=0x7eff24104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:29.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.068+0000 7eff23200700 1 --2- 172.21.15.131:0/1653840849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24104130 0x7eff24104510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff140099c0 tx=0x7eff14009cd0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.069+0000 7eff23200700 1 -- 172.21.15.131:0/1653840849 shutdown_connections 2024-04-26T20:10:29.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.069+0000 7eff23200700 1 --2- 172.21.15.131:0/1653840849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24102610 0x7eff24112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.069+0000 7eff23200700 1 --2- 172.21.15.131:0/1653840849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24104130 0x7eff24104510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.069+0000 7eff23200700 1 --2- 172.21.15.131:0/1653840849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff24101c50 0x7eff241020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.069+0000 7eff23200700 1 -- 172.21.15.131:0/1653840849 >> 172.21.15.131:0/1653840849 conn(0x7eff240775d0 msgr2=0x7eff240779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:29.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.069+0000 7eff23200700 1 -- 172.21.15.131:0/1653840849 shutdown_connections 2024-04-26T20:10:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.069+0000 7eff23200700 1 -- 172.21.15.131:0/1653840849 wait complete. 2024-04-26T20:10:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.070+0000 7eff23200700 1 Processor -- start 2024-04-26T20:10:29.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.070+0000 7eff23200700 1 -- start start 2024-04-26T20:10:29.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff23200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24101c50 0x7eff2419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:29.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff23200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24102610 0x7eff2419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:29.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff23200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff24104130 0x7eff241a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:29.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff23200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff24115900 con 0x7eff24102610 2024-04-26T20:10:29.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff23200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff24115780 con 0x7eff24104130 2024-04-26T20:10:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff23200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff24115a80 con 0x7eff24101c50 2024-04-26T20:10:29.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff21400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24102610 0x7eff2419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff21e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24101c50 0x7eff2419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff21e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24101c50 0x7eff2419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55322/0 (socket says 172.21.15.131:55322) 2024-04-26T20:10:29.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff21e00700 1 -- 172.21.15.131:0/1961627819 learned_addr learned my addr 172.21.15.131:0/1961627819 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:29.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff22800700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff24104130 0x7eff241a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:29.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff21e00700 1 -- 172.21.15.131:0/1961627819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff24104130 msgr2=0x7eff241a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:29.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.071+0000 7eff21e00700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff24104130 0x7eff241a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff21e00700 1 -- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24102610 msgr2=0x7eff2419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:29.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff21e00700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24102610 0x7eff2419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff21e00700 1 -- 172.21.15.131:0/1961627819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff241a0db0 con 0x7eff24101c50 2024-04-26T20:10:29.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff22800700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff24104130 0x7eff241a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:10:29.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff21e00700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24101c50 0x7eff2419bdf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff14006010 tx=0x7eff14005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:29.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff1aa00700 1 -- 172.21.15.131:0/1961627819 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14026070 con 0x7eff24101c50 2024-04-26T20:10:29.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff241a1040 con 0x7eff24101c50 2024-04-26T20:10:29.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff1aa00700 1 -- 172.21.15.131:0/1961627819 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff14003c70 con 0x7eff24101c50 2024-04-26T20:10:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.072+0000 7eff1aa00700 1 -- 172.21.15.131:0/1961627819 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1400fb40 con 0x7eff24101c50 2024-04-26T20:10:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.073+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff241a8920 con 0x7eff24101c50 2024-04-26T20:10:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.074+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2406a110 con 0x7eff24101c50 2024-04-26T20:10:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.076+0000 7eff1aa00700 1 -- 172.21.15.131:0/1961627819 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7eff14029070 con 0x7eff24101c50 2024-04-26T20:10:29.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.077+0000 7eff1aa00700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7eff080a3cd0 0x7eff080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:29.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.077+0000 7eff1aa00700 1 -- 172.21.15.131:0/1961627819 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7eff140d1260 con 0x7eff24101c50 2024-04-26T20:10:29.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.078+0000 7eff21400700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7eff080a3cd0 0x7eff080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:29.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.078+0000 7eff21400700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7eff080a3cd0 0x7eff080a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7eff24077c40 tx=0x7eff10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:29.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.081+0000 7eff1aa00700 1 -- 172.21.15.131:0/1961627819 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7eff1405e060 con 0x7eff24101c50 2024-04-26T20:10:29.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.399+0000 7f0e41200700 1 -- 172.21.15.131:0/2325757172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c06bad0 msgr2=0x7f0e3c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:29.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.399+0000 7f0e41200700 1 --2- 172.21.15.131:0/2325757172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c06bad0 0x7f0e3c1102e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b0a0 tx=0x7f0e3400b3b0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.400+0000 7f0e41200700 1 -- 172.21.15.131:0/2325757172 shutdown_connections 2024-04-26T20:10:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.400+0000 7f0e41200700 1 --2- 172.21.15.131:0/2325757172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c06bad0 0x7f0e3c1102e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.400+0000 7f0e41200700 1 --2- 172.21.15.131:0/2325757172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3c1094a0 0x7f0e3c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.400+0000 7f0e41200700 1 --2- 172.21.15.131:0/2325757172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e3c06b130 0x7f0e3c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:29.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.400+0000 7f0e41200700 1 -- 172.21.15.131:0/2325757172 >> 172.21.15.131:0/2325757172 conn(0x7f0e3c0792d0 msgr2=0x7f0e3c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:30.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.400+0000 7f0e41200700 1 -- 172.21.15.131:0/2325757172 shutdown_connections 2024-04-26T20:10:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.400+0000 7f0e41200700 1 -- 172.21.15.131:0/2325757172 wait complete. 2024-04-26T20:10:30.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.401+0000 7f0e41200700 1 Processor -- start 2024-04-26T20:10:30.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.401+0000 7f0e41200700 1 -- start start 2024-04-26T20:10:30.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e41200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3c06b130 0x7f0e3c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:30.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e41200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e3c06bad0 0x7f0e3c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:30.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e41200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c1094a0 0x7f0e3c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e41200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1135f0 con 0x7f0e3c06bad0 2024-04-26T20:10:30.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e41200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c113470 con 0x7f0e3c06b130 2024-04-26T20:10:30.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e41200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c113770 con 0x7f0e3c1094a0 2024-04-26T20:10:30.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e39400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e3c06bad0 0x7f0e3c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:30.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e3a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c1094a0 0x7f0e3c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:30.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e3a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c1094a0 0x7f0e3c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55314/0 (socket says 172.21.15.131:55314) 2024-04-26T20:10:30.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e3a800700 1 -- 172.21.15.131:0/3833813807 learned_addr learned my addr 172.21.15.131:0/3833813807 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:30.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e39e00700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3c06b130 0x7f0e3c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e3a800700 1 -- 172.21.15.131:0/3833813807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3c06b130 msgr2=0x7f0e3c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e3a800700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3c06b130 0x7f0e3c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:30.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.402+0000 7f0e3a800700 1 -- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e3c06bad0 msgr2=0x7f0e3c1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:30.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e3a800700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e3c06bad0 0x7f0e3c1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:30.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e3a800700 1 -- 172.21.15.131:0/3833813807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c19c240 con 0x7f0e3c1094a0 2024-04-26T20:10:30.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e39400700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e3c06bad0 0x7f0e3c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:30.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e3a800700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c1094a0 0x7f0e3c19bae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b070 tx=0x7f0e34004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:30.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e32a00700 1 -- 172.21.15.131:0/3833813807 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34021070 con 0x7f0e3c1094a0 2024-04-26T20:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e32a00700 1 -- 172.21.15.131:0/3833813807 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e34007e30 con 0x7f0e3c1094a0 2024-04-26T20:10:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e32a00700 1 -- 172.21.15.131:0/3833813807 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340046d0 con 0x7f0e3c1094a0 2024-04-26T20:10:30.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c19c4d0 con 0x7f0e3c1094a0 2024-04-26T20:10:30.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.403+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c1a87f0 con 0x7f0e3c1094a0 2024-04-26T20:10:30.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.405+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c06a110 con 0x7f0e3c1094a0 2024-04-26T20:10:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.406+0000 7f0e32a00700 1 -- 172.21.15.131:0/3833813807 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0e34007980 con 0x7f0e3c1094a0 2024-04-26T20:10:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.407+0000 7f0e32a00700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0e200a3d50 0x7f0e200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:30.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.408+0000 7f0e32a00700 1 -- 172.21.15.131:0/3833813807 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0e340d1850 con 0x7f0e3c1094a0 2024-04-26T20:10:30.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.408+0000 7f0e39e00700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0e200a3d50 0x7f0e200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:30.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.408+0000 7f0e39e00700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0e200a3d50 0x7f0e200a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c007b60 tx=0x7f0e2c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:30.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.411+0000 7f0e32a00700 1 -- 172.21.15.131:0/3833813807 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0e3409dd10 con 0x7f0e3c1094a0 2024-04-26T20:10:30.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.652+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0e3c02e490 con 0x7f0e3c1094a0 2024-04-26T20:10:30.873 INFO:teuthology.orchestra.run.smithi131.stdout:141733920861 2024-04-26T20:10:31.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.653+0000 7f0e32a00700 1 -- 172.21.15.131:0/3833813807 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0e34004870 con 0x7f0e3c1094a0 2024-04-26T20:10:31.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0e200a3d50 msgr2=0x7f0e200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:31.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0e200a3d50 0x7f0e200a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c007b60 tx=0x7f0e2c0073d0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c1094a0 msgr2=0x7f0e3c19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:31.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c1094a0 0x7f0e3c19bae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b070 tx=0x7f0e34004cf0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 shutdown_connections 2024-04-26T20:10:31.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e3c1094a0 0x7f0e3c19bae0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0e200a3d50 0x7f0e200a6210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3c06b130 0x7f0e3c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 --2- 172.21.15.131:0/3833813807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e3c06bad0 0x7f0e3c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 >> 172.21.15.131:0/3833813807 conn(0x7f0e3c0792d0 msgr2=0x7f0e3c10e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:31.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 shutdown_connections 2024-04-26T20:10:31.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:28.656+0000 7f0e41200700 1 -- 172.21.15.131:0/3833813807 wait complete. 2024-04-26T20:10:31.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.358+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7eff2402e490 con 0x7eff24101c50 2024-04-26T20:10:31.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.359+0000 7eff1aa00700 1 -- 172.21.15.131:0/1961627819 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7eff14013070 con 0x7eff24101c50 2024-04-26T20:10:31.369 INFO:teuthology.orchestra.run.smithi131.stdout:163208757335 2024-04-26T20:10:31.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7eff080a3cd0 msgr2=0x7eff080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:31.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7eff080a3cd0 0x7eff080a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7eff24077c40 tx=0x7eff10009260 comp rx=0 tx=0).stop 2024-04-26T20:10:31.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24101c50 msgr2=0x7eff2419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24101c50 0x7eff2419bdf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff14006010 tx=0x7eff14005280 comp rx=0 tx=0).stop 2024-04-26T20:10:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 shutdown_connections 2024-04-26T20:10:31.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7eff080a3cd0 0x7eff080a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff24104130 0x7eff241a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff24101c50 0x7eff2419bdf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 --2- 172.21.15.131:0/1961627819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff24102610 0x7eff2419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 >> 172.21.15.131:0/1961627819 conn(0x7eff240775d0 msgr2=0x7eff240ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 shutdown_connections 2024-04-26T20:10:31.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:29.362+0000 7eff23200700 1 -- 172.21.15.131:0/1961627819 wait complete. 2024-04-26T20:10:31.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:29 smithi131 ceph-mon[27919]: pgmap v424: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:31.663 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:29 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3833813807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T20:10:31.669 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:29 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1961627819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T20:10:31.676 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:29 smithi148 ceph-mon[32281]: pgmap v424: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:31.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:29 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3833813807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T20:10:31.690 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:29 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1961627819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T20:10:31.773 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:29 smithi157 ceph-mon[32703]: pgmap v424: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:10:31.783 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:29 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3833813807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T20:10:31.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:29 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1961627819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T20:10:31.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.668+0000 7f42a7200700 1 -- 172.21.15.131:0/2096836493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a010b600 msgr2=0x7f42a010b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:31.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.668+0000 7f42a7200700 1 --2- 172.21.15.131:0/2096836493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a010b600 0x7f42a010b9e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f42940099c0 tx=0x7f4294009cd0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.669+0000 7f42a7200700 1 -- 172.21.15.131:0/2096836493 shutdown_connections 2024-04-26T20:10:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.669+0000 7f42a7200700 1 --2- 172.21.15.131:0/2096836493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0105f10 0x7f42a01123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.669+0000 7f42a7200700 1 --2- 172.21.15.131:0/2096836493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a010b600 0x7f42a010b9e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.669+0000 7f42a7200700 1 --2- 172.21.15.131:0/2096836493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42a0105550 0x7f42a01059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:31.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.669+0000 7f42a7200700 1 -- 172.21.15.131:0/2096836493 >> 172.21.15.131:0/2096836493 conn(0x7f42a0101000 msgr2=0x7f42a0103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:31.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.669+0000 7f42a7200700 1 -- 172.21.15.131:0/2096836493 shutdown_connections 2024-04-26T20:10:31.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.669+0000 7f42a7200700 1 -- 172.21.15.131:0/2096836493 wait complete. 2024-04-26T20:10:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 Processor -- start 2024-04-26T20:10:31.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 -- start start 2024-04-26T20:10:31.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42a0105550 0x7f42a01a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0105f10 0x7f42a01a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:31.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a010b600 0x7f42a019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:32.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42a0115720 con 0x7f42a0105f10 2024-04-26T20:10:32.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42a01155a0 con 0x7f42a0105550 2024-04-26T20:10:32.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.670+0000 7f42a7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42a01158a0 con 0x7f42a010b600 2024-04-26T20:10:32.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0105f10 0x7f42a01a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:32.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0105f10 0x7f42a01a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39166/0 (socket says 172.21.15.131:39166) 2024-04-26T20:10:32.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5400700 1 -- 172.21.15.131:0/1180018015 learned_addr learned my addr 172.21.15.131:0/1180018015 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:32.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5e00700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42a0105550 0x7f42a01a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a6800700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a010b600 0x7f42a019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5e00700 1 -- 172.21.15.131:0/1180018015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a010b600 msgr2=0x7f42a019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:32.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5e00700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a010b600 0x7f42a019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5e00700 1 -- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0105f10 msgr2=0x7f42a01a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:32.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5e00700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0105f10 0x7f42a01a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:32.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5e00700 1 -- 172.21.15.131:0/1180018015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a019c0a0 con 0x7f42a0105550 2024-04-26T20:10:32.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5400700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0105f10 0x7f42a01a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:32.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a6800700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a010b600 0x7f42a019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:10:32.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.671+0000 7f42a5e00700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42a0105550 0x7f42a01a1780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42940098e0 tx=0x7f4294004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:32.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.672+0000 7f429aa00700 1 -- 172.21.15.131:0/1180018015 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294026070 con 0x7f42a0105550 2024-04-26T20:10:32.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.672+0000 7f429aa00700 1 -- 172.21.15.131:0/1180018015 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4294003c70 con 0x7f42a0105550 2024-04-26T20:10:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.672+0000 7f429aa00700 1 -- 172.21.15.131:0/1180018015 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429400faa0 con 0x7f42a0105550 2024-04-26T20:10:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.672+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a019c330 con 0x7f42a0105550 2024-04-26T20:10:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.672+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a01a8650 con 0x7f42a0105550 2024-04-26T20:10:32.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.673+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a010da90 con 0x7f42a0105550 2024-04-26T20:10:32.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.675+0000 7f429aa00700 1 -- 172.21.15.131:0/1180018015 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4294029070 con 0x7f42a0105550 2024-04-26T20:10:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.676+0000 7f429aa00700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f42880a3cd0 0x7f42880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:32.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.676+0000 7f429aa00700 1 -- 172.21.15.131:0/1180018015 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f42940d1160 con 0x7f42a0105550 2024-04-26T20:10:32.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.676+0000 7f42a5400700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f42880a3cd0 0x7f42880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.677+0000 7f42a5400700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f42880a3cd0 0x7f42880a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f42a010abc0 tx=0x7f4290009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.680+0000 7f429aa00700 1 -- 172.21.15.131:0/1180018015 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f429405e060 con 0x7f42a0105550 2024-04-26T20:10:32.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.922+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f42a019ce70 con 0x7f42a0105550 2024-04-26T20:10:32.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.922+0000 7f429aa00700 1 -- 172.21.15.131:0/1180018015 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4294013070 con 0x7f42a0105550 2024-04-26T20:10:32.586 INFO:teuthology.orchestra.run.smithi131.stdout:219043332171 2024-04-26T20:10:32.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f42880a3cd0 msgr2=0x7f42880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:32.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f42880a3cd0 0x7f42880a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f42a010abc0 tx=0x7f4290009260 comp rx=0 tx=0).stop 2024-04-26T20:10:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42a0105550 msgr2=0x7f42a01a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42a0105550 0x7f42a01a1780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42940098e0 tx=0x7f4294004270 comp rx=0 tx=0).stop 2024-04-26T20:10:32.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 shutdown_connections 2024-04-26T20:10:32.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a010b600 0x7f42a019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f42880a3cd0 0x7f42880a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:32.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42a0105550 0x7f42a01a1780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:32.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 --2- 172.21.15.131:0/1180018015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0105f10 0x7f42a01a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.925+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 >> 172.21.15.131:0/1180018015 conn(0x7f42a0101000 msgr2=0x7f42a0102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:32.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.926+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 shutdown_connections 2024-04-26T20:10:32.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:30.926+0000 7f42a7200700 1 -- 172.21.15.131:0/1180018015 wait complete. 2024-04-26T20:10:32.894 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: pgmap v425: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:32.959 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:10:32.966 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1180018015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T20:10:32.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3570487799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T20:10:32.980 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:32.986 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:32.991 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:33.071 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:31 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:33.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: pgmap v425: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:33.209 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:10:33.215 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1180018015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T20:10:33.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3570487799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T20:10:33.256 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:33.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:33.395 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:33.402 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:31 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:33.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.049+0000 7f571b800700 1 -- 172.21.15.131:0/3374944391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 msgr2=0x7f57141140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:33.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.049+0000 7f571b800700 1 --2- 172.21.15.131:0/3374944391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141140b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5700009990 tx=0x7f5700009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:33.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.050+0000 7f571b800700 1 -- 172.21.15.131:0/3374944391 shutdown_connections 2024-04-26T20:10:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.050+0000 7f571b800700 1 --2- 172.21.15.131:0/3374944391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57141145f0 0x7f57141169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.050+0000 7f571b800700 1 --2- 172.21.15.131:0/3374944391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5714075650 0x7f5714075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:33.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.050+0000 7f571b800700 1 --2- 172.21.15.131:0/3374944391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141140b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:33.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.050+0000 7f571b800700 1 -- 172.21.15.131:0/3374944391 >> 172.21.15.131:0/3374944391 conn(0x7f571407a5d0 msgr2=0x7f571407a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:33.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.050+0000 7f571b800700 1 -- 172.21.15.131:0/3374944391 shutdown_connections 2024-04-26T20:10:33.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.050+0000 7f571b800700 1 -- 172.21.15.131:0/3374944391 wait complete. 2024-04-26T20:10:33.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.051+0000 7f571b800700 1 Processor -- start 2024-04-26T20:10:33.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.051+0000 7f571b800700 1 -- start start 2024-04-26T20:10:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f571b800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5714075650 0x7f57141a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:33.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f571b800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:33.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f571b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57141145f0 0x7f57141a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:33.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f571b800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5714119c80 con 0x7f5714075650 2024-04-26T20:10:33.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f571b800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5714119b00 con 0x7f5714075f70 2024-04-26T20:10:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f571b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5714119e00 con 0x7f57141145f0 2024-04-26T20:10:33.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f5713e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:33.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f5713e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:37056/0 (socket says 172.21.15.131:37056) 2024-04-26T20:10:33.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f5713e00700 1 -- 172.21.15.131:0/3570487799 learned_addr learned my addr 172.21.15.131:0/3570487799 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f5719800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57141145f0 0x7f57141a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:33.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.052+0000 7f5718e00700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5714075650 0x7f57141a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:33.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5713e00700 1 -- 172.21.15.131:0/3570487799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57141145f0 msgr2=0x7f57141a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:33.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5713e00700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57141145f0 0x7f57141a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:33.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5713e00700 1 -- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5714075650 msgr2=0x7f57141a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5713e00700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5714075650 0x7f57141a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:33.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5713e00700 1 -- 172.21.15.131:0/3570487799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57141a5140 con 0x7f5714075f70 2024-04-26T20:10:33.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5718e00700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5714075650 0x7f57141a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:10:34.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5719800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57141145f0 0x7f57141a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:34.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.053+0000 7f5713e00700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141a0670 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5700000c00 tx=0x7f57000057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:34.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.054+0000 7f5711600700 1 -- 172.21.15.131:0/3570487799 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700027070 con 0x7f5714075f70 2024-04-26T20:10:34.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.054+0000 7f5711600700 1 -- 172.21.15.131:0/3570487799 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5700022470 con 0x7f5714075f70 2024-04-26T20:10:34.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.054+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57141a53d0 con 0x7f5714075f70 2024-04-26T20:10:34.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.054+0000 7f5711600700 1 -- 172.21.15.131:0/3570487799 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700017650 con 0x7f5714075f70 2024-04-26T20:10:34.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.054+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141accb0 con 0x7f5714075f70 2024-04-26T20:10:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.055+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571406a110 con 0x7f5714075f70 2024-04-26T20:10:34.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.057+0000 7f5711600700 1 -- 172.21.15.131:0/3570487799 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f570002a070 con 0x7f5714075f70 2024-04-26T20:10:34.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.059+0000 7f5711600700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f56fc0a3d50 0x7f56fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:34.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.060+0000 7f5711600700 1 -- 172.21.15.131:0/3570487799 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f57000d1700 con 0x7f5714075f70 2024-04-26T20:10:34.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.060+0000 7f5718e00700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f56fc0a3d50 0x7f56fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:34.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.060+0000 7f5718e00700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f56fc0a3d50 0x7f56fc0a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5708009cb0 tx=0x7f5708009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:34.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.061+0000 7f5711600700 1 -- 172.21.15.131:0/3570487799 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5700013070 con 0x7f5714075f70 2024-04-26T20:10:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.294+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f57141ace80 con 0x7f5714075f70 2024-04-26T20:10:34.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.294+0000 7f5711600700 1 -- 172.21.15.131:0/3570487799 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f570009dbc0 con 0x7f5714075f70 2024-04-26T20:10:34.442 INFO:teuthology.orchestra.run.smithi131.stdout:55834574957 2024-04-26T20:10:34.452 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:10:34.462 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:10:34.551 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:34.560 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:32 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3703642351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T20:10:34.565 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:33 smithi148 ceph-mon[32281]: pgmap v426: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:34.572 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:10:34.580 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:10:34.637 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:34.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:32 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3703642351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T20:10:34.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:33 smithi157 ceph-mon[32703]: pgmap v426: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:34.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f56fc0a3d50 msgr2=0x7f56fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:34.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f56fc0a3d50 0x7f56fc0a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5708009cb0 tx=0x7f5708009260 comp rx=0 tx=0).stop 2024-04-26T20:10:34.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 msgr2=0x7f57141a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:34.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141a0670 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5700000c00 tx=0x7f57000057c0 comp rx=0 tx=0).stop 2024-04-26T20:10:34.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 shutdown_connections 2024-04-26T20:10:34.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57141145f0 0x7f57141a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5714075650 0x7f57141a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f56fc0a3d50 0x7f56fc0a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:34.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 --2- 172.21.15.131:0/3570487799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5714075f70 0x7f57141a0670 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.297+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 >> 172.21.15.131:0/3570487799 conn(0x7f571407a5d0 msgr2=0x7f5714110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:34.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.298+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 shutdown_connections 2024-04-26T20:10:34.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:31.298+0000 7f571b800700 1 -- 172.21.15.131:0/3570487799 wait complete. 2024-04-26T20:10:35.007 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920854 got 141733920861 for osd.4 2024-04-26T20:10:35.014 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:35.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757328 got 163208757335 for osd.5 2024-04-26T20:10:35.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: pgmap v425: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:35.185 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:10:35.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1180018015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T20:10:35.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3570487799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T20:10:35.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:35.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:35.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:35.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:31 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:35.427 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:10:35.433 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:10:35.444 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:10:35.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:32 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3703642351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T20:10:35.584 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:35.596 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574950 got 55834574957 for osd.1 2024-04-26T20:10:35.609 INFO:teuthology.orchestra.run.smithi131.stdout:253403070534 2024-04-26T20:10:35.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 -- 172.21.15.131:0/3941437575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918106020 msgr2=0x7fd918112530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3941437575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918106020 0x7fd918112530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b0a0 tx=0x7fd91000b3b0 comp rx=0 tx=0).stop 2024-04-26T20:10:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 -- 172.21.15.131:0/3941437575 shutdown_connections 2024-04-26T20:10:35.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3941437575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918106020 0x7fd918112530 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3941437575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd91810b660 0x7fd91810ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3941437575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd918105660 0x7fd918105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 -- 172.21.15.131:0/3941437575 >> 172.21.15.131:0/3941437575 conn(0x7fd918101150 msgr2=0x7fd918103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:35.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 -- 172.21.15.131:0/3941437575 shutdown_connections 2024-04-26T20:10:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.024+0000 7fd91fe00700 1 -- 172.21.15.131:0/3941437575 wait complete. 2024-04-26T20:10:35.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.025+0000 7fd91fe00700 1 Processor -- start 2024-04-26T20:10:35.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.025+0000 7fd91fe00700 1 -- start start 2024-04-26T20:10:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918105660 0x7fd9181a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:35.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd918106020 0x7fd9181a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:36.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91810b660 0x7fd91819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181158f0 con 0x7fd918105660 2024-04-26T20:10:36.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91fe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd918115770 con 0x7fd91810b660 2024-04-26T20:10:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd918115a70 con 0x7fd918106020 2024-04-26T20:10:36.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918105660 0x7fd9181a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:36.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd918106020 0x7fd9181a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:36.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91de00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91810b660 0x7fd91819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:36.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918105660 0x7fd9181a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39198/0 (socket says 172.21.15.131:39198) 2024-04-26T20:10:36.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.026+0000 7fd91d400700 1 -- 172.21.15.131:0/3703642351 learned_addr learned my addr 172.21.15.131:0/3703642351 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:36.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91ca00700 1 -- 172.21.15.131:0/3703642351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91810b660 msgr2=0x7fd91819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:36.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91ca00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91810b660 0x7fd91819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91ca00700 1 -- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918105660 msgr2=0x7fd9181a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:36.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91ca00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918105660 0x7fd9181a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:36.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91ca00700 1 -- 172.21.15.131:0/3703642351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91819c450 con 0x7fd918106020 2024-04-26T20:10:36.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91de00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91810b660 0x7fd91819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91d400700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918105660 0x7fd9181a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91ca00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd918106020 0x7fd9181a1f90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e9f0 tx=0x7fd90400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:36.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd916000700 1 -- 172.21.15.131:0/3703642351 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90400c9c0 con 0x7fd918106020 2024-04-26T20:10:36.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.027+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91819c740 con 0x7fd918106020 2024-04-26T20:10:36.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.028+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9181a89d0 con 0x7fd918106020 2024-04-26T20:10:36.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.028+0000 7fd916000700 1 -- 172.21.15.131:0/3703642351 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd90400cb60 con 0x7fd918106020 2024-04-26T20:10:36.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.028+0000 7fd916000700 1 -- 172.21.15.131:0/3703642351 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904018420 con 0x7fd918106020 2024-04-26T20:10:36.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.029+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd918106da0 con 0x7fd918106020 2024-04-26T20:10:36.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.031+0000 7fd916000700 1 -- 172.21.15.131:0/3703642351 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd9040185c0 con 0x7fd918106020 2024-04-26T20:10:36.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.032+0000 7fd916000700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd9000a3cd0 0x7fd9000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.032+0000 7fd916000700 1 -- 172.21.15.131:0/3703642351 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd9040d0b60 con 0x7fd918106020 2024-04-26T20:10:36.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.032+0000 7fd91d400700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd9000a3cd0 0x7fd9000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:36.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.033+0000 7fd91d400700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd9000a3cd0 0x7fd9000a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd90c011550 tx=0x7fd90c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:36.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.036+0000 7fd916000700 1 -- 172.21.15.131:0/3703642351 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd90409cf90 con 0x7fd918106020 2024-04-26T20:10:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.295+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd91819cfe0 con 0x7fd918106020 2024-04-26T20:10:36.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.296+0000 7fd916000700 1 -- 172.21.15.131:0/3703642351 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd904085e20 con 0x7fd918106020 2024-04-26T20:10:36.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd9000a3cd0 msgr2=0x7fd9000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd9000a3cd0 0x7fd9000a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd90c011550 tx=0x7fd90c009260 comp rx=0 tx=0).stop 2024-04-26T20:10:36.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd918106020 msgr2=0x7fd9181a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:36.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd918106020 0x7fd9181a1f90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e9f0 tx=0x7fd90400ed00 comp rx=0 tx=0).stop 2024-04-26T20:10:36.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 shutdown_connections 2024-04-26T20:10:36.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd9000a3cd0 0x7fd9000a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:36.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91810b660 0x7fd91819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:36.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd918105660 0x7fd9181a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:36.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 --2- 172.21.15.131:0/3703642351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd918106020 0x7fd9181a1f90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.298+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 >> 172.21.15.131:0/3703642351 conn(0x7fd918101150 msgr2=0x7fd918101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.299+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 shutdown_connections 2024-04-26T20:10:36.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:32.299+0000 7fd91fe00700 1 -- 172.21.15.131:0/3703642351 wait complete. 2024-04-26T20:10:36.889 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:36.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:33 smithi131 ceph-mon[27919]: pgmap v426: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:36.902 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332165 got 219043332171 for osd.7 2024-04-26T20:10:36.911 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:37.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:35 smithi148 ceph-mon[32281]: pgmap v427: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:37.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:35 smithi157 ceph-mon[32703]: pgmap v427: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:37.068 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070528 got 253403070534 for osd.8 2024-04-26T20:10:37.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:35 smithi131 ceph-mon[27919]: pgmap v427: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:37.088 DEBUG:teuthology.parallel:result is None 2024-04-26T20:10:37.100 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T20:10:37.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T20:10:37.610 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:37.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:37 smithi131 ceph-mon[27919]: pgmap v428: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:37.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:37 smithi157 ceph-mon[32703]: pgmap v428: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:38.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:37 smithi148 ceph-mon[32281]: pgmap v428: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:39.754 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:39 smithi131 ceph-mon[27919]: pgmap v429: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:39.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 -- 172.21.15.131:0/83394144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 msgr2=0x7f4760116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:39.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 --2- 172.21.15.131:0/83394144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f4760116920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f475800b0a0 tx=0x7f475800b3b0 comp rx=0 tx=0).stop 2024-04-26T20:10:39.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 -- 172.21.15.131:0/83394144 shutdown_connections 2024-04-26T20:10:39.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 --2- 172.21.15.131:0/83394144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f4760116920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 --2- 172.21.15.131:0/83394144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f476010fa50 0x7f476010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 --2- 172.21.15.131:0/83394144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4760109a50 0x7f4760109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:40.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 -- 172.21.15.131:0/83394144 >> 172.21.15.131:0/83394144 conn(0x7f4760076bf0 msgr2=0x7f4760108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 -- 172.21.15.131:0/83394144 shutdown_connections 2024-04-26T20:10:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.926+0000 7f4766800700 1 -- 172.21.15.131:0/83394144 wait complete. 2024-04-26T20:10:40.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:39 smithi148 ceph-mon[32281]: pgmap v429: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:40.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:39 smithi157 ceph-mon[32703]: pgmap v429: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:40.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.928+0000 7f4766800700 1 Processor -- start 2024-04-26T20:10:40.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.928+0000 7f4766800700 1 -- start start 2024-04-26T20:10:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.928+0000 7f4766800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4760109a50 0x7f47601a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.928+0000 7f4766800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f47601a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.928+0000 7f4766800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476010fa50 0x7f47601a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:40.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f4766800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4760119ce0 con 0x7f4760109a50 2024-04-26T20:10:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f4766800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4760119b60 con 0x7f476010a410 2024-04-26T20:10:40.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f4766800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4760119e60 con 0x7f476010fa50 2024-04-26T20:10:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f47601a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:40.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f47601a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:33078/0 (socket says 172.21.15.131:33078) 2024-04-26T20:10:40.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 -- 172.21.15.131:0/1052794791 learned_addr learned my addr 172.21.15.131:0/1052794791 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:40.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475f400700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4760109a50 0x7f47601a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:40.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475fe00700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476010fa50 0x7f47601a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:40.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 -- 172.21.15.131:0/1052794791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476010fa50 msgr2=0x7f47601a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:40.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476010fa50 0x7f47601a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 -- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4760109a50 msgr2=0x7f47601a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4760109a50 0x7f47601a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:40.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 -- 172.21.15.131:0/1052794791 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47601a50e0 con 0x7f476010a410 2024-04-26T20:10:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.929+0000 7f475ea00700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f47601a0610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9a0 tx=0x7f474c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:40.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.930+0000 7f4757e00700 1 -- 172.21.15.131:0/1052794791 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00ca10 con 0x7f476010a410 2024-04-26T20:10:40.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.930+0000 7f4757e00700 1 -- 172.21.15.131:0/1052794791 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c00cbb0 con 0x7f476010a410 2024-04-26T20:10:40.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.930+0000 7f4757e00700 1 -- 172.21.15.131:0/1052794791 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c018680 con 0x7f476010a410 2024-04-26T20:10:40.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.930+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47601a53d0 con 0x7f476010a410 2024-04-26T20:10:40.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.930+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47601acc50 con 0x7f476010a410 2024-04-26T20:10:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.931+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476006a110 con 0x7f476010a410 2024-04-26T20:10:40.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.934+0000 7f4757e00700 1 -- 172.21.15.131:0/1052794791 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f474c010430 con 0x7f476010a410 2024-04-26T20:10:40.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.935+0000 7f4757e00700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47440a3d50 0x7f47440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:40.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.935+0000 7f4757e00700 1 -- 172.21.15.131:0/1052794791 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f474c0d0db0 con 0x7f476010a410 2024-04-26T20:10:41.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.935+0000 7f475f400700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47440a3d50 0x7f47440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:41.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.936+0000 7f475f400700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47440a3d50 0x7f47440a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4750009be0 tx=0x7f4750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:41.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:39.936+0000 7f4757e00700 1 -- 172.21.15.131:0/1052794791 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f474c09d1e0 con 0x7f476010a410 2024-04-26T20:10:41.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.159+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47601a1430 con 0x7f47440a3d50 2024-04-26T20:10:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.177+0000 7f4757e00700 1 -- 172.21.15.131:0/1052794791 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f47601a1430 con 0x7f47440a3d50 2024-04-26T20:10:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:10:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47440a3d50 msgr2=0x7f47440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47440a3d50 0x7f47440a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4750009be0 tx=0x7f4750009260 comp rx=0 tx=0).stop 2024-04-26T20:10:41.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 msgr2=0x7f47601a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f47601a0610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9a0 tx=0x7f474c00ecb0 comp rx=0 tx=0).stop 2024-04-26T20:10:41.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 shutdown_connections 2024-04-26T20:10:41.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476010fa50 0x7f47601a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4760109a50 0x7f47601a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:41.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47440a3d50 0x7f47440a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:41.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 --2- 172.21.15.131:0/1052794791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f476010a410 0x7f47601a0610 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:41.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 >> 172.21.15.131:0/1052794791 conn(0x7f4760076bf0 msgr2=0x7f4760077170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:41.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 shutdown_connections 2024-04-26T20:10:41.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:40.185+0000 7f4766800700 1 -- 172.21.15.131:0/1052794791 wait complete. 2024-04-26T20:10:41.451 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-26T20:10:41.458 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-04-26T20:10:39.672830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650718,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":783,"num_read_kb":502,"num_write":350,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368488,"kb_used_data":15128,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573336,"statfs":{"total":1151940427776,"available":1151563096064,"internally_reserved":0,"allocated":15491072,"data_stored":8578892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005524"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789445+0000","last_change":"2024-04-26T20:07:34.771977+0000","last_active":"2024-04-26T20:09:15.789445+0000","last_peered":"2024-04-26T20:09:15.789445+0000","last_clean":"2024-04-26T20:09:15.789445+0000","last_became_active":"2024-04-26T20:07:34.771658+0000","last_became_peered":"2024-04-26T20:07:34.771658+0000","last_unstale":"2024-04-26T20:09:15.789445+0000","last_undegraded":"2024-04-26T20:09:15.789445+0000","last_fullsized":"2024-04-26T20:09:15.789445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:38.419057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.800449+0000","last_change":"2024-04-26T20:07:34.768774+0000","last_active":"2024-04-26T20:09:15.800449+0000","last_peered":"2024-04-26T20:09:15.800449+0000","last_clean":"2024-04-26T20:09:15.800449+0000","last_became_active":"2024-04-26T20:07:34.768383+0000","last_became_peered":"2024-04-26T20:07:34.768383+0000","last_unstale":"2024-04-26T20:09:15.800449+0000","last_undegraded":"2024-04-26T20:09:15.800449+0000","last_fullsized":"2024-04-26T20:09:15.800449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:43.813261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.798842+0000","last_change":"2024-04-26T20:07:34.761796+0000","last_active":"2024-04-26T20:09:15.798842+0000","last_peered":"2024-04-26T20:09:15.798842+0000","last_clean":"2024-04-26T20:09:15.798842+0000","last_became_active":"2024-04-26T20:07:34.761403+0000","last_became_peered":"2024-04-26T20:07:34.761403+0000","last_unstale":"2024-04-26T20:09:15.798842+0000","last_undegraded":"2024-04-26T20:09:15.798842+0000","last_fullsized":"2024-04-26T20:09:15.798842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:06.832304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087419+0000","last_change":"2024-04-26T20:07:34.756728+0000","last_active":"2024-04-26T20:09:16.087419+0000","last_peered":"2024-04-26T20:09:16.087419+0000","last_clean":"2024-04-26T20:09:16.087419+0000","last_became_active":"2024-04-26T20:07:34.756406+0000","last_became_peered":"2024-04-26T20:07:34.756406+0000","last_unstale":"2024-04-26T20:09:16.087419+0000","last_undegraded":"2024-04-26T20:09:16.087419+0000","last_fullsized":"2024-04-26T20:09:16.087419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:27.753674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364503+0000","last_change":"2024-04-26T20:07:34.758243+0000","last_active":"2024-04-26T20:09:16.364503+0000","last_peered":"2024-04-26T20:09:16.364503+0000","last_clean":"2024-04-26T20:09:16.364503+0000","last_became_active":"2024-04-26T20:07:34.757868+0000","last_became_peered":"2024-04-26T20:07:34.757868+0000","last_unstale":"2024-04-26T20:09:16.364503+0000","last_undegraded":"2024-04-26T20:09:16.364503+0000","last_fullsized":"2024-04-26T20:09:16.364503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:53.361835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786953+0000","last_change":"2024-04-26T20:08:15.585311+0000","last_active":"2024-04-26T20:09:15.786953+0000","last_peered":"2024-04-26T20:09:15.786953+0000","last_clean":"2024-04-26T20:09:15.786953+0000","last_became_active":"2024-04-26T20:08:15.584982+0000","last_became_peered":"2024-04-26T20:08:15.584982+0000","last_unstale":"2024-04-26T20:09:15.786953+0000","last_undegraded":"2024-04-26T20:09:15.786953+0000","last_fullsized":"2024-04-26T20:09:15.786953+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:56.828218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863732+0000","last_change":"2024-04-26T20:07:34.757529+0000","last_active":"2024-04-26T20:09:15.863732+0000","last_peered":"2024-04-26T20:09:15.863732+0000","last_clean":"2024-04-26T20:09:15.863732+0000","last_became_active":"2024-04-26T20:07:34.757166+0000","last_became_peered":"2024-04-26T20:07:34.757166+0000","last_unstale":"2024-04-26T20:09:15.863732+0000","last_undegraded":"2024-04-26T20:09:15.863732+0000","last_fullsized":"2024-04-26T20:09:15.863732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:01.309698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788256+0000","last_change":"2024-04-26T20:07:34.765352+0000","last_active":"2024-04-26T20:09:15.788256+0000","last_peered":"2024-04-26T20:09:15.788256+0000","last_clean":"2024-04-26T20:09:15.788256+0000","last_became_active":"2024-04-26T20:07:34.758400+0000","last_became_peered":"2024-04-26T20:07:34.758400+0000","last_unstale":"2024-04-26T20:09:15.788256+0000","last_undegraded":"2024-04-26T20:09:15.788256+0000","last_fullsized":"2024-04-26T20:09:15.788256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:43.940746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.267191+0000","last_change":"2024-04-26T20:07:34.754413+0000","last_active":"2024-04-26T20:09:16.267191+0000","last_peered":"2024-04-26T20:09:16.267191+0000","last_clean":"2024-04-26T20:09:16.267191+0000","last_became_active":"2024-04-26T20:07:34.754073+0000","last_became_peered":"2024-04-26T20:07:34.754073+0000","last_unstale":"2024-04-26T20:09:16.267191+0000","last_undegraded":"2024-04-26T20:09:16.267191+0000","last_fullsized":"2024-04-26T20:09:16.267191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:25.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787376+0000","last_change":"2024-04-26T20:07:34.748739+0000","last_active":"2024-04-26T20:09:15.787376+0000","last_peered":"2024-04-26T20:09:15.787376+0000","last_clean":"2024-04-26T20:09:15.787376+0000","last_became_active":"2024-04-26T20:07:34.748378+0000","last_became_peered":"2024-04-26T20:07:34.748378+0000","last_unstale":"2024-04-26T20:09:15.787376+0000","last_undegraded":"2024-04-26T20:09:15.787376+0000","last_fullsized":"2024-04-26T20:09:15.787376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:13.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864290+0000","last_change":"2024-04-26T20:07:34.747402+0000","last_active":"2024-04-26T20:09:15.864290+0000","last_peered":"2024-04-26T20:09:15.864290+0000","last_clean":"2024-04-26T20:09:15.864290+0000","last_became_active":"2024-04-26T20:07:34.746965+0000","last_became_peered":"2024-04-26T20:07:34.746965+0000","last_unstale":"2024-04-26T20:09:15.864290+0000","last_undegraded":"2024-04-26T20:09:15.864290+0000","last_fullsized":"2024-04-26T20:09:15.864290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:32.066286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.366157+0000","last_change":"2024-04-26T20:09:16.366157+0000","last_active":"2024-04-26T20:09:16.366157+0000","last_peered":"2024-04-26T20:09:16.366157+0000","last_clean":"2024-04-26T20:09:16.366157+0000","last_became_active":"2024-04-26T20:09:16.365132+0000","last_became_peered":"2024-04-26T20:09:16.365132+0000","last_unstale":"2024-04-26T20:09:16.366157+0000","last_undegraded":"2024-04-26T20:09:16.366157+0000","last_fullsized":"2024-04-26T20:09:16.366157+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.559981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313915+0000","last_change":"2024-04-26T20:07:34.766542+0000","last_active":"2024-04-26T20:09:16.313915+0000","last_peered":"2024-04-26T20:09:16.313915+0000","last_clean":"2024-04-26T20:09:16.313915+0000","last_became_active":"2024-04-26T20:07:34.766185+0000","last_became_peered":"2024-04-26T20:07:34.766185+0000","last_unstale":"2024-04-26T20:09:16.313915+0000","last_undegraded":"2024-04-26T20:09:16.313915+0000","last_fullsized":"2024-04-26T20:09:16.313915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:34.499145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.797698+0000","last_change":"2024-04-26T20:07:34.762610+0000","last_active":"2024-04-26T20:09:15.797698+0000","last_peered":"2024-04-26T20:09:15.797698+0000","last_clean":"2024-04-26T20:09:15.797698+0000","last_became_active":"2024-04-26T20:07:34.762100+0000","last_became_peered":"2024-04-26T20:07:34.762100+0000","last_unstale":"2024-04-26T20:09:15.797698+0000","last_undegraded":"2024-04-26T20:09:15.797698+0000","last_fullsized":"2024-04-26T20:09:15.797698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:18.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.798758+0000","last_change":"2024-04-26T20:07:34.766754+0000","last_active":"2024-04-26T20:09:15.798758+0000","last_peered":"2024-04-26T20:09:15.798758+0000","last_clean":"2024-04-26T20:09:15.798758+0000","last_became_active":"2024-04-26T20:07:34.762204+0000","last_became_peered":"2024-04-26T20:07:34.762204+0000","last_unstale":"2024-04-26T20:09:15.798758+0000","last_undegraded":"2024-04-26T20:09:15.798758+0000","last_fullsized":"2024-04-26T20:09:15.798758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:25.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266493+0000","last_change":"2024-04-26T20:07:34.769739+0000","last_active":"2024-04-26T20:09:16.266493+0000","last_peered":"2024-04-26T20:09:16.266493+0000","last_clean":"2024-04-26T20:09:16.266493+0000","last_became_active":"2024-04-26T20:07:34.769383+0000","last_became_peered":"2024-04-26T20:07:34.769383+0000","last_unstale":"2024-04-26T20:09:16.266493+0000","last_undegraded":"2024-04-26T20:09:16.266493+0000","last_fullsized":"2024-04-26T20:09:16.266493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:08.771590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789111+0000","last_change":"2024-04-26T20:07:34.772010+0000","last_active":"2024-04-26T20:09:15.789111+0000","last_peered":"2024-04-26T20:09:15.789111+0000","last_clean":"2024-04-26T20:09:15.789111+0000","last_became_active":"2024-04-26T20:07:34.771682+0000","last_became_peered":"2024-04-26T20:07:34.771682+0000","last_unstale":"2024-04-26T20:09:15.789111+0000","last_undegraded":"2024-04-26T20:09:15.789111+0000","last_fullsized":"2024-04-26T20:09:15.789111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:57.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793188+0000","last_change":"2024-04-26T20:07:34.757547+0000","last_active":"2024-04-26T20:09:15.793188+0000","last_peered":"2024-04-26T20:09:15.793188+0000","last_clean":"2024-04-26T20:09:15.793188+0000","last_became_active":"2024-04-26T20:07:34.757180+0000","last_became_peered":"2024-04-26T20:07:34.757180+0000","last_unstale":"2024-04-26T20:09:15.793188+0000","last_undegraded":"2024-04-26T20:09:15.793188+0000","last_fullsized":"2024-04-26T20:09:15.793188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:32.365944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787602+0000","last_change":"2024-04-26T20:07:34.767363+0000","last_active":"2024-04-26T20:09:15.787602+0000","last_peered":"2024-04-26T20:09:15.787602+0000","last_clean":"2024-04-26T20:09:15.787602+0000","last_became_active":"2024-04-26T20:07:34.766909+0000","last_became_peered":"2024-04-26T20:07:34.766909+0000","last_unstale":"2024-04-26T20:09:15.787602+0000","last_undegraded":"2024-04-26T20:09:15.787602+0000","last_fullsized":"2024-04-26T20:09:15.787602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:32:31.500141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787072+0000","last_change":"2024-04-26T20:07:38.823959+0000","last_active":"2024-04-26T20:09:15.787072+0000","last_peered":"2024-04-26T20:09:15.787072+0000","last_clean":"2024-04-26T20:09:15.787072+0000","last_became_active":"2024-04-26T20:07:38.823524+0000","last_became_peered":"2024-04-26T20:07:38.823524+0000","last_unstale":"2024-04-26T20:09:15.787072+0000","last_undegraded":"2024-04-26T20:09:15.787072+0000","last_fullsized":"2024-04-26T20:09:15.787072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:23.348376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'9","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793563+0000","last_change":"2024-04-26T20:07:31.721324+0000","last_active":"2024-04-26T20:09:15.793563+0000","last_peered":"2024-04-26T20:09:15.793563+0000","last_clean":"2024-04-26T20:09:15.793563+0000","last_became_active":"2024-04-26T20:07:31.720140+0000","last_became_peered":"2024-04-26T20:07:31.720140+0000","last_unstale":"2024-04-26T20:09:15.793563+0000","last_undegraded":"2024-04-26T20:09:15.793563+0000","last_fullsized":"2024-04-26T20:09:15.793563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:27.636660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789819+0000","last_change":"2024-04-26T20:07:34.760387+0000","last_active":"2024-04-26T20:09:15.789819+0000","last_peered":"2024-04-26T20:09:15.789819+0000","last_clean":"2024-04-26T20:09:15.789819+0000","last_became_active":"2024-04-26T20:07:34.760042+0000","last_became_peered":"2024-04-26T20:07:34.760042+0000","last_unstale":"2024-04-26T20:09:15.789819+0000","last_undegraded":"2024-04-26T20:09:15.789819+0000","last_fullsized":"2024-04-26T20:09:15.789819+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:39.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.086859+0000","last_change":"2024-04-26T20:08:15.587181+0000","last_active":"2024-04-26T20:09:16.086859+0000","last_peered":"2024-04-26T20:09:16.086859+0000","last_clean":"2024-04-26T20:09:16.086859+0000","last_became_active":"2024-04-26T20:08:15.586847+0000","last_became_peered":"2024-04-26T20:08:15.586847+0000","last_unstale":"2024-04-26T20:09:16.086859+0000","last_undegraded":"2024-04-26T20:09:16.086859+0000","last_fullsized":"2024-04-26T20:09:16.086859+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:12.495523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.790501+0000","last_change":"2024-04-26T20:07:31.711623+0000","last_active":"2024-04-26T20:09:15.790501+0000","last_peered":"2024-04-26T20:09:15.790501+0000","last_clean":"2024-04-26T20:09:15.790501+0000","last_became_active":"2024-04-26T20:07:31.710488+0000","last_became_peered":"2024-04-26T20:07:31.710488+0000","last_unstale":"2024-04-26T20:09:15.790501+0000","last_undegraded":"2024-04-26T20:09:15.790501+0000","last_fullsized":"2024-04-26T20:09:15.790501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:04.747170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.802365+0000","last_change":"2024-04-26T20:09:15.801039+0000","last_active":"2024-04-26T20:09:15.802365+0000","last_peered":"2024-04-26T20:09:15.802365+0000","last_clean":"2024-04-26T20:09:15.802365+0000","last_became_active":"2024-04-26T20:09:15.800049+0000","last_became_peered":"2024-04-26T20:09:15.800049+0000","last_unstale":"2024-04-26T20:09:15.802365+0000","last_undegraded":"2024-04-26T20:09:15.802365+0000","last_fullsized":"2024-04-26T20:09:15.802365+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:49.168625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,9],"acting":[0,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087028+0000","last_change":"2024-04-26T20:07:38.822443+0000","last_active":"2024-04-26T20:09:16.087028+0000","last_peered":"2024-04-26T20:09:16.087028+0000","last_clean":"2024-04-26T20:09:16.087028+0000","last_became_active":"2024-04-26T20:07:38.822046+0000","last_became_peered":"2024-04-26T20:07:38.822046+0000","last_unstale":"2024-04-26T20:09:16.087028+0000","last_undegraded":"2024-04-26T20:09:16.087028+0000","last_fullsized":"2024-04-26T20:09:16.087028+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:26.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364978+0000","last_change":"2024-04-26T20:07:31.715319+0000","last_active":"2024-04-26T20:09:16.364978+0000","last_peered":"2024-04-26T20:09:16.364978+0000","last_clean":"2024-04-26T20:09:16.364978+0000","last_became_active":"2024-04-26T20:07:31.714915+0000","last_became_peered":"2024-04-26T20:07:31.714915+0000","last_unstale":"2024-04-26T20:09:16.364978+0000","last_undegraded":"2024-04-26T20:09:16.364978+0000","last_fullsized":"2024-04-26T20:09:16.364978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:27.898652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266828+0000","last_change":"2024-04-26T20:07:34.753422+0000","last_active":"2024-04-26T20:09:16.266828+0000","last_peered":"2024-04-26T20:09:16.266828+0000","last_clean":"2024-04-26T20:09:16.266828+0000","last_became_active":"2024-04-26T20:07:34.753009+0000","last_became_peered":"2024-04-26T20:07:34.753009+0000","last_unstale":"2024-04-26T20:09:16.266828+0000","last_undegraded":"2024-04-26T20:09:16.266828+0000","last_fullsized":"2024-04-26T20:09:16.266828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.369715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.086930+0000","last_change":"2024-04-26T20:07:38.830239+0000","last_active":"2024-04-26T20:09:16.086930+0000","last_peered":"2024-04-26T20:09:16.086930+0000","last_clean":"2024-04-26T20:09:16.086930+0000","last_became_active":"2024-04-26T20:07:38.829509+0000","last_became_peered":"2024-04-26T20:07:38.829509+0000","last_unstale":"2024-04-26T20:09:16.086930+0000","last_undegraded":"2024-04-26T20:09:16.086930+0000","last_fullsized":"2024-04-26T20:09:16.086930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:53.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788780+0000","last_change":"2024-04-26T20:08:14.576129+0000","last_active":"2024-04-26T20:09:15.788780+0000","last_peered":"2024-04-26T20:09:15.788780+0000","last_clean":"2024-04-26T20:09:15.788780+0000","last_became_active":"2024-04-26T20:08:14.573168+0000","last_became_peered":"2024-04-26T20:08:14.573168+0000","last_unstale":"2024-04-26T20:09:15.788780+0000","last_undegraded":"2024-04-26T20:09:15.788780+0000","last_fullsized":"2024-04-26T20:09:15.788780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:38.234154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176162+0000","last_change":"2024-04-26T20:07:34.749270+0000","last_active":"2024-04-26T20:09:16.176162+0000","last_peered":"2024-04-26T20:09:16.176162+0000","last_clean":"2024-04-26T20:09:16.176162+0000","last_became_active":"2024-04-26T20:07:34.748585+0000","last_became_peered":"2024-04-26T20:07:34.748585+0000","last_unstale":"2024-04-26T20:09:16.176162+0000","last_undegraded":"2024-04-26T20:09:16.176162+0000","last_fullsized":"2024-04-26T20:09:16.176162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:03.091720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787354+0000","last_change":"2024-04-26T20:07:38.806247+0000","last_active":"2024-04-26T20:09:15.787354+0000","last_peered":"2024-04-26T20:09:15.787354+0000","last_clean":"2024-04-26T20:09:15.787354+0000","last_became_active":"2024-04-26T20:07:38.805748+0000","last_became_peered":"2024-04-26T20:07:38.805748+0000","last_unstale":"2024-04-26T20:09:15.787354+0000","last_undegraded":"2024-04-26T20:09:15.787354+0000","last_fullsized":"2024-04-26T20:09:15.787354+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:16.962494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":155,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:36.406200+0000","last_change":"2024-04-26T20:07:31.713618+0000","last_active":"2024-04-26T20:10:36.406200+0000","last_peered":"2024-04-26T20:10:36.406200+0000","last_clean":"2024-04-26T20:10:36.406200+0000","last_became_active":"2024-04-26T20:07:31.713259+0000","last_became_peered":"2024-04-26T20:07:31.713259+0000","last_unstale":"2024-04-26T20:10:36.406200+0000","last_undegraded":"2024-04-26T20:10:36.406200+0000","last_fullsized":"2024-04-26T20:10:36.406200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:25.215087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364422+0000","last_change":"2024-04-26T20:08:14.585688+0000","last_active":"2024-04-26T20:09:16.364422+0000","last_peered":"2024-04-26T20:09:16.364422+0000","last_clean":"2024-04-26T20:09:16.364422+0000","last_became_active":"2024-04-26T20:08:14.585337+0000","last_became_peered":"2024-04-26T20:08:14.585337+0000","last_unstale":"2024-04-26T20:09:16.364422+0000","last_undegraded":"2024-04-26T20:09:16.364422+0000","last_fullsized":"2024-04-26T20:09:16.364422+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:03.515457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786995+0000","last_change":"2024-04-26T20:07:38.829772+0000","last_active":"2024-04-26T20:09:15.786995+0000","last_peered":"2024-04-26T20:09:15.786995+0000","last_clean":"2024-04-26T20:09:15.786995+0000","last_became_active":"2024-04-26T20:07:38.829327+0000","last_became_peered":"2024-04-26T20:07:38.829327+0000","last_unstale":"2024-04-26T20:09:15.786995+0000","last_undegraded":"2024-04-26T20:09:15.786995+0000","last_fullsized":"2024-04-26T20:09:15.786995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:37.252863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"95'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786886+0000","last_change":"2024-04-26T20:07:31.728654+0000","last_active":"2024-04-26T20:09:15.786886+0000","last_peered":"2024-04-26T20:09:15.786886+0000","last_clean":"2024-04-26T20:09:15.786886+0000","last_became_active":"2024-04-26T20:07:31.728278+0000","last_became_peered":"2024-04-26T20:07:31.728278+0000","last_unstale":"2024-04-26T20:09:15.786886+0000","last_undegraded":"2024-04-26T20:09:15.786886+0000","last_fullsized":"2024-04-26T20:09:15.786886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176199+0000","last_change":"2024-04-26T20:07:34.739575+0000","last_active":"2024-04-26T20:09:16.176199+0000","last_peered":"2024-04-26T20:09:16.176199+0000","last_clean":"2024-04-26T20:09:16.176199+0000","last_became_active":"2024-04-26T20:07:34.739264+0000","last_became_peered":"2024-04-26T20:07:34.739264+0000","last_unstale":"2024-04-26T20:09:16.176199+0000","last_undegraded":"2024-04-26T20:09:16.176199+0000","last_fullsized":"2024-04-26T20:09:16.176199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:25.130064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788948+0000","last_change":"2024-04-26T20:07:38.822669+0000","last_active":"2024-04-26T20:09:15.788948+0000","last_peered":"2024-04-26T20:09:15.788948+0000","last_clean":"2024-04-26T20:09:15.788948+0000","last_became_active":"2024-04-26T20:07:38.822197+0000","last_became_peered":"2024-04-26T20:07:38.822197+0000","last_unstale":"2024-04-26T20:09:15.788948+0000","last_undegraded":"2024-04-26T20:09:15.788948+0000","last_fullsized":"2024-04-26T20:09:15.788948+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:56.782794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"95'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313329+0000","last_change":"2024-04-26T20:07:31.723527+0000","last_active":"2024-04-26T20:09:16.313329+0000","last_peered":"2024-04-26T20:09:16.313329+0000","last_clean":"2024-04-26T20:09:16.313329+0000","last_became_active":"2024-04-26T20:07:31.722574+0000","last_became_peered":"2024-04-26T20:07:31.722574+0000","last_unstale":"2024-04-26T20:09:16.313329+0000","last_undegraded":"2024-04-26T20:09:16.313329+0000","last_fullsized":"2024-04-26T20:09:16.313329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.521613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789625+0000","last_change":"2024-04-26T20:07:34.771459+0000","last_active":"2024-04-26T20:09:15.789625+0000","last_peered":"2024-04-26T20:09:15.789625+0000","last_clean":"2024-04-26T20:09:15.789625+0000","last_became_active":"2024-04-26T20:07:34.771124+0000","last_became_peered":"2024-04-26T20:07:34.771124+0000","last_unstale":"2024-04-26T20:09:15.789625+0000","last_undegraded":"2024-04-26T20:09:15.789625+0000","last_fullsized":"2024-04-26T20:09:15.789625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.560440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.800310+0000","last_change":"2024-04-26T20:07:38.838858+0000","last_active":"2024-04-26T20:09:15.800310+0000","last_peered":"2024-04-26T20:09:15.800310+0000","last_clean":"2024-04-26T20:09:15.800310+0000","last_became_active":"2024-04-26T20:07:38.838125+0000","last_became_peered":"2024-04-26T20:07:38.838125+0000","last_unstale":"2024-04-26T20:09:15.800310+0000","last_undegraded":"2024-04-26T20:09:15.800310+0000","last_fullsized":"2024-04-26T20:09:15.800310+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:01.856633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"95'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176407+0000","last_change":"2024-04-26T20:07:31.710690+0000","last_active":"2024-04-26T20:09:16.176407+0000","last_peered":"2024-04-26T20:09:16.176407+0000","last_clean":"2024-04-26T20:09:16.176407+0000","last_became_active":"2024-04-26T20:07:31.710353+0000","last_became_peered":"2024-04-26T20:07:31.710353+0000","last_unstale":"2024-04-26T20:09:16.176407+0000","last_undegraded":"2024-04-26T20:09:16.176407+0000","last_fullsized":"2024-04-26T20:09:16.176407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:47.442673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791085+0000","last_change":"2024-04-26T20:07:34.776023+0000","last_active":"2024-04-26T20:09:15.791085+0000","last_peered":"2024-04-26T20:09:15.791085+0000","last_clean":"2024-04-26T20:09:15.791085+0000","last_became_active":"2024-04-26T20:07:34.775571+0000","last_became_peered":"2024-04-26T20:07:34.775571+0000","last_unstale":"2024-04-26T20:09:15.791085+0000","last_undegraded":"2024-04-26T20:09:15.791085+0000","last_fullsized":"2024-04-26T20:09:15.791085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:40.573046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788839+0000","last_change":"2024-04-26T20:07:38.820636+0000","last_active":"2024-04-26T20:09:15.788839+0000","last_peered":"2024-04-26T20:09:15.788839+0000","last_clean":"2024-04-26T20:09:15.788839+0000","last_became_active":"2024-04-26T20:07:38.819390+0000","last_became_peered":"2024-04-26T20:07:38.819390+0000","last_unstale":"2024-04-26T20:09:15.788839+0000","last_undegraded":"2024-04-26T20:09:15.788839+0000","last_fullsized":"2024-04-26T20:09:15.788839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:58.793479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'3","reported_seq":148,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:36.405671+0000","last_change":"2024-04-26T20:07:31.724192+0000","last_active":"2024-04-26T20:10:36.405671+0000","last_peered":"2024-04-26T20:10:36.405671+0000","last_clean":"2024-04-26T20:10:36.405671+0000","last_became_active":"2024-04-26T20:07:31.723816+0000","last_became_peered":"2024-04-26T20:07:31.723816+0000","last_unstale":"2024-04-26T20:10:36.405671+0000","last_undegraded":"2024-04-26T20:10:36.405671+0000","last_fullsized":"2024-04-26T20:10:36.405671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:31.707282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.267056+0000","last_change":"2024-04-26T20:07:34.759027+0000","last_active":"2024-04-26T20:09:16.267056+0000","last_peered":"2024-04-26T20:09:16.267056+0000","last_clean":"2024-04-26T20:09:16.267056+0000","last_became_active":"2024-04-26T20:07:34.758689+0000","last_became_peered":"2024-04-26T20:07:34.758689+0000","last_unstale":"2024-04-26T20:09:16.267056+0000","last_undegraded":"2024-04-26T20:09:16.267056+0000","last_fullsized":"2024-04-26T20:09:16.267056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:29.110222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791568+0000","last_change":"2024-04-26T20:07:38.819000+0000","last_active":"2024-04-26T20:09:15.791568+0000","last_peered":"2024-04-26T20:09:15.791568+0000","last_clean":"2024-04-26T20:09:15.791568+0000","last_became_active":"2024-04-26T20:07:38.818526+0000","last_became_peered":"2024-04-26T20:07:38.818526+0000","last_unstale":"2024-04-26T20:09:15.791568+0000","last_undegraded":"2024-04-26T20:09:15.791568+0000","last_fullsized":"2024-04-26T20:09:15.791568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:28.830096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864231+0000","last_change":"2024-04-26T20:07:31.712952+0000","last_active":"2024-04-26T20:09:15.864231+0000","last_peered":"2024-04-26T20:09:15.864231+0000","last_clean":"2024-04-26T20:09:15.864231+0000","last_became_active":"2024-04-26T20:07:31.712081+0000","last_became_peered":"2024-04-26T20:07:31.712081+0000","last_unstale":"2024-04-26T20:09:15.864231+0000","last_undegraded":"2024-04-26T20:09:15.864231+0000","last_fullsized":"2024-04-26T20:09:15.864231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:46.923536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.368893+0000","last_change":"2024-04-26T20:09:16.367690+0000","last_active":"2024-04-26T20:09:16.368893+0000","last_peered":"2024-04-26T20:09:16.368893+0000","last_clean":"2024-04-26T20:09:16.368893+0000","last_became_active":"2024-04-26T20:09:16.366496+0000","last_became_peered":"2024-04-26T20:09:16.366496+0000","last_unstale":"2024-04-26T20:09:16.368893+0000","last_undegraded":"2024-04-26T20:09:16.368893+0000","last_fullsized":"2024-04-26T20:09:16.368893+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:36.154062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087235+0000","last_change":"2024-04-26T20:07:38.822520+0000","last_active":"2024-04-26T20:09:16.087235+0000","last_peered":"2024-04-26T20:09:16.087235+0000","last_clean":"2024-04-26T20:09:16.087235+0000","last_became_active":"2024-04-26T20:07:38.822056+0000","last_became_peered":"2024-04-26T20:07:38.822056+0000","last_unstale":"2024-04-26T20:09:16.087235+0000","last_undegraded":"2024-04-26T20:09:16.087235+0000","last_fullsized":"2024-04-26T20:09:16.087235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.324996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"95'4","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176248+0000","last_change":"2024-04-26T20:07:31.724015+0000","last_active":"2024-04-26T20:09:16.176248+0000","last_peered":"2024-04-26T20:09:16.176248+0000","last_clean":"2024-04-26T20:09:16.176248+0000","last_became_active":"2024-04-26T20:07:31.723686+0000","last_became_peered":"2024-04-26T20:07:31.723686+0000","last_unstale":"2024-04-26T20:09:16.176248+0000","last_undegraded":"2024-04-26T20:09:16.176248+0000","last_fullsized":"2024-04-26T20:09:16.176248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:55.925280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266959+0000","last_change":"2024-04-26T20:07:34.750276+0000","last_active":"2024-04-26T20:09:16.266959+0000","last_peered":"2024-04-26T20:09:16.266959+0000","last_clean":"2024-04-26T20:09:16.266959+0000","last_became_active":"2024-04-26T20:07:34.749716+0000","last_became_peered":"2024-04-26T20:07:34.749716+0000","last_unstale":"2024-04-26T20:09:16.266959+0000","last_undegraded":"2024-04-26T20:09:16.266959+0000","last_fullsized":"2024-04-26T20:09:16.266959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:22.429780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791432+0000","last_change":"2024-04-26T20:07:38.815636+0000","last_active":"2024-04-26T20:09:15.791432+0000","last_peered":"2024-04-26T20:09:15.791432+0000","last_clean":"2024-04-26T20:09:15.791432+0000","last_became_active":"2024-04-26T20:07:38.815278+0000","last_became_peered":"2024-04-26T20:07:38.815278+0000","last_unstale":"2024-04-26T20:09:15.791432+0000","last_undegraded":"2024-04-26T20:09:15.791432+0000","last_fullsized":"2024-04-26T20:09:15.791432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:27.522132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'33","reported_seq":292,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:37.407081+0000","last_change":"2024-04-26T20:07:31.711393+0000","last_active":"2024-04-26T20:10:37.407081+0000","last_peered":"2024-04-26T20:10:37.407081+0000","last_clean":"2024-04-26T20:10:37.407081+0000","last_became_active":"2024-04-26T20:07:31.710998+0000","last_became_peered":"2024-04-26T20:07:31.710998+0000","last_unstale":"2024-04-26T20:10:37.407081+0000","last_undegraded":"2024-04-26T20:10:37.407081+0000","last_fullsized":"2024-04-26T20:10:37.407081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:00.674950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56115,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":219,"num_read_kb":164,"num_write":34,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364772+0000","last_change":"2024-04-26T20:07:34.752774+0000","last_active":"2024-04-26T20:09:16.364772+0000","last_peered":"2024-04-26T20:09:16.364772+0000","last_clean":"2024-04-26T20:09:16.364772+0000","last_became_active":"2024-04-26T20:07:34.752471+0000","last_became_peered":"2024-04-26T20:07:34.752471+0000","last_unstale":"2024-04-26T20:09:16.364772+0000","last_undegraded":"2024-04-26T20:09:16.364772+0000","last_fullsized":"2024-04-26T20:09:16.364772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.646231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176573+0000","last_change":"2024-04-26T20:07:38.830322+0000","last_active":"2024-04-26T20:09:16.176573+0000","last_peered":"2024-04-26T20:09:16.176573+0000","last_clean":"2024-04-26T20:09:16.176573+0000","last_became_active":"2024-04-26T20:07:38.829910+0000","last_became_peered":"2024-04-26T20:07:38.829910+0000","last_unstale":"2024-04-26T20:09:16.176573+0000","last_undegraded":"2024-04-26T20:09:16.176573+0000","last_fullsized":"2024-04-26T20:09:16.176573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:01.979793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"95'6","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787229+0000","last_change":"2024-04-26T20:07:31.712054+0000","last_active":"2024-04-26T20:09:15.787229+0000","last_peered":"2024-04-26T20:09:15.787229+0000","last_clean":"2024-04-26T20:09:15.787229+0000","last_became_active":"2024-04-26T20:07:31.711618+0000","last_became_peered":"2024-04-26T20:07:31.711618+0000","last_unstale":"2024-04-26T20:09:15.787229+0000","last_undegraded":"2024-04-26T20:09:15.787229+0000","last_fullsized":"2024-04-26T20:09:15.787229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:46.465777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313432+0000","last_change":"2024-04-26T20:07:34.772993+0000","last_active":"2024-04-26T20:09:16.313432+0000","last_peered":"2024-04-26T20:09:16.313432+0000","last_clean":"2024-04-26T20:09:16.313432+0000","last_became_active":"2024-04-26T20:07:34.772587+0000","last_became_peered":"2024-04-26T20:07:34.772587+0000","last_unstale":"2024-04-26T20:09:16.313432+0000","last_undegraded":"2024-04-26T20:09:16.313432+0000","last_fullsized":"2024-04-26T20:09:16.313432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:38.459869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788987+0000","last_change":"2024-04-26T20:08:15.588307+0000","last_active":"2024-04-26T20:09:15.788987+0000","last_peered":"2024-04-26T20:09:15.788987+0000","last_clean":"2024-04-26T20:09:15.788987+0000","last_became_active":"2024-04-26T20:08:15.587978+0000","last_became_peered":"2024-04-26T20:08:15.587978+0000","last_unstale":"2024-04-26T20:09:15.788987+0000","last_undegraded":"2024-04-26T20:09:15.788987+0000","last_fullsized":"2024-04-26T20:09:15.788987+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:36.491892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175976+0000","last_change":"2024-04-26T20:07:31.720938+0000","last_active":"2024-04-26T20:09:16.175976+0000","last_peered":"2024-04-26T20:09:16.175976+0000","last_clean":"2024-04-26T20:09:16.175976+0000","last_became_active":"2024-04-26T20:07:31.720579+0000","last_became_peered":"2024-04-26T20:07:31.720579+0000","last_unstale":"2024-04-26T20:09:16.175976+0000","last_undegraded":"2024-04-26T20:09:16.175976+0000","last_fullsized":"2024-04-26T20:09:16.175976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:55.737976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864109+0000","last_change":"2024-04-26T20:07:34.764802+0000","last_active":"2024-04-26T20:09:15.864109+0000","last_peered":"2024-04-26T20:09:15.864109+0000","last_clean":"2024-04-26T20:09:15.864109+0000","last_became_active":"2024-04-26T20:07:34.763468+0000","last_became_peered":"2024-04-26T20:07:34.763468+0000","last_unstale":"2024-04-26T20:09:15.864109+0000","last_undegraded":"2024-04-26T20:09:15.864109+0000","last_fullsized":"2024-04-26T20:09:15.864109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:23.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793284+0000","last_change":"2024-04-26T20:07:38.830392+0000","last_active":"2024-04-26T20:09:15.793284+0000","last_peered":"2024-04-26T20:09:15.793284+0000","last_clean":"2024-04-26T20:09:15.793284+0000","last_became_active":"2024-04-26T20:07:38.822964+0000","last_became_peered":"2024-04-26T20:07:38.822964+0000","last_unstale":"2024-04-26T20:09:15.793284+0000","last_undegraded":"2024-04-26T20:09:15.793284+0000","last_fullsized":"2024-04-26T20:09:15.793284+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:17.992368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'3","reported_seq":149,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:35.486859+0000","last_change":"2024-04-26T20:07:31.710567+0000","last_active":"2024-04-26T20:10:35.486859+0000","last_peered":"2024-04-26T20:10:35.486859+0000","last_clean":"2024-04-26T20:10:35.486859+0000","last_became_active":"2024-04-26T20:07:31.710197+0000","last_became_peered":"2024-04-26T20:07:31.710197+0000","last_unstale":"2024-04-26T20:10:35.486859+0000","last_undegraded":"2024-04-26T20:10:35.486859+0000","last_fullsized":"2024-04-26T20:10:35.486859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:10.648320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863839+0000","last_change":"2024-04-26T20:07:34.743237+0000","last_active":"2024-04-26T20:09:15.863839+0000","last_peered":"2024-04-26T20:09:15.863839+0000","last_clean":"2024-04-26T20:09:15.863839+0000","last_became_active":"2024-04-26T20:07:34.742146+0000","last_became_peered":"2024-04-26T20:07:34.742146+0000","last_unstale":"2024-04-26T20:09:15.863839+0000","last_undegraded":"2024-04-26T20:09:15.863839+0000","last_fullsized":"2024-04-26T20:09:15.863839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:15.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314505+0000","last_change":"2024-04-26T20:08:15.591942+0000","last_active":"2024-04-26T20:09:16.314505+0000","last_peered":"2024-04-26T20:09:16.314505+0000","last_clean":"2024-04-26T20:09:16.314505+0000","last_became_active":"2024-04-26T20:08:15.590842+0000","last_became_peered":"2024-04-26T20:08:15.590842+0000","last_unstale":"2024-04-26T20:09:16.314505+0000","last_undegraded":"2024-04-26T20:09:16.314505+0000","last_fullsized":"2024-04-26T20:09:16.314505+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:44.344152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"95'15","reported_seq":56,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364235+0000","last_change":"2024-04-26T20:07:31.716877+0000","last_active":"2024-04-26T20:09:16.364235+0000","last_peered":"2024-04-26T20:09:16.364235+0000","last_clean":"2024-04-26T20:09:16.364235+0000","last_became_active":"2024-04-26T20:07:31.716560+0000","last_became_peered":"2024-04-26T20:07:31.716560+0000","last_unstale":"2024-04-26T20:09:16.364235+0000","last_undegraded":"2024-04-26T20:09:16.364235+0000","last_fullsized":"2024-04-26T20:09:16.364235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:08:47.680991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175825+0000","last_change":"2024-04-26T20:07:34.749893+0000","last_active":"2024-04-26T20:09:16.175825+0000","last_peered":"2024-04-26T20:09:16.175825+0000","last_clean":"2024-04-26T20:09:16.175825+0000","last_became_active":"2024-04-26T20:07:34.748535+0000","last_became_peered":"2024-04-26T20:07:34.748535+0000","last_unstale":"2024-04-26T20:09:16.175825+0000","last_undegraded":"2024-04-26T20:09:16.175825+0000","last_fullsized":"2024-04-26T20:09:16.175825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:32.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175765+0000","last_change":"2024-04-26T20:07:38.824046+0000","last_active":"2024-04-26T20:09:16.175765+0000","last_peered":"2024-04-26T20:09:16.175765+0000","last_clean":"2024-04-26T20:09:16.175765+0000","last_became_active":"2024-04-26T20:07:38.823549+0000","last_became_peered":"2024-04-26T20:07:38.823549+0000","last_unstale":"2024-04-26T20:09:16.175765+0000","last_undegraded":"2024-04-26T20:09:16.175765+0000","last_fullsized":"2024-04-26T20:09:16.175765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:26.218201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.365016+0000","last_change":"2024-04-26T20:07:31.711968+0000","last_active":"2024-04-26T20:09:16.365016+0000","last_peered":"2024-04-26T20:09:16.365016+0000","last_clean":"2024-04-26T20:09:16.365016+0000","last_became_active":"2024-04-26T20:07:31.710982+0000","last_became_peered":"2024-04-26T20:07:31.710982+0000","last_unstale":"2024-04-26T20:09:16.365016+0000","last_undegraded":"2024-04-26T20:09:16.365016+0000","last_fullsized":"2024-04-26T20:09:16.365016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:29.526536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789172+0000","last_change":"2024-04-26T20:07:34.758101+0000","last_active":"2024-04-26T20:09:15.789172+0000","last_peered":"2024-04-26T20:09:15.789172+0000","last_clean":"2024-04-26T20:09:15.789172+0000","last_became_active":"2024-04-26T20:07:34.757747+0000","last_became_peered":"2024-04-26T20:07:34.757747+0000","last_unstale":"2024-04-26T20:09:15.789172+0000","last_undegraded":"2024-04-26T20:09:15.789172+0000","last_fullsized":"2024-04-26T20:09:15.789172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.337040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364569+0000","last_change":"2024-04-26T20:07:39.312976+0000","last_active":"2024-04-26T20:09:16.364569+0000","last_peered":"2024-04-26T20:09:16.364569+0000","last_clean":"2024-04-26T20:09:16.364569+0000","last_became_active":"2024-04-26T20:07:39.312443+0000","last_became_peered":"2024-04-26T20:07:39.312443+0000","last_unstale":"2024-04-26T20:09:16.364569+0000","last_undegraded":"2024-04-26T20:09:16.364569+0000","last_fullsized":"2024-04-26T20:09:16.364569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:23.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.792536+0000","last_change":"2024-04-26T20:08:14.587592+0000","last_active":"2024-04-26T20:09:15.792536+0000","last_peered":"2024-04-26T20:09:15.792536+0000","last_clean":"2024-04-26T20:09:15.792536+0000","last_became_active":"2024-04-26T20:08:14.587157+0000","last_became_peered":"2024-04-26T20:08:14.587157+0000","last_unstale":"2024-04-26T20:09:15.792536+0000","last_undegraded":"2024-04-26T20:09:15.792536+0000","last_fullsized":"2024-04-26T20:09:15.792536+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:22.184416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787061+0000","last_change":"2024-04-26T20:07:34.764376+0000","last_active":"2024-04-26T20:09:15.787061+0000","last_peered":"2024-04-26T20:09:15.787061+0000","last_clean":"2024-04-26T20:09:15.787061+0000","last_became_active":"2024-04-26T20:07:34.764040+0000","last_became_peered":"2024-04-26T20:07:34.764040+0000","last_unstale":"2024-04-26T20:09:15.787061+0000","last_undegraded":"2024-04-26T20:09:15.787061+0000","last_fullsized":"2024-04-26T20:09:15.787061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:43.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313757+0000","last_change":"2024-04-26T20:07:38.837075+0000","last_active":"2024-04-26T20:09:16.313757+0000","last_peered":"2024-04-26T20:09:16.313757+0000","last_clean":"2024-04-26T20:09:16.313757+0000","last_became_active":"2024-04-26T20:07:38.835912+0000","last_became_peered":"2024-04-26T20:07:38.835912+0000","last_unstale":"2024-04-26T20:09:16.313757+0000","last_undegraded":"2024-04-26T20:09:16.313757+0000","last_fullsized":"2024-04-26T20:09:16.313757+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.853262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863906+0000","last_change":"2024-04-26T20:07:31.728087+0000","last_active":"2024-04-26T20:09:15.863906+0000","last_peered":"2024-04-26T20:09:15.863906+0000","last_clean":"2024-04-26T20:09:15.863906+0000","last_became_active":"2024-04-26T20:07:31.727688+0000","last_became_peered":"2024-04-26T20:07:31.727688+0000","last_unstale":"2024-04-26T20:09:15.863906+0000","last_undegraded":"2024-04-26T20:09:15.863906+0000","last_fullsized":"2024-04-26T20:09:15.863906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:16.220049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787166+0000","last_change":"2024-04-26T20:07:34.765161+0000","last_active":"2024-04-26T20:09:15.787166+0000","last_peered":"2024-04-26T20:09:15.787166+0000","last_clean":"2024-04-26T20:09:15.787166+0000","last_became_active":"2024-04-26T20:07:34.764789+0000","last_became_peered":"2024-04-26T20:07:34.764789+0000","last_unstale":"2024-04-26T20:09:15.787166+0000","last_undegraded":"2024-04-26T20:09:15.787166+0000","last_fullsized":"2024-04-26T20:09:15.787166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:52.417157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314242+0000","last_change":"2024-04-26T20:07:39.313159+0000","last_active":"2024-04-26T20:09:16.314242+0000","last_peered":"2024-04-26T20:09:16.314242+0000","last_clean":"2024-04-26T20:09:16.314242+0000","last_became_active":"2024-04-26T20:07:39.312637+0000","last_became_peered":"2024-04-26T20:07:39.312637+0000","last_unstale":"2024-04-26T20:09:16.314242+0000","last_undegraded":"2024-04-26T20:09:16.314242+0000","last_fullsized":"2024-04-26T20:09:16.314242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:10.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176663+0000","last_change":"2024-04-26T20:08:14.586352+0000","last_active":"2024-04-26T20:09:16.176663+0000","last_peered":"2024-04-26T20:09:16.176663+0000","last_clean":"2024-04-26T20:09:16.176663+0000","last_became_active":"2024-04-26T20:08:14.585975+0000","last_became_peered":"2024-04-26T20:08:14.585975+0000","last_unstale":"2024-04-26T20:09:16.176663+0000","last_undegraded":"2024-04-26T20:09:16.176663+0000","last_fullsized":"2024-04-26T20:09:16.176663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:27.676683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'4","reported_seq":148,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:36.488299+0000","last_change":"2024-04-26T20:07:31.726272+0000","last_active":"2024-04-26T20:10:36.488299+0000","last_peered":"2024-04-26T20:10:36.488299+0000","last_clean":"2024-04-26T20:10:36.488299+0000","last_became_active":"2024-04-26T20:07:31.725889+0000","last_became_peered":"2024-04-26T20:07:31.725889+0000","last_unstale":"2024-04-26T20:10:36.488299+0000","last_undegraded":"2024-04-26T20:10:36.488299+0000","last_fullsized":"2024-04-26T20:10:36.488299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:03.091138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864225+0000","last_change":"2024-04-26T20:07:38.815171+0000","last_active":"2024-04-26T20:09:15.864225+0000","last_peered":"2024-04-26T20:09:15.864225+0000","last_clean":"2024-04-26T20:09:15.864225+0000","last_became_active":"2024-04-26T20:07:38.814769+0000","last_became_peered":"2024-04-26T20:07:38.814769+0000","last_unstale":"2024-04-26T20:09:15.864225+0000","last_undegraded":"2024-04-26T20:09:15.864225+0000","last_fullsized":"2024-04-26T20:09:15.864225+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789245+0000","last_change":"2024-04-26T20:07:34.766538+0000","last_active":"2024-04-26T20:09:15.789245+0000","last_peered":"2024-04-26T20:09:15.789245+0000","last_clean":"2024-04-26T20:09:15.789245+0000","last_became_active":"2024-04-26T20:07:34.761651+0000","last_became_peered":"2024-04-26T20:07:34.761651+0000","last_unstale":"2024-04-26T20:09:15.789245+0000","last_undegraded":"2024-04-26T20:09:15.789245+0000","last_fullsized":"2024-04-26T20:09:15.789245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:11.213046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087329+0000","last_change":"2024-04-26T20:07:31.720351+0000","last_active":"2024-04-26T20:09:16.087329+0000","last_peered":"2024-04-26T20:09:16.087329+0000","last_clean":"2024-04-26T20:09:16.087329+0000","last_became_active":"2024-04-26T20:07:31.719966+0000","last_became_peered":"2024-04-26T20:07:31.719966+0000","last_unstale":"2024-04-26T20:09:16.087329+0000","last_undegraded":"2024-04-26T20:09:16.087329+0000","last_fullsized":"2024-04-26T20:09:16.087329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:17.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863629+0000","last_change":"2024-04-26T20:05:56.337783+0000","last_active":"2024-04-26T20:09:15.863629+0000","last_peered":"2024-04-26T20:09:15.863629+0000","last_clean":"2024-04-26T20:09:15.863629+0000","last_became_active":"2024-04-26T20:05:56.337301+0000","last_became_peered":"2024-04-26T20:05:56.337301+0000","last_unstale":"2024-04-26T20:09:15.863629+0000","last_undegraded":"2024-04-26T20:09:15.863629+0000","last_fullsized":"2024-04-26T20:09:15.863629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_clean_scrub_stamp":"2024-04-26T20:01:58.332403+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:45.808606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176441+0000","last_change":"2024-04-26T20:07:39.313210+0000","last_active":"2024-04-26T20:09:16.176441+0000","last_peered":"2024-04-26T20:09:16.176441+0000","last_clean":"2024-04-26T20:09:16.176441+0000","last_became_active":"2024-04-26T20:07:39.312575+0000","last_became_peered":"2024-04-26T20:07:39.312575+0000","last_unstale":"2024-04-26T20:09:16.176441+0000","last_undegraded":"2024-04-26T20:09:16.176441+0000","last_fullsized":"2024-04-26T20:09:16.176441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:02.877951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313066+0000","last_change":"2024-04-26T20:07:34.759474+0000","last_active":"2024-04-26T20:09:16.313066+0000","last_peered":"2024-04-26T20:09:16.313066+0000","last_clean":"2024-04-26T20:09:16.313066+0000","last_became_active":"2024-04-26T20:07:34.759127+0000","last_became_peered":"2024-04-26T20:07:34.759127+0000","last_unstale":"2024-04-26T20:09:16.313066+0000","last_undegraded":"2024-04-26T20:09:16.313066+0000","last_fullsized":"2024-04-26T20:09:16.313066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.874866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863782+0000","last_change":"2024-04-26T20:07:31.710633+0000","last_active":"2024-04-26T20:09:15.863782+0000","last_peered":"2024-04-26T20:09:15.863782+0000","last_clean":"2024-04-26T20:09:15.863782+0000","last_became_active":"2024-04-26T20:07:31.710290+0000","last_became_peered":"2024-04-26T20:07:31.710290+0000","last_unstale":"2024-04-26T20:09:15.863782+0000","last_undegraded":"2024-04-26T20:09:15.863782+0000","last_fullsized":"2024-04-26T20:09:15.863782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:18.114770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788979+0000","last_change":"2024-04-26T20:07:38.829768+0000","last_active":"2024-04-26T20:09:15.788979+0000","last_peered":"2024-04-26T20:09:15.788979+0000","last_clean":"2024-04-26T20:09:15.788979+0000","last_became_active":"2024-04-26T20:07:38.829364+0000","last_became_peered":"2024-04-26T20:07:38.829364+0000","last_unstale":"2024-04-26T20:09:15.788979+0000","last_undegraded":"2024-04-26T20:09:15.788979+0000","last_fullsized":"2024-04-26T20:09:15.788979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:56.887062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"95'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364841+0000","last_change":"2024-04-26T20:07:31.720296+0000","last_active":"2024-04-26T20:09:16.364841+0000","last_peered":"2024-04-26T20:09:16.364841+0000","last_clean":"2024-04-26T20:09:16.364841+0000","last_became_active":"2024-04-26T20:07:31.719918+0000","last_became_peered":"2024-04-26T20:07:31.719918+0000","last_unstale":"2024-04-26T20:09:16.364841+0000","last_undegraded":"2024-04-26T20:09:16.364841+0000","last_fullsized":"2024-04-26T20:09:16.364841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:43.783104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314016+0000","last_change":"2024-04-26T20:07:34.770634+0000","last_active":"2024-04-26T20:09:16.314016+0000","last_peered":"2024-04-26T20:09:16.314016+0000","last_clean":"2024-04-26T20:09:16.314016+0000","last_became_active":"2024-04-26T20:07:34.770170+0000","last_became_peered":"2024-04-26T20:07:34.770170+0000","last_unstale":"2024-04-26T20:09:16.314016+0000","last_undegraded":"2024-04-26T20:09:16.314016+0000","last_fullsized":"2024-04-26T20:09:16.314016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:21.528735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.790102+0000","last_change":"2024-04-26T20:07:39.317345+0000","last_active":"2024-04-26T20:09:15.790102+0000","last_peered":"2024-04-26T20:09:15.790102+0000","last_clean":"2024-04-26T20:09:15.790102+0000","last_became_active":"2024-04-26T20:07:39.315998+0000","last_became_peered":"2024-04-26T20:07:39.315998+0000","last_unstale":"2024-04-26T20:09:15.790102+0000","last_undegraded":"2024-04-26T20:09:15.790102+0000","last_fullsized":"2024-04-26T20:09:15.790102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:18.715213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'10","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:40.712959+0000","last_change":"2024-04-26T20:07:31.729195+0000","last_active":"2024-04-26T20:09:40.712959+0000","last_peered":"2024-04-26T20:09:40.712959+0000","last_clean":"2024-04-26T20:09:40.712959+0000","last_became_active":"2024-04-26T20:07:31.728746+0000","last_became_peered":"2024-04-26T20:07:31.728746+0000","last_unstale":"2024-04-26T20:09:40.712959+0000","last_undegraded":"2024-04-26T20:09:40.712959+0000","last_fullsized":"2024-04-26T20:09:40.712959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:34.443091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791405+0000","last_change":"2024-04-26T20:07:34.765013+0000","last_active":"2024-04-26T20:09:15.791405+0000","last_peered":"2024-04-26T20:09:15.791405+0000","last_clean":"2024-04-26T20:09:15.791405+0000","last_became_active":"2024-04-26T20:07:34.764597+0000","last_became_peered":"2024-04-26T20:07:34.764597+0000","last_unstale":"2024-04-26T20:09:15.791405+0000","last_undegraded":"2024-04-26T20:09:15.791405+0000","last_fullsized":"2024-04-26T20:09:15.791405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:51.513026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786785+0000","last_change":"2024-04-26T20:07:38.805910+0000","last_active":"2024-04-26T20:09:15.786785+0000","last_peered":"2024-04-26T20:09:15.786785+0000","last_clean":"2024-04-26T20:09:15.786785+0000","last_became_active":"2024-04-26T20:07:38.804580+0000","last_became_peered":"2024-04-26T20:07:38.804580+0000","last_unstale":"2024-04-26T20:09:15.786785+0000","last_undegraded":"2024-04-26T20:09:15.786785+0000","last_fullsized":"2024-04-26T20:09:15.786785+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:41.321084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"95'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.086791+0000","last_change":"2024-04-26T20:07:31.915771+0000","last_active":"2024-04-26T20:09:16.086791+0000","last_peered":"2024-04-26T20:09:16.086791+0000","last_clean":"2024-04-26T20:09:16.086791+0000","last_became_active":"2024-04-26T20:07:31.915422+0000","last_became_peered":"2024-04-26T20:07:31.915422+0000","last_unstale":"2024-04-26T20:09:16.086791+0000","last_undegraded":"2024-04-26T20:09:16.086791+0000","last_fullsized":"2024-04-26T20:09:16.086791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:33.309270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.090395+0000","last_change":"2024-04-26T20:09:16.089155+0000","last_active":"2024-04-26T20:09:16.090395+0000","last_peered":"2024-04-26T20:09:16.090395+0000","last_clean":"2024-04-26T20:09:16.090395+0000","last_became_active":"2024-04-26T20:09:16.088425+0000","last_became_peered":"2024-04-26T20:09:16.088425+0000","last_unstale":"2024-04-26T20:09:16.090395+0000","last_undegraded":"2024-04-26T20:09:16.090395+0000","last_fullsized":"2024-04-26T20:09:16.090395+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:00.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087281+0000","last_change":"2024-04-26T20:07:39.313079+0000","last_active":"2024-04-26T20:09:16.087281+0000","last_peered":"2024-04-26T20:09:16.087281+0000","last_clean":"2024-04-26T20:09:16.087281+0000","last_became_active":"2024-04-26T20:07:39.312603+0000","last_became_peered":"2024-04-26T20:07:39.312603+0000","last_unstale":"2024-04-26T20:09:16.087281+0000","last_undegraded":"2024-04-26T20:09:16.087281+0000","last_fullsized":"2024-04-26T20:09:16.087281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:45.717375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266559+0000","last_change":"2024-04-26T20:07:31.715373+0000","last_active":"2024-04-26T20:09:16.266559+0000","last_peered":"2024-04-26T20:09:16.266559+0000","last_clean":"2024-04-26T20:09:16.266559+0000","last_became_active":"2024-04-26T20:07:31.714973+0000","last_became_peered":"2024-04-26T20:07:31.714973+0000","last_unstale":"2024-04-26T20:09:16.266559+0000","last_undegraded":"2024-04-26T20:09:16.266559+0000","last_fullsized":"2024-04-26T20:09:16.266559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.772671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364255+0000","last_change":"2024-04-26T20:07:34.753619+0000","last_active":"2024-04-26T20:09:16.364255+0000","last_peered":"2024-04-26T20:09:16.364255+0000","last_clean":"2024-04-26T20:09:16.364255+0000","last_became_active":"2024-04-26T20:07:34.753263+0000","last_became_peered":"2024-04-26T20:07:34.753263+0000","last_unstale":"2024-04-26T20:09:16.364255+0000","last_undegraded":"2024-04-26T20:09:16.364255+0000","last_fullsized":"2024-04-26T20:09:16.364255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.358153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793451+0000","last_change":"2024-04-26T20:08:15.840243+0000","last_active":"2024-04-26T20:09:15.793451+0000","last_peered":"2024-04-26T20:09:15.793451+0000","last_clean":"2024-04-26T20:09:15.793451+0000","last_became_active":"2024-04-26T20:08:15.838891+0000","last_became_peered":"2024-04-26T20:08:15.838891+0000","last_unstale":"2024-04-26T20:09:15.793451+0000","last_undegraded":"2024-04-26T20:09:15.793451+0000","last_fullsized":"2024-04-26T20:09:15.793451+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:15.018199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788938+0000","last_change":"2024-04-26T20:07:31.722516+0000","last_active":"2024-04-26T20:09:15.788938+0000","last_peered":"2024-04-26T20:09:15.788938+0000","last_clean":"2024-04-26T20:09:15.788938+0000","last_became_active":"2024-04-26T20:07:31.722096+0000","last_became_peered":"2024-04-26T20:07:31.722096+0000","last_unstale":"2024-04-26T20:09:15.788938+0000","last_undegraded":"2024-04-26T20:09:15.788938+0000","last_fullsized":"2024-04-26T20:09:15.788938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:57.693947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788865+0000","last_change":"2024-04-26T20:07:34.768754+0000","last_active":"2024-04-26T20:09:15.788865+0000","last_peered":"2024-04-26T20:09:15.788865+0000","last_clean":"2024-04-26T20:09:15.788865+0000","last_became_active":"2024-04-26T20:07:34.768405+0000","last_became_peered":"2024-04-26T20:07:34.768405+0000","last_unstale":"2024-04-26T20:09:15.788865+0000","last_undegraded":"2024-04-26T20:09:15.788865+0000","last_fullsized":"2024-04-26T20:09:15.788865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:08.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087279+0000","last_change":"2024-04-26T20:07:38.828804+0000","last_active":"2024-04-26T20:09:16.087279+0000","last_peered":"2024-04-26T20:09:16.087279+0000","last_clean":"2024-04-26T20:09:16.087279+0000","last_became_active":"2024-04-26T20:07:38.828452+0000","last_became_peered":"2024-04-26T20:07:38.828452+0000","last_unstale":"2024-04-26T20:09:16.087279+0000","last_undegraded":"2024-04-26T20:09:16.087279+0000","last_fullsized":"2024-04-26T20:09:16.087279+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:29.905102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'1","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787059+0000","last_change":"2024-04-26T20:07:31.710696+0000","last_active":"2024-04-26T20:09:15.787059+0000","last_peered":"2024-04-26T20:09:15.787059+0000","last_clean":"2024-04-26T20:09:15.787059+0000","last_became_active":"2024-04-26T20:07:31.710039+0000","last_became_peered":"2024-04-26T20:07:31.710039+0000","last_unstale":"2024-04-26T20:09:15.787059+0000","last_undegraded":"2024-04-26T20:09:15.787059+0000","last_fullsized":"2024-04-26T20:09:15.787059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:50.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.798031+0000","last_change":"2024-04-26T20:09:15.798031+0000","last_active":"2024-04-26T20:09:15.798031+0000","last_peered":"2024-04-26T20:09:15.798031+0000","last_clean":"2024-04-26T20:09:15.798031+0000","last_became_active":"2024-04-26T20:09:15.797535+0000","last_became_peered":"2024-04-26T20:09:15.797535+0000","last_unstale":"2024-04-26T20:09:15.798031+0000","last_undegraded":"2024-04-26T20:09:15.798031+0000","last_fullsized":"2024-04-26T20:09:15.798031+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:03.059618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087243+0000","last_change":"2024-04-26T20:07:39.312800+0000","last_active":"2024-04-26T20:09:16.087243+0000","last_peered":"2024-04-26T20:09:16.087243+0000","last_clean":"2024-04-26T20:09:16.087243+0000","last_became_active":"2024-04-26T20:07:39.312309+0000","last_became_peered":"2024-04-26T20:07:39.312309+0000","last_unstale":"2024-04-26T20:09:16.087243+0000","last_undegraded":"2024-04-26T20:09:16.087243+0000","last_fullsized":"2024-04-26T20:09:16.087243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:02.013468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364648+0000","last_change":"2024-04-26T20:07:31.715509+0000","last_active":"2024-04-26T20:09:16.364648+0000","last_peered":"2024-04-26T20:09:16.364648+0000","last_clean":"2024-04-26T20:09:16.364648+0000","last_became_active":"2024-04-26T20:07:31.714606+0000","last_became_peered":"2024-04-26T20:07:31.714606+0000","last_unstale":"2024-04-26T20:09:16.364648+0000","last_undegraded":"2024-04-26T20:09:16.364648+0000","last_fullsized":"2024-04-26T20:09:16.364648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:03.979681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863942+0000","last_change":"2024-04-26T20:07:34.763771+0000","last_active":"2024-04-26T20:09:15.863942+0000","last_peered":"2024-04-26T20:09:15.863942+0000","last_clean":"2024-04-26T20:09:15.863942+0000","last_became_active":"2024-04-26T20:07:34.763422+0000","last_became_peered":"2024-04-26T20:07:34.763422+0000","last_unstale":"2024-04-26T20:09:15.863942+0000","last_undegraded":"2024-04-26T20:09:15.863942+0000","last_fullsized":"2024-04-26T20:09:15.863942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:27.224024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266640+0000","last_change":"2024-04-26T20:07:38.830156+0000","last_active":"2024-04-26T20:09:16.266640+0000","last_peered":"2024-04-26T20:09:16.266640+0000","last_clean":"2024-04-26T20:09:16.266640+0000","last_became_active":"2024-04-26T20:07:38.829530+0000","last_became_peered":"2024-04-26T20:07:38.829530+0000","last_unstale":"2024-04-26T20:09:16.266640+0000","last_undegraded":"2024-04-26T20:09:16.266640+0000","last_fullsized":"2024-04-26T20:09:16.266640+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:21.720974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787310+0000","last_change":"2024-04-26T20:07:31.915638+0000","last_active":"2024-04-26T20:09:15.787310+0000","last_peered":"2024-04-26T20:09:15.787310+0000","last_clean":"2024-04-26T20:09:15.787310+0000","last_became_active":"2024-04-26T20:07:31.915266+0000","last_became_peered":"2024-04-26T20:07:31.915266+0000","last_unstale":"2024-04-26T20:09:15.787310+0000","last_undegraded":"2024-04-26T20:09:15.787310+0000","last_fullsized":"2024-04-26T20:09:15.787310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:51.452828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864392+0000","last_change":"2024-04-26T20:07:34.754476+0000","last_active":"2024-04-26T20:09:15.864392+0000","last_peered":"2024-04-26T20:09:15.864392+0000","last_clean":"2024-04-26T20:09:15.864392+0000","last_became_active":"2024-04-26T20:07:34.754054+0000","last_became_peered":"2024-04-26T20:07:34.754054+0000","last_unstale":"2024-04-26T20:09:15.864392+0000","last_undegraded":"2024-04-26T20:09:15.864392+0000","last_fullsized":"2024-04-26T20:09:15.864392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.497083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.799393+0000","last_change":"2024-04-26T20:07:38.835265+0000","last_active":"2024-04-26T20:09:15.799393+0000","last_peered":"2024-04-26T20:09:15.799393+0000","last_clean":"2024-04-26T20:09:15.799393+0000","last_became_active":"2024-04-26T20:07:38.833873+0000","last_became_peered":"2024-04-26T20:07:38.833873+0000","last_unstale":"2024-04-26T20:09:15.799393+0000","last_undegraded":"2024-04-26T20:09:15.799393+0000","last_fullsized":"2024-04-26T20:09:15.799393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:17.850236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176410+0000","last_change":"2024-04-26T20:07:31.725576+0000","last_active":"2024-04-26T20:09:16.176410+0000","last_peered":"2024-04-26T20:09:16.176410+0000","last_clean":"2024-04-26T20:09:16.176410+0000","last_became_active":"2024-04-26T20:07:31.725166+0000","last_became_peered":"2024-04-26T20:07:31.725166+0000","last_unstale":"2024-04-26T20:09:16.176410+0000","last_undegraded":"2024-04-26T20:09:16.176410+0000","last_fullsized":"2024-04-26T20:09:16.176410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:04.539215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864110+0000","last_change":"2024-04-26T20:07:34.748263+0000","last_active":"2024-04-26T20:09:15.864110+0000","last_peered":"2024-04-26T20:09:15.864110+0000","last_clean":"2024-04-26T20:09:15.864110+0000","last_became_active":"2024-04-26T20:07:34.747889+0000","last_became_peered":"2024-04-26T20:07:34.747889+0000","last_unstale":"2024-04-26T20:09:15.864110+0000","last_undegraded":"2024-04-26T20:09:15.864110+0000","last_fullsized":"2024-04-26T20:09:15.864110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.257405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175579+0000","last_change":"2024-04-26T20:07:38.803999+0000","last_active":"2024-04-26T20:09:16.175579+0000","last_peered":"2024-04-26T20:09:16.175579+0000","last_clean":"2024-04-26T20:09:16.175579+0000","last_became_active":"2024-04-26T20:07:38.803585+0000","last_became_peered":"2024-04-26T20:07:38.803585+0000","last_unstale":"2024-04-26T20:09:16.175579+0000","last_undegraded":"2024-04-26T20:09:16.175579+0000","last_fullsized":"2024-04-26T20:09:16.175579+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:58.863511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314386+0000","last_change":"2024-04-26T20:07:31.730445+0000","last_active":"2024-04-26T20:09:16.314386+0000","last_peered":"2024-04-26T20:09:16.314386+0000","last_clean":"2024-04-26T20:09:16.314386+0000","last_became_active":"2024-04-26T20:07:31.729944+0000","last_became_peered":"2024-04-26T20:07:31.729944+0000","last_unstale":"2024-04-26T20:09:16.314386+0000","last_undegraded":"2024-04-26T20:09:16.314386+0000","last_fullsized":"2024-04-26T20:09:16.314386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:06.274097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864441+0000","last_change":"2024-04-26T20:07:34.755449+0000","last_active":"2024-04-26T20:09:15.864441+0000","last_peered":"2024-04-26T20:09:15.864441+0000","last_clean":"2024-04-26T20:09:15.864441+0000","last_became_active":"2024-04-26T20:07:34.755100+0000","last_became_peered":"2024-04-26T20:07:34.755100+0000","last_unstale":"2024-04-26T20:09:15.864441+0000","last_undegraded":"2024-04-26T20:09:15.864441+0000","last_fullsized":"2024-04-26T20:09:15.864441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.774751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.178064+0000","last_change":"2024-04-26T20:09:16.177597+0000","last_active":"2024-04-26T20:09:16.178064+0000","last_peered":"2024-04-26T20:09:16.178064+0000","last_clean":"2024-04-26T20:09:16.178064+0000","last_became_active":"2024-04-26T20:09:16.177150+0000","last_became_peered":"2024-04-26T20:09:16.177150+0000","last_unstale":"2024-04-26T20:09:16.178064+0000","last_undegraded":"2024-04-26T20:09:16.178064+0000","last_fullsized":"2024-04-26T20:09:16.178064+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:50.910457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,5],"acting":[11,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787233+0000","last_change":"2024-04-26T20:07:34.747723+0000","last_active":"2024-04-26T20:09:15.787233+0000","last_peered":"2024-04-26T20:09:15.787233+0000","last_clean":"2024-04-26T20:09:15.787233+0000","last_became_active":"2024-04-26T20:07:34.747411+0000","last_became_peered":"2024-04-26T20:07:34.747411+0000","last_unstale":"2024-04-26T20:09:15.787233+0000","last_undegraded":"2024-04-26T20:09:15.787233+0000","last_fullsized":"2024-04-26T20:09:15.787233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786983+0000","last_change":"2024-04-26T20:07:34.773012+0000","last_active":"2024-04-26T20:09:15.786983+0000","last_peered":"2024-04-26T20:09:15.786983+0000","last_clean":"2024-04-26T20:09:15.786983+0000","last_became_active":"2024-04-26T20:07:34.772735+0000","last_became_peered":"2024-04-26T20:07:34.772735+0000","last_unstale":"2024-04-26T20:09:15.786983+0000","last_undegraded":"2024-04-26T20:09:15.786983+0000","last_fullsized":"2024-04-26T20:09:15.786983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:08.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.799589+0000","last_change":"2024-04-26T20:07:34.762977+0000","last_active":"2024-04-26T20:09:15.799589+0000","last_peered":"2024-04-26T20:09:15.799589+0000","last_clean":"2024-04-26T20:09:15.799589+0000","last_became_active":"2024-04-26T20:07:34.762530+0000","last_became_peered":"2024-04-26T20:07:34.762530+0000","last_unstale":"2024-04-26T20:09:15.799589+0000","last_undegraded":"2024-04-26T20:09:15.799589+0000","last_fullsized":"2024-04-26T20:09:15.799589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:24.811825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314034+0000","last_change":"2024-04-26T20:07:34.769986+0000","last_active":"2024-04-26T20:09:16.314034+0000","last_peered":"2024-04-26T20:09:16.314034+0000","last_clean":"2024-04-26T20:09:16.314034+0000","last_became_active":"2024-04-26T20:07:34.769591+0000","last_became_peered":"2024-04-26T20:07:34.769591+0000","last_unstale":"2024-04-26T20:09:16.314034+0000","last_undegraded":"2024-04-26T20:09:16.314034+0000","last_fullsized":"2024-04-26T20:09:16.314034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:27.466675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788921+0000","last_change":"2024-04-26T20:07:34.754733+0000","last_active":"2024-04-26T20:09:15.788921+0000","last_peered":"2024-04-26T20:09:15.788921+0000","last_clean":"2024-04-26T20:09:15.788921+0000","last_became_active":"2024-04-26T20:07:34.754374+0000","last_became_peered":"2024-04-26T20:07:34.754374+0000","last_unstale":"2024-04-26T20:09:15.788921+0000","last_undegraded":"2024-04-26T20:09:15.788921+0000","last_fullsized":"2024-04-26T20:09:15.788921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:02.732025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787681+0000","last_change":"2024-04-26T20:07:34.750329+0000","last_active":"2024-04-26T20:09:15.787681+0000","last_peered":"2024-04-26T20:09:15.787681+0000","last_clean":"2024-04-26T20:09:15.787681+0000","last_became_active":"2024-04-26T20:07:34.749970+0000","last_became_peered":"2024-04-26T20:07:34.749970+0000","last_unstale":"2024-04-26T20:09:15.787681+0000","last_undegraded":"2024-04-26T20:09:15.787681+0000","last_fullsized":"2024-04-26T20:09:15.787681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:50.375128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791609+0000","last_change":"2024-04-26T20:07:34.750623+0000","last_active":"2024-04-26T20:09:15.791609+0000","last_peered":"2024-04-26T20:09:15.791609+0000","last_clean":"2024-04-26T20:09:15.791609+0000","last_became_active":"2024-04-26T20:07:34.750314+0000","last_became_peered":"2024-04-26T20:07:34.750314+0000","last_unstale":"2024-04-26T20:09:15.791609+0000","last_undegraded":"2024-04-26T20:09:15.791609+0000","last_fullsized":"2024-04-26T20:09:15.791609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:16.773986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176551+0000","last_change":"2024-04-26T20:07:34.755474+0000","last_active":"2024-04-26T20:09:16.176551+0000","last_peered":"2024-04-26T20:09:16.176551+0000","last_clean":"2024-04-26T20:09:16.176551+0000","last_became_active":"2024-04-26T20:07:34.755133+0000","last_became_peered":"2024-04-26T20:07:34.755133+0000","last_unstale":"2024-04-26T20:09:16.176551+0000","last_undegraded":"2024-04-26T20:09:16.176551+0000","last_fullsized":"2024-04-26T20:09:16.176551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:02.861643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791743+0000","last_change":"2024-04-26T20:07:34.752727+0000","last_active":"2024-04-26T20:09:15.791743+0000","last_peered":"2024-04-26T20:09:15.791743+0000","last_clean":"2024-04-26T20:09:15.791743+0000","last_became_active":"2024-04-26T20:07:34.752420+0000","last_became_peered":"2024-04-26T20:07:34.752420+0000","last_unstale":"2024-04-26T20:09:15.791743+0000","last_undegraded":"2024-04-26T20:09:15.791743+0000","last_fullsized":"2024-04-26T20:09:15.791743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:45.076837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266861+0000","last_change":"2024-04-26T20:07:34.775116+0000","last_active":"2024-04-26T20:09:16.266861+0000","last_peered":"2024-04-26T20:09:16.266861+0000","last_clean":"2024-04-26T20:09:16.266861+0000","last_became_active":"2024-04-26T20:07:34.774710+0000","last_became_peered":"2024-04-26T20:07:34.774710+0000","last_unstale":"2024-04-26T20:09:16.266861+0000","last_undegraded":"2024-04-26T20:09:16.266861+0000","last_fullsized":"2024-04-26T20:09:16.266861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:53.203918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313636+0000","last_change":"2024-04-26T20:07:34.773245+0000","last_active":"2024-04-26T20:09:16.313636+0000","last_peered":"2024-04-26T20:09:16.313636+0000","last_clean":"2024-04-26T20:09:16.313636+0000","last_became_active":"2024-04-26T20:07:34.772879+0000","last_became_peered":"2024-04-26T20:07:34.772879+0000","last_unstale":"2024-04-26T20:09:16.313636+0000","last_undegraded":"2024-04-26T20:09:16.313636+0000","last_fullsized":"2024-04-26T20:09:16.313636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:54.450454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789326+0000","last_change":"2024-04-26T20:07:34.753716+0000","last_active":"2024-04-26T20:09:15.789326+0000","last_peered":"2024-04-26T20:09:15.789326+0000","last_clean":"2024-04-26T20:09:15.789326+0000","last_became_active":"2024-04-26T20:07:34.753378+0000","last_became_peered":"2024-04-26T20:07:34.753378+0000","last_unstale":"2024-04-26T20:09:15.789326+0000","last_undegraded":"2024-04-26T20:09:15.789326+0000","last_fullsized":"2024-04-26T20:09:15.789326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:47.165305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60116,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":657,"num_read_kb":393,"num_write":167,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":180348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383734,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":554234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28772,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716380,"statfs":{"total":95995035648,"available":95965573120,"internally_reserved":0,"allocated":1798144,"data_stored":1197617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808898,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":558566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070536,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":554390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332173,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1150299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495699,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":549646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":605671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738485,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":551034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40664,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704488,"statfs":{"total":95995035648,"available":95953395712,"internally_reserved":0,"allocated":1196032,"data_stored":607412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182501,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1122304,"data_stored":549840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":554296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1732608,"data_stored":1145887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:10:41.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T20:10:41.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:41 smithi157 ceph-mon[32703]: pgmap v430: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:41.995 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:41 smithi157 ceph-mon[32703]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:10:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:41 smithi131 ceph-mon[27919]: pgmap v430: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:42.043 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:41 smithi131 ceph-mon[27919]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:10:42.093 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:41 smithi148 ceph-mon[32281]: pgmap v430: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:42.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:41 smithi148 ceph-mon[32281]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:10:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:42.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:42 smithi131 ceph-mon[27919]: pgmap v431: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:42.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:42 smithi157 ceph-mon[32703]: pgmap v431: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:43.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:42 smithi148 ceph-mon[32281]: pgmap v431: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:44.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 -- 172.21.15.131:0/1772862115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c1094a0 msgr2=0x7f5a1c109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:44.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 --2- 172.21.15.131:0/1772862115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c1094a0 0x7f5a1c109880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009990 tx=0x7f5a10009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:44.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 -- 172.21.15.131:0/1772862115 shutdown_connections 2024-04-26T20:10:44.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 --2- 172.21.15.131:0/1772862115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c06bad0 0x7f5a1c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:44.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 --2- 172.21.15.131:0/1772862115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c1094a0 0x7f5a1c109880 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:44.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 --2- 172.21.15.131:0/1772862115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a1c06b130 0x7f5a1c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:44.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 -- 172.21.15.131:0/1772862115 >> 172.21.15.131:0/1772862115 conn(0x7f5a1c0792d0 msgr2=0x7f5a1c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:44.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 -- 172.21.15.131:0/1772862115 shutdown_connections 2024-04-26T20:10:44.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.356+0000 7f5a24000700 1 -- 172.21.15.131:0/1772862115 wait complete. 2024-04-26T20:10:44.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 Processor -- start 2024-04-26T20:10:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 -- start start 2024-04-26T20:10:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c06b130 0x7f5a1c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a1c06bad0 0x7f5a1c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:44.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c1094a0 0x7f5a1c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:44.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1135e0 con 0x7f5a1c1094a0 2024-04-26T20:10:44.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c113460 con 0x7f5a1c06b130 2024-04-26T20:10:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.358+0000 7f5a24000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c113760 con 0x7f5a1c06bad0 2024-04-26T20:10:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c06b130 0x7f5a1c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:44.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a20c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a1c06bad0 0x7f5a1c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c06b130 0x7f5a1c1023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:45880/0 (socket says 172.21.15.131:45880) 2024-04-26T20:10:44.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 -- 172.21.15.131:0/741154778 learned_addr learned my addr 172.21.15.131:0/741154778 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a22000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c1094a0 0x7f5a1c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:45.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 -- 172.21.15.131:0/741154778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a1c06bad0 msgr2=0x7f5a1c102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:45.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a1c06bad0 0x7f5a1c102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 -- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c1094a0 msgr2=0x7f5a1c102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:45.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c1094a0 0x7f5a1c102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a21600700 1 -- 172.21.15.131:0/741154778 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c1a8080 con 0x7f5a1c06b130 2024-04-26T20:10:45.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a20c00700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a1c06bad0 0x7f5a1c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:45.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.359+0000 7f5a22000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c1094a0 0x7f5a1c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:45.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.360+0000 7f5a21600700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c06b130 0x7f5a1c1023f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000c00 tx=0x7f5a100057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:45.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.360+0000 7f5a1a000700 1 -- 172.21.15.131:0/741154778 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10027070 con 0x7f5a1c06b130 2024-04-26T20:10:45.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.360+0000 7f5a1a000700 1 -- 172.21.15.131:0/741154778 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a10022470 con 0x7f5a1c06b130 2024-04-26T20:10:45.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.360+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c1a8310 con 0x7f5a1c06b130 2024-04-26T20:10:45.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.360+0000 7f5a1a000700 1 -- 172.21.15.131:0/741154778 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10017650 con 0x7f5a1c06b130 2024-04-26T20:10:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.360+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1a87c0 con 0x7f5a1c06b130 2024-04-26T20:10:45.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.361+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c06a110 con 0x7f5a1c06b130 2024-04-26T20:10:45.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.363+0000 7f5a1a000700 1 -- 172.21.15.131:0/741154778 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5a1002a070 con 0x7f5a1c06b130 2024-04-26T20:10:45.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.365+0000 7f5a1a000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a040a3d50 0x7f5a040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.365+0000 7f5a1a000700 1 -- 172.21.15.131:0/741154778 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5a100d1600 con 0x7f5a1c06b130 2024-04-26T20:10:45.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.365+0000 7f5a20c00700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a040a3d50 0x7f5a040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:45.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.366+0000 7f5a20c00700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a040a3d50 0x7f5a040a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c079a10 tx=0x7f5a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:45.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.367+0000 7f5a1a000700 1 -- 172.21.15.131:0/741154778 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5a10013070 con 0x7f5a1c06b130 2024-04-26T20:10:45.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.587+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a1c063860 con 0x7f5a040a3d50 2024-04-26T20:10:45.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.617+0000 7f5a1a000700 1 -- 172.21.15.131:0/741154778 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f5a1c063860 con 0x7f5a040a3d50 2024-04-26T20:10:45.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:44 smithi148 ceph-mon[32281]: pgmap v432: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:45.400 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:44 smithi148 ceph-mon[32281]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:10:45.406 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:44 smithi157 ceph-mon[32703]: pgmap v432: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:45.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:44 smithi157 ceph-mon[32703]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:10:45.422 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:10:45.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a040a3d50 msgr2=0x7f5a040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:45.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a040a3d50 0x7f5a040a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c079a10 tx=0x7f5a08009260 comp rx=0 tx=0).stop 2024-04-26T20:10:45.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c06b130 msgr2=0x7f5a1c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:45.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c06b130 0x7f5a1c1023f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000c00 tx=0x7f5a100057c0 comp rx=0 tx=0).stop 2024-04-26T20:10:45.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 shutdown_connections 2024-04-26T20:10:45.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c1094a0 0x7f5a1c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:45.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5a040a3d50 0x7f5a040a6210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:45.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c06b130 0x7f5a1c1023f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:45.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 --2- 172.21.15.131:0/741154778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a1c06bad0 0x7f5a1c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:45.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 >> 172.21.15.131:0/741154778 conn(0x7f5a1c0792d0 msgr2=0x7f5a1c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:45.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 shutdown_connections 2024-04-26T20:10:45.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:44.623+0000 7f5a24000700 1 -- 172.21.15.131:0/741154778 wait complete. 2024-04-26T20:10:45.824 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-26T20:10:45.833 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:44 smithi131 ceph-mon[27919]: pgmap v432: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:45.842 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:44 smithi131 ceph-mon[27919]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:10:45.852 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-26T20:10:43.674292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650718,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":813,"num_read_kb":517,"num_write":350,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368488,"kb_used_data":15128,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573336,"statfs":{"total":1151940427776,"available":1151563096064,"internally_reserved":0,"allocated":15491072,"data_stored":8578892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005230"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789445+0000","last_change":"2024-04-26T20:07:34.771977+0000","last_active":"2024-04-26T20:09:15.789445+0000","last_peered":"2024-04-26T20:09:15.789445+0000","last_clean":"2024-04-26T20:09:15.789445+0000","last_became_active":"2024-04-26T20:07:34.771658+0000","last_became_peered":"2024-04-26T20:07:34.771658+0000","last_unstale":"2024-04-26T20:09:15.789445+0000","last_undegraded":"2024-04-26T20:09:15.789445+0000","last_fullsized":"2024-04-26T20:09:15.789445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:38.419057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.800449+0000","last_change":"2024-04-26T20:07:34.768774+0000","last_active":"2024-04-26T20:09:15.800449+0000","last_peered":"2024-04-26T20:09:15.800449+0000","last_clean":"2024-04-26T20:09:15.800449+0000","last_became_active":"2024-04-26T20:07:34.768383+0000","last_became_peered":"2024-04-26T20:07:34.768383+0000","last_unstale":"2024-04-26T20:09:15.800449+0000","last_undegraded":"2024-04-26T20:09:15.800449+0000","last_fullsized":"2024-04-26T20:09:15.800449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:43.813261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.798842+0000","last_change":"2024-04-26T20:07:34.761796+0000","last_active":"2024-04-26T20:09:15.798842+0000","last_peered":"2024-04-26T20:09:15.798842+0000","last_clean":"2024-04-26T20:09:15.798842+0000","last_became_active":"2024-04-26T20:07:34.761403+0000","last_became_peered":"2024-04-26T20:07:34.761403+0000","last_unstale":"2024-04-26T20:09:15.798842+0000","last_undegraded":"2024-04-26T20:09:15.798842+0000","last_fullsized":"2024-04-26T20:09:15.798842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:06.832304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087419+0000","last_change":"2024-04-26T20:07:34.756728+0000","last_active":"2024-04-26T20:09:16.087419+0000","last_peered":"2024-04-26T20:09:16.087419+0000","last_clean":"2024-04-26T20:09:16.087419+0000","last_became_active":"2024-04-26T20:07:34.756406+0000","last_became_peered":"2024-04-26T20:07:34.756406+0000","last_unstale":"2024-04-26T20:09:16.087419+0000","last_undegraded":"2024-04-26T20:09:16.087419+0000","last_fullsized":"2024-04-26T20:09:16.087419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:27.753674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364503+0000","last_change":"2024-04-26T20:07:34.758243+0000","last_active":"2024-04-26T20:09:16.364503+0000","last_peered":"2024-04-26T20:09:16.364503+0000","last_clean":"2024-04-26T20:09:16.364503+0000","last_became_active":"2024-04-26T20:07:34.757868+0000","last_became_peered":"2024-04-26T20:07:34.757868+0000","last_unstale":"2024-04-26T20:09:16.364503+0000","last_undegraded":"2024-04-26T20:09:16.364503+0000","last_fullsized":"2024-04-26T20:09:16.364503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:53.361835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786953+0000","last_change":"2024-04-26T20:08:15.585311+0000","last_active":"2024-04-26T20:09:15.786953+0000","last_peered":"2024-04-26T20:09:15.786953+0000","last_clean":"2024-04-26T20:09:15.786953+0000","last_became_active":"2024-04-26T20:08:15.584982+0000","last_became_peered":"2024-04-26T20:08:15.584982+0000","last_unstale":"2024-04-26T20:09:15.786953+0000","last_undegraded":"2024-04-26T20:09:15.786953+0000","last_fullsized":"2024-04-26T20:09:15.786953+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:56.828218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863732+0000","last_change":"2024-04-26T20:07:34.757529+0000","last_active":"2024-04-26T20:09:15.863732+0000","last_peered":"2024-04-26T20:09:15.863732+0000","last_clean":"2024-04-26T20:09:15.863732+0000","last_became_active":"2024-04-26T20:07:34.757166+0000","last_became_peered":"2024-04-26T20:07:34.757166+0000","last_unstale":"2024-04-26T20:09:15.863732+0000","last_undegraded":"2024-04-26T20:09:15.863732+0000","last_fullsized":"2024-04-26T20:09:15.863732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:01.309698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788256+0000","last_change":"2024-04-26T20:07:34.765352+0000","last_active":"2024-04-26T20:09:15.788256+0000","last_peered":"2024-04-26T20:09:15.788256+0000","last_clean":"2024-04-26T20:09:15.788256+0000","last_became_active":"2024-04-26T20:07:34.758400+0000","last_became_peered":"2024-04-26T20:07:34.758400+0000","last_unstale":"2024-04-26T20:09:15.788256+0000","last_undegraded":"2024-04-26T20:09:15.788256+0000","last_fullsized":"2024-04-26T20:09:15.788256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:43.940746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.267191+0000","last_change":"2024-04-26T20:07:34.754413+0000","last_active":"2024-04-26T20:09:16.267191+0000","last_peered":"2024-04-26T20:09:16.267191+0000","last_clean":"2024-04-26T20:09:16.267191+0000","last_became_active":"2024-04-26T20:07:34.754073+0000","last_became_peered":"2024-04-26T20:07:34.754073+0000","last_unstale":"2024-04-26T20:09:16.267191+0000","last_undegraded":"2024-04-26T20:09:16.267191+0000","last_fullsized":"2024-04-26T20:09:16.267191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:25.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787376+0000","last_change":"2024-04-26T20:07:34.748739+0000","last_active":"2024-04-26T20:09:15.787376+0000","last_peered":"2024-04-26T20:09:15.787376+0000","last_clean":"2024-04-26T20:09:15.787376+0000","last_became_active":"2024-04-26T20:07:34.748378+0000","last_became_peered":"2024-04-26T20:07:34.748378+0000","last_unstale":"2024-04-26T20:09:15.787376+0000","last_undegraded":"2024-04-26T20:09:15.787376+0000","last_fullsized":"2024-04-26T20:09:15.787376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:13.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864290+0000","last_change":"2024-04-26T20:07:34.747402+0000","last_active":"2024-04-26T20:09:15.864290+0000","last_peered":"2024-04-26T20:09:15.864290+0000","last_clean":"2024-04-26T20:09:15.864290+0000","last_became_active":"2024-04-26T20:07:34.746965+0000","last_became_peered":"2024-04-26T20:07:34.746965+0000","last_unstale":"2024-04-26T20:09:15.864290+0000","last_undegraded":"2024-04-26T20:09:15.864290+0000","last_fullsized":"2024-04-26T20:09:15.864290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:32.066286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.366157+0000","last_change":"2024-04-26T20:09:16.366157+0000","last_active":"2024-04-26T20:09:16.366157+0000","last_peered":"2024-04-26T20:09:16.366157+0000","last_clean":"2024-04-26T20:09:16.366157+0000","last_became_active":"2024-04-26T20:09:16.365132+0000","last_became_peered":"2024-04-26T20:09:16.365132+0000","last_unstale":"2024-04-26T20:09:16.366157+0000","last_undegraded":"2024-04-26T20:09:16.366157+0000","last_fullsized":"2024-04-26T20:09:16.366157+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.559981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313915+0000","last_change":"2024-04-26T20:07:34.766542+0000","last_active":"2024-04-26T20:09:16.313915+0000","last_peered":"2024-04-26T20:09:16.313915+0000","last_clean":"2024-04-26T20:09:16.313915+0000","last_became_active":"2024-04-26T20:07:34.766185+0000","last_became_peered":"2024-04-26T20:07:34.766185+0000","last_unstale":"2024-04-26T20:09:16.313915+0000","last_undegraded":"2024-04-26T20:09:16.313915+0000","last_fullsized":"2024-04-26T20:09:16.313915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:34.499145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.797698+0000","last_change":"2024-04-26T20:07:34.762610+0000","last_active":"2024-04-26T20:09:15.797698+0000","last_peered":"2024-04-26T20:09:15.797698+0000","last_clean":"2024-04-26T20:09:15.797698+0000","last_became_active":"2024-04-26T20:07:34.762100+0000","last_became_peered":"2024-04-26T20:07:34.762100+0000","last_unstale":"2024-04-26T20:09:15.797698+0000","last_undegraded":"2024-04-26T20:09:15.797698+0000","last_fullsized":"2024-04-26T20:09:15.797698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:18.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.798758+0000","last_change":"2024-04-26T20:07:34.766754+0000","last_active":"2024-04-26T20:09:15.798758+0000","last_peered":"2024-04-26T20:09:15.798758+0000","last_clean":"2024-04-26T20:09:15.798758+0000","last_became_active":"2024-04-26T20:07:34.762204+0000","last_became_peered":"2024-04-26T20:07:34.762204+0000","last_unstale":"2024-04-26T20:09:15.798758+0000","last_undegraded":"2024-04-26T20:09:15.798758+0000","last_fullsized":"2024-04-26T20:09:15.798758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:25.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266493+0000","last_change":"2024-04-26T20:07:34.769739+0000","last_active":"2024-04-26T20:09:16.266493+0000","last_peered":"2024-04-26T20:09:16.266493+0000","last_clean":"2024-04-26T20:09:16.266493+0000","last_became_active":"2024-04-26T20:07:34.769383+0000","last_became_peered":"2024-04-26T20:07:34.769383+0000","last_unstale":"2024-04-26T20:09:16.266493+0000","last_undegraded":"2024-04-26T20:09:16.266493+0000","last_fullsized":"2024-04-26T20:09:16.266493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:08.771590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789111+0000","last_change":"2024-04-26T20:07:34.772010+0000","last_active":"2024-04-26T20:09:15.789111+0000","last_peered":"2024-04-26T20:09:15.789111+0000","last_clean":"2024-04-26T20:09:15.789111+0000","last_became_active":"2024-04-26T20:07:34.771682+0000","last_became_peered":"2024-04-26T20:07:34.771682+0000","last_unstale":"2024-04-26T20:09:15.789111+0000","last_undegraded":"2024-04-26T20:09:15.789111+0000","last_fullsized":"2024-04-26T20:09:15.789111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:57.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793188+0000","last_change":"2024-04-26T20:07:34.757547+0000","last_active":"2024-04-26T20:09:15.793188+0000","last_peered":"2024-04-26T20:09:15.793188+0000","last_clean":"2024-04-26T20:09:15.793188+0000","last_became_active":"2024-04-26T20:07:34.757180+0000","last_became_peered":"2024-04-26T20:07:34.757180+0000","last_unstale":"2024-04-26T20:09:15.793188+0000","last_undegraded":"2024-04-26T20:09:15.793188+0000","last_fullsized":"2024-04-26T20:09:15.793188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:32.365944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787602+0000","last_change":"2024-04-26T20:07:34.767363+0000","last_active":"2024-04-26T20:09:15.787602+0000","last_peered":"2024-04-26T20:09:15.787602+0000","last_clean":"2024-04-26T20:09:15.787602+0000","last_became_active":"2024-04-26T20:07:34.766909+0000","last_became_peered":"2024-04-26T20:07:34.766909+0000","last_unstale":"2024-04-26T20:09:15.787602+0000","last_undegraded":"2024-04-26T20:09:15.787602+0000","last_fullsized":"2024-04-26T20:09:15.787602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:32:31.500141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787072+0000","last_change":"2024-04-26T20:07:38.823959+0000","last_active":"2024-04-26T20:09:15.787072+0000","last_peered":"2024-04-26T20:09:15.787072+0000","last_clean":"2024-04-26T20:09:15.787072+0000","last_became_active":"2024-04-26T20:07:38.823524+0000","last_became_peered":"2024-04-26T20:07:38.823524+0000","last_unstale":"2024-04-26T20:09:15.787072+0000","last_undegraded":"2024-04-26T20:09:15.787072+0000","last_fullsized":"2024-04-26T20:09:15.787072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:23.348376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'9","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793563+0000","last_change":"2024-04-26T20:07:31.721324+0000","last_active":"2024-04-26T20:09:15.793563+0000","last_peered":"2024-04-26T20:09:15.793563+0000","last_clean":"2024-04-26T20:09:15.793563+0000","last_became_active":"2024-04-26T20:07:31.720140+0000","last_became_peered":"2024-04-26T20:07:31.720140+0000","last_unstale":"2024-04-26T20:09:15.793563+0000","last_undegraded":"2024-04-26T20:09:15.793563+0000","last_fullsized":"2024-04-26T20:09:15.793563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:27.636660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789819+0000","last_change":"2024-04-26T20:07:34.760387+0000","last_active":"2024-04-26T20:09:15.789819+0000","last_peered":"2024-04-26T20:09:15.789819+0000","last_clean":"2024-04-26T20:09:15.789819+0000","last_became_active":"2024-04-26T20:07:34.760042+0000","last_became_peered":"2024-04-26T20:07:34.760042+0000","last_unstale":"2024-04-26T20:09:15.789819+0000","last_undegraded":"2024-04-26T20:09:15.789819+0000","last_fullsized":"2024-04-26T20:09:15.789819+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:39.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.086859+0000","last_change":"2024-04-26T20:08:15.587181+0000","last_active":"2024-04-26T20:09:16.086859+0000","last_peered":"2024-04-26T20:09:16.086859+0000","last_clean":"2024-04-26T20:09:16.086859+0000","last_became_active":"2024-04-26T20:08:15.586847+0000","last_became_peered":"2024-04-26T20:08:15.586847+0000","last_unstale":"2024-04-26T20:09:16.086859+0000","last_undegraded":"2024-04-26T20:09:16.086859+0000","last_fullsized":"2024-04-26T20:09:16.086859+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:12.495523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.790501+0000","last_change":"2024-04-26T20:07:31.711623+0000","last_active":"2024-04-26T20:09:15.790501+0000","last_peered":"2024-04-26T20:09:15.790501+0000","last_clean":"2024-04-26T20:09:15.790501+0000","last_became_active":"2024-04-26T20:07:31.710488+0000","last_became_peered":"2024-04-26T20:07:31.710488+0000","last_unstale":"2024-04-26T20:09:15.790501+0000","last_undegraded":"2024-04-26T20:09:15.790501+0000","last_fullsized":"2024-04-26T20:09:15.790501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:04.747170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.802365+0000","last_change":"2024-04-26T20:09:15.801039+0000","last_active":"2024-04-26T20:09:15.802365+0000","last_peered":"2024-04-26T20:09:15.802365+0000","last_clean":"2024-04-26T20:09:15.802365+0000","last_became_active":"2024-04-26T20:09:15.800049+0000","last_became_peered":"2024-04-26T20:09:15.800049+0000","last_unstale":"2024-04-26T20:09:15.802365+0000","last_undegraded":"2024-04-26T20:09:15.802365+0000","last_fullsized":"2024-04-26T20:09:15.802365+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:49.168625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,9],"acting":[0,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087028+0000","last_change":"2024-04-26T20:07:38.822443+0000","last_active":"2024-04-26T20:09:16.087028+0000","last_peered":"2024-04-26T20:09:16.087028+0000","last_clean":"2024-04-26T20:09:16.087028+0000","last_became_active":"2024-04-26T20:07:38.822046+0000","last_became_peered":"2024-04-26T20:07:38.822046+0000","last_unstale":"2024-04-26T20:09:16.087028+0000","last_undegraded":"2024-04-26T20:09:16.087028+0000","last_fullsized":"2024-04-26T20:09:16.087028+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:26.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364978+0000","last_change":"2024-04-26T20:07:31.715319+0000","last_active":"2024-04-26T20:09:16.364978+0000","last_peered":"2024-04-26T20:09:16.364978+0000","last_clean":"2024-04-26T20:09:16.364978+0000","last_became_active":"2024-04-26T20:07:31.714915+0000","last_became_peered":"2024-04-26T20:07:31.714915+0000","last_unstale":"2024-04-26T20:09:16.364978+0000","last_undegraded":"2024-04-26T20:09:16.364978+0000","last_fullsized":"2024-04-26T20:09:16.364978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:27.898652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266828+0000","last_change":"2024-04-26T20:07:34.753422+0000","last_active":"2024-04-26T20:09:16.266828+0000","last_peered":"2024-04-26T20:09:16.266828+0000","last_clean":"2024-04-26T20:09:16.266828+0000","last_became_active":"2024-04-26T20:07:34.753009+0000","last_became_peered":"2024-04-26T20:07:34.753009+0000","last_unstale":"2024-04-26T20:09:16.266828+0000","last_undegraded":"2024-04-26T20:09:16.266828+0000","last_fullsized":"2024-04-26T20:09:16.266828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.369715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.086930+0000","last_change":"2024-04-26T20:07:38.830239+0000","last_active":"2024-04-26T20:09:16.086930+0000","last_peered":"2024-04-26T20:09:16.086930+0000","last_clean":"2024-04-26T20:09:16.086930+0000","last_became_active":"2024-04-26T20:07:38.829509+0000","last_became_peered":"2024-04-26T20:07:38.829509+0000","last_unstale":"2024-04-26T20:09:16.086930+0000","last_undegraded":"2024-04-26T20:09:16.086930+0000","last_fullsized":"2024-04-26T20:09:16.086930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:53.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788780+0000","last_change":"2024-04-26T20:08:14.576129+0000","last_active":"2024-04-26T20:09:15.788780+0000","last_peered":"2024-04-26T20:09:15.788780+0000","last_clean":"2024-04-26T20:09:15.788780+0000","last_became_active":"2024-04-26T20:08:14.573168+0000","last_became_peered":"2024-04-26T20:08:14.573168+0000","last_unstale":"2024-04-26T20:09:15.788780+0000","last_undegraded":"2024-04-26T20:09:15.788780+0000","last_fullsized":"2024-04-26T20:09:15.788780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:38.234154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176162+0000","last_change":"2024-04-26T20:07:34.749270+0000","last_active":"2024-04-26T20:09:16.176162+0000","last_peered":"2024-04-26T20:09:16.176162+0000","last_clean":"2024-04-26T20:09:16.176162+0000","last_became_active":"2024-04-26T20:07:34.748585+0000","last_became_peered":"2024-04-26T20:07:34.748585+0000","last_unstale":"2024-04-26T20:09:16.176162+0000","last_undegraded":"2024-04-26T20:09:16.176162+0000","last_fullsized":"2024-04-26T20:09:16.176162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:03.091720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787354+0000","last_change":"2024-04-26T20:07:38.806247+0000","last_active":"2024-04-26T20:09:15.787354+0000","last_peered":"2024-04-26T20:09:15.787354+0000","last_clean":"2024-04-26T20:09:15.787354+0000","last_became_active":"2024-04-26T20:07:38.805748+0000","last_became_peered":"2024-04-26T20:07:38.805748+0000","last_unstale":"2024-04-26T20:09:15.787354+0000","last_undegraded":"2024-04-26T20:09:15.787354+0000","last_fullsized":"2024-04-26T20:09:15.787354+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:16.962494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":160,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:41.412901+0000","last_change":"2024-04-26T20:07:31.713618+0000","last_active":"2024-04-26T20:10:41.412901+0000","last_peered":"2024-04-26T20:10:41.412901+0000","last_clean":"2024-04-26T20:10:41.412901+0000","last_became_active":"2024-04-26T20:07:31.713259+0000","last_became_peered":"2024-04-26T20:07:31.713259+0000","last_unstale":"2024-04-26T20:10:41.412901+0000","last_undegraded":"2024-04-26T20:10:41.412901+0000","last_fullsized":"2024-04-26T20:10:41.412901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:25.215087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364422+0000","last_change":"2024-04-26T20:08:14.585688+0000","last_active":"2024-04-26T20:09:16.364422+0000","last_peered":"2024-04-26T20:09:16.364422+0000","last_clean":"2024-04-26T20:09:16.364422+0000","last_became_active":"2024-04-26T20:08:14.585337+0000","last_became_peered":"2024-04-26T20:08:14.585337+0000","last_unstale":"2024-04-26T20:09:16.364422+0000","last_undegraded":"2024-04-26T20:09:16.364422+0000","last_fullsized":"2024-04-26T20:09:16.364422+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:03.515457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786995+0000","last_change":"2024-04-26T20:07:38.829772+0000","last_active":"2024-04-26T20:09:15.786995+0000","last_peered":"2024-04-26T20:09:15.786995+0000","last_clean":"2024-04-26T20:09:15.786995+0000","last_became_active":"2024-04-26T20:07:38.829327+0000","last_became_peered":"2024-04-26T20:07:38.829327+0000","last_unstale":"2024-04-26T20:09:15.786995+0000","last_undegraded":"2024-04-26T20:09:15.786995+0000","last_fullsized":"2024-04-26T20:09:15.786995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:37.252863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"95'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786886+0000","last_change":"2024-04-26T20:07:31.728654+0000","last_active":"2024-04-26T20:09:15.786886+0000","last_peered":"2024-04-26T20:09:15.786886+0000","last_clean":"2024-04-26T20:09:15.786886+0000","last_became_active":"2024-04-26T20:07:31.728278+0000","last_became_peered":"2024-04-26T20:07:31.728278+0000","last_unstale":"2024-04-26T20:09:15.786886+0000","last_undegraded":"2024-04-26T20:09:15.786886+0000","last_fullsized":"2024-04-26T20:09:15.786886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176199+0000","last_change":"2024-04-26T20:07:34.739575+0000","last_active":"2024-04-26T20:09:16.176199+0000","last_peered":"2024-04-26T20:09:16.176199+0000","last_clean":"2024-04-26T20:09:16.176199+0000","last_became_active":"2024-04-26T20:07:34.739264+0000","last_became_peered":"2024-04-26T20:07:34.739264+0000","last_unstale":"2024-04-26T20:09:16.176199+0000","last_undegraded":"2024-04-26T20:09:16.176199+0000","last_fullsized":"2024-04-26T20:09:16.176199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:25.130064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788948+0000","last_change":"2024-04-26T20:07:38.822669+0000","last_active":"2024-04-26T20:09:15.788948+0000","last_peered":"2024-04-26T20:09:15.788948+0000","last_clean":"2024-04-26T20:09:15.788948+0000","last_became_active":"2024-04-26T20:07:38.822197+0000","last_became_peered":"2024-04-26T20:07:38.822197+0000","last_unstale":"2024-04-26T20:09:15.788948+0000","last_undegraded":"2024-04-26T20:09:15.788948+0000","last_fullsized":"2024-04-26T20:09:15.788948+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:56.782794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"95'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313329+0000","last_change":"2024-04-26T20:07:31.723527+0000","last_active":"2024-04-26T20:09:16.313329+0000","last_peered":"2024-04-26T20:09:16.313329+0000","last_clean":"2024-04-26T20:09:16.313329+0000","last_became_active":"2024-04-26T20:07:31.722574+0000","last_became_peered":"2024-04-26T20:07:31.722574+0000","last_unstale":"2024-04-26T20:09:16.313329+0000","last_undegraded":"2024-04-26T20:09:16.313329+0000","last_fullsized":"2024-04-26T20:09:16.313329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.521613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789625+0000","last_change":"2024-04-26T20:07:34.771459+0000","last_active":"2024-04-26T20:09:15.789625+0000","last_peered":"2024-04-26T20:09:15.789625+0000","last_clean":"2024-04-26T20:09:15.789625+0000","last_became_active":"2024-04-26T20:07:34.771124+0000","last_became_peered":"2024-04-26T20:07:34.771124+0000","last_unstale":"2024-04-26T20:09:15.789625+0000","last_undegraded":"2024-04-26T20:09:15.789625+0000","last_fullsized":"2024-04-26T20:09:15.789625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.560440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.800310+0000","last_change":"2024-04-26T20:07:38.838858+0000","last_active":"2024-04-26T20:09:15.800310+0000","last_peered":"2024-04-26T20:09:15.800310+0000","last_clean":"2024-04-26T20:09:15.800310+0000","last_became_active":"2024-04-26T20:07:38.838125+0000","last_became_peered":"2024-04-26T20:07:38.838125+0000","last_unstale":"2024-04-26T20:09:15.800310+0000","last_undegraded":"2024-04-26T20:09:15.800310+0000","last_fullsized":"2024-04-26T20:09:15.800310+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:01.856633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"95'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176407+0000","last_change":"2024-04-26T20:07:31.710690+0000","last_active":"2024-04-26T20:09:16.176407+0000","last_peered":"2024-04-26T20:09:16.176407+0000","last_clean":"2024-04-26T20:09:16.176407+0000","last_became_active":"2024-04-26T20:07:31.710353+0000","last_became_peered":"2024-04-26T20:07:31.710353+0000","last_unstale":"2024-04-26T20:09:16.176407+0000","last_undegraded":"2024-04-26T20:09:16.176407+0000","last_fullsized":"2024-04-26T20:09:16.176407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:47.442673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791085+0000","last_change":"2024-04-26T20:07:34.776023+0000","last_active":"2024-04-26T20:09:15.791085+0000","last_peered":"2024-04-26T20:09:15.791085+0000","last_clean":"2024-04-26T20:09:15.791085+0000","last_became_active":"2024-04-26T20:07:34.775571+0000","last_became_peered":"2024-04-26T20:07:34.775571+0000","last_unstale":"2024-04-26T20:09:15.791085+0000","last_undegraded":"2024-04-26T20:09:15.791085+0000","last_fullsized":"2024-04-26T20:09:15.791085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:40.573046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788839+0000","last_change":"2024-04-26T20:07:38.820636+0000","last_active":"2024-04-26T20:09:15.788839+0000","last_peered":"2024-04-26T20:09:15.788839+0000","last_clean":"2024-04-26T20:09:15.788839+0000","last_became_active":"2024-04-26T20:07:38.819390+0000","last_became_peered":"2024-04-26T20:07:38.819390+0000","last_unstale":"2024-04-26T20:09:15.788839+0000","last_undegraded":"2024-04-26T20:09:15.788839+0000","last_fullsized":"2024-04-26T20:09:15.788839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:58.793479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'3","reported_seq":153,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:41.412265+0000","last_change":"2024-04-26T20:07:31.724192+0000","last_active":"2024-04-26T20:10:41.412265+0000","last_peered":"2024-04-26T20:10:41.412265+0000","last_clean":"2024-04-26T20:10:41.412265+0000","last_became_active":"2024-04-26T20:07:31.723816+0000","last_became_peered":"2024-04-26T20:07:31.723816+0000","last_unstale":"2024-04-26T20:10:41.412265+0000","last_undegraded":"2024-04-26T20:10:41.412265+0000","last_fullsized":"2024-04-26T20:10:41.412265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:31.707282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.267056+0000","last_change":"2024-04-26T20:07:34.759027+0000","last_active":"2024-04-26T20:09:16.267056+0000","last_peered":"2024-04-26T20:09:16.267056+0000","last_clean":"2024-04-26T20:09:16.267056+0000","last_became_active":"2024-04-26T20:07:34.758689+0000","last_became_peered":"2024-04-26T20:07:34.758689+0000","last_unstale":"2024-04-26T20:09:16.267056+0000","last_undegraded":"2024-04-26T20:09:16.267056+0000","last_fullsized":"2024-04-26T20:09:16.267056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:29.110222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791568+0000","last_change":"2024-04-26T20:07:38.819000+0000","last_active":"2024-04-26T20:09:15.791568+0000","last_peered":"2024-04-26T20:09:15.791568+0000","last_clean":"2024-04-26T20:09:15.791568+0000","last_became_active":"2024-04-26T20:07:38.818526+0000","last_became_peered":"2024-04-26T20:07:38.818526+0000","last_unstale":"2024-04-26T20:09:15.791568+0000","last_undegraded":"2024-04-26T20:09:15.791568+0000","last_fullsized":"2024-04-26T20:09:15.791568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:28.830096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864231+0000","last_change":"2024-04-26T20:07:31.712952+0000","last_active":"2024-04-26T20:09:15.864231+0000","last_peered":"2024-04-26T20:09:15.864231+0000","last_clean":"2024-04-26T20:09:15.864231+0000","last_became_active":"2024-04-26T20:07:31.712081+0000","last_became_peered":"2024-04-26T20:07:31.712081+0000","last_unstale":"2024-04-26T20:09:15.864231+0000","last_undegraded":"2024-04-26T20:09:15.864231+0000","last_fullsized":"2024-04-26T20:09:15.864231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:46.923536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.368893+0000","last_change":"2024-04-26T20:09:16.367690+0000","last_active":"2024-04-26T20:09:16.368893+0000","last_peered":"2024-04-26T20:09:16.368893+0000","last_clean":"2024-04-26T20:09:16.368893+0000","last_became_active":"2024-04-26T20:09:16.366496+0000","last_became_peered":"2024-04-26T20:09:16.366496+0000","last_unstale":"2024-04-26T20:09:16.368893+0000","last_undegraded":"2024-04-26T20:09:16.368893+0000","last_fullsized":"2024-04-26T20:09:16.368893+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:36.154062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087235+0000","last_change":"2024-04-26T20:07:38.822520+0000","last_active":"2024-04-26T20:09:16.087235+0000","last_peered":"2024-04-26T20:09:16.087235+0000","last_clean":"2024-04-26T20:09:16.087235+0000","last_became_active":"2024-04-26T20:07:38.822056+0000","last_became_peered":"2024-04-26T20:07:38.822056+0000","last_unstale":"2024-04-26T20:09:16.087235+0000","last_undegraded":"2024-04-26T20:09:16.087235+0000","last_fullsized":"2024-04-26T20:09:16.087235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.324996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"95'4","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176248+0000","last_change":"2024-04-26T20:07:31.724015+0000","last_active":"2024-04-26T20:09:16.176248+0000","last_peered":"2024-04-26T20:09:16.176248+0000","last_clean":"2024-04-26T20:09:16.176248+0000","last_became_active":"2024-04-26T20:07:31.723686+0000","last_became_peered":"2024-04-26T20:07:31.723686+0000","last_unstale":"2024-04-26T20:09:16.176248+0000","last_undegraded":"2024-04-26T20:09:16.176248+0000","last_fullsized":"2024-04-26T20:09:16.176248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:55.925280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266959+0000","last_change":"2024-04-26T20:07:34.750276+0000","last_active":"2024-04-26T20:09:16.266959+0000","last_peered":"2024-04-26T20:09:16.266959+0000","last_clean":"2024-04-26T20:09:16.266959+0000","last_became_active":"2024-04-26T20:07:34.749716+0000","last_became_peered":"2024-04-26T20:07:34.749716+0000","last_unstale":"2024-04-26T20:09:16.266959+0000","last_undegraded":"2024-04-26T20:09:16.266959+0000","last_fullsized":"2024-04-26T20:09:16.266959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:22.429780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791432+0000","last_change":"2024-04-26T20:07:38.815636+0000","last_active":"2024-04-26T20:09:15.791432+0000","last_peered":"2024-04-26T20:09:15.791432+0000","last_clean":"2024-04-26T20:09:15.791432+0000","last_became_active":"2024-04-26T20:07:38.815278+0000","last_became_peered":"2024-04-26T20:07:38.815278+0000","last_unstale":"2024-04-26T20:09:15.791432+0000","last_undegraded":"2024-04-26T20:09:15.791432+0000","last_fullsized":"2024-04-26T20:09:15.791432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:27.522132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'33","reported_seq":302,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:42.413853+0000","last_change":"2024-04-26T20:07:31.711393+0000","last_active":"2024-04-26T20:10:42.413853+0000","last_peered":"2024-04-26T20:10:42.413853+0000","last_clean":"2024-04-26T20:10:42.413853+0000","last_became_active":"2024-04-26T20:07:31.710998+0000","last_became_peered":"2024-04-26T20:07:31.710998+0000","last_unstale":"2024-04-26T20:10:42.413853+0000","last_undegraded":"2024-04-26T20:10:42.413853+0000","last_fullsized":"2024-04-26T20:10:42.413853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:00.674950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56115,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":169,"num_write":34,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364772+0000","last_change":"2024-04-26T20:07:34.752774+0000","last_active":"2024-04-26T20:09:16.364772+0000","last_peered":"2024-04-26T20:09:16.364772+0000","last_clean":"2024-04-26T20:09:16.364772+0000","last_became_active":"2024-04-26T20:07:34.752471+0000","last_became_peered":"2024-04-26T20:07:34.752471+0000","last_unstale":"2024-04-26T20:09:16.364772+0000","last_undegraded":"2024-04-26T20:09:16.364772+0000","last_fullsized":"2024-04-26T20:09:16.364772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.646231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176573+0000","last_change":"2024-04-26T20:07:38.830322+0000","last_active":"2024-04-26T20:09:16.176573+0000","last_peered":"2024-04-26T20:09:16.176573+0000","last_clean":"2024-04-26T20:09:16.176573+0000","last_became_active":"2024-04-26T20:07:38.829910+0000","last_became_peered":"2024-04-26T20:07:38.829910+0000","last_unstale":"2024-04-26T20:09:16.176573+0000","last_undegraded":"2024-04-26T20:09:16.176573+0000","last_fullsized":"2024-04-26T20:09:16.176573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:01.979793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"95'6","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787229+0000","last_change":"2024-04-26T20:07:31.712054+0000","last_active":"2024-04-26T20:09:15.787229+0000","last_peered":"2024-04-26T20:09:15.787229+0000","last_clean":"2024-04-26T20:09:15.787229+0000","last_became_active":"2024-04-26T20:07:31.711618+0000","last_became_peered":"2024-04-26T20:07:31.711618+0000","last_unstale":"2024-04-26T20:09:15.787229+0000","last_undegraded":"2024-04-26T20:09:15.787229+0000","last_fullsized":"2024-04-26T20:09:15.787229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:46.465777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313432+0000","last_change":"2024-04-26T20:07:34.772993+0000","last_active":"2024-04-26T20:09:16.313432+0000","last_peered":"2024-04-26T20:09:16.313432+0000","last_clean":"2024-04-26T20:09:16.313432+0000","last_became_active":"2024-04-26T20:07:34.772587+0000","last_became_peered":"2024-04-26T20:07:34.772587+0000","last_unstale":"2024-04-26T20:09:16.313432+0000","last_undegraded":"2024-04-26T20:09:16.313432+0000","last_fullsized":"2024-04-26T20:09:16.313432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:38.459869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788987+0000","last_change":"2024-04-26T20:08:15.588307+0000","last_active":"2024-04-26T20:09:15.788987+0000","last_peered":"2024-04-26T20:09:15.788987+0000","last_clean":"2024-04-26T20:09:15.788987+0000","last_became_active":"2024-04-26T20:08:15.587978+0000","last_became_peered":"2024-04-26T20:08:15.587978+0000","last_unstale":"2024-04-26T20:09:15.788987+0000","last_undegraded":"2024-04-26T20:09:15.788987+0000","last_fullsized":"2024-04-26T20:09:15.788987+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:36.491892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175976+0000","last_change":"2024-04-26T20:07:31.720938+0000","last_active":"2024-04-26T20:09:16.175976+0000","last_peered":"2024-04-26T20:09:16.175976+0000","last_clean":"2024-04-26T20:09:16.175976+0000","last_became_active":"2024-04-26T20:07:31.720579+0000","last_became_peered":"2024-04-26T20:07:31.720579+0000","last_unstale":"2024-04-26T20:09:16.175976+0000","last_undegraded":"2024-04-26T20:09:16.175976+0000","last_fullsized":"2024-04-26T20:09:16.175976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:55.737976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864109+0000","last_change":"2024-04-26T20:07:34.764802+0000","last_active":"2024-04-26T20:09:15.864109+0000","last_peered":"2024-04-26T20:09:15.864109+0000","last_clean":"2024-04-26T20:09:15.864109+0000","last_became_active":"2024-04-26T20:07:34.763468+0000","last_became_peered":"2024-04-26T20:07:34.763468+0000","last_unstale":"2024-04-26T20:09:15.864109+0000","last_undegraded":"2024-04-26T20:09:15.864109+0000","last_fullsized":"2024-04-26T20:09:15.864109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:23.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793284+0000","last_change":"2024-04-26T20:07:38.830392+0000","last_active":"2024-04-26T20:09:15.793284+0000","last_peered":"2024-04-26T20:09:15.793284+0000","last_clean":"2024-04-26T20:09:15.793284+0000","last_became_active":"2024-04-26T20:07:38.822964+0000","last_became_peered":"2024-04-26T20:07:38.822964+0000","last_unstale":"2024-04-26T20:09:15.793284+0000","last_undegraded":"2024-04-26T20:09:15.793284+0000","last_fullsized":"2024-04-26T20:09:15.793284+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:17.992368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'3","reported_seq":154,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:40.497521+0000","last_change":"2024-04-26T20:07:31.710567+0000","last_active":"2024-04-26T20:10:40.497521+0000","last_peered":"2024-04-26T20:10:40.497521+0000","last_clean":"2024-04-26T20:10:40.497521+0000","last_became_active":"2024-04-26T20:07:31.710197+0000","last_became_peered":"2024-04-26T20:07:31.710197+0000","last_unstale":"2024-04-26T20:10:40.497521+0000","last_undegraded":"2024-04-26T20:10:40.497521+0000","last_fullsized":"2024-04-26T20:10:40.497521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:10.648320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863839+0000","last_change":"2024-04-26T20:07:34.743237+0000","last_active":"2024-04-26T20:09:15.863839+0000","last_peered":"2024-04-26T20:09:15.863839+0000","last_clean":"2024-04-26T20:09:15.863839+0000","last_became_active":"2024-04-26T20:07:34.742146+0000","last_became_peered":"2024-04-26T20:07:34.742146+0000","last_unstale":"2024-04-26T20:09:15.863839+0000","last_undegraded":"2024-04-26T20:09:15.863839+0000","last_fullsized":"2024-04-26T20:09:15.863839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:15.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314505+0000","last_change":"2024-04-26T20:08:15.591942+0000","last_active":"2024-04-26T20:09:16.314505+0000","last_peered":"2024-04-26T20:09:16.314505+0000","last_clean":"2024-04-26T20:09:16.314505+0000","last_became_active":"2024-04-26T20:08:15.590842+0000","last_became_peered":"2024-04-26T20:08:15.590842+0000","last_unstale":"2024-04-26T20:09:16.314505+0000","last_undegraded":"2024-04-26T20:09:16.314505+0000","last_fullsized":"2024-04-26T20:09:16.314505+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:44.344152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"95'15","reported_seq":56,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364235+0000","last_change":"2024-04-26T20:07:31.716877+0000","last_active":"2024-04-26T20:09:16.364235+0000","last_peered":"2024-04-26T20:09:16.364235+0000","last_clean":"2024-04-26T20:09:16.364235+0000","last_became_active":"2024-04-26T20:07:31.716560+0000","last_became_peered":"2024-04-26T20:07:31.716560+0000","last_unstale":"2024-04-26T20:09:16.364235+0000","last_undegraded":"2024-04-26T20:09:16.364235+0000","last_fullsized":"2024-04-26T20:09:16.364235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:08:47.680991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175825+0000","last_change":"2024-04-26T20:07:34.749893+0000","last_active":"2024-04-26T20:09:16.175825+0000","last_peered":"2024-04-26T20:09:16.175825+0000","last_clean":"2024-04-26T20:09:16.175825+0000","last_became_active":"2024-04-26T20:07:34.748535+0000","last_became_peered":"2024-04-26T20:07:34.748535+0000","last_unstale":"2024-04-26T20:09:16.175825+0000","last_undegraded":"2024-04-26T20:09:16.175825+0000","last_fullsized":"2024-04-26T20:09:16.175825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:32.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175765+0000","last_change":"2024-04-26T20:07:38.824046+0000","last_active":"2024-04-26T20:09:16.175765+0000","last_peered":"2024-04-26T20:09:16.175765+0000","last_clean":"2024-04-26T20:09:16.175765+0000","last_became_active":"2024-04-26T20:07:38.823549+0000","last_became_peered":"2024-04-26T20:07:38.823549+0000","last_unstale":"2024-04-26T20:09:16.175765+0000","last_undegraded":"2024-04-26T20:09:16.175765+0000","last_fullsized":"2024-04-26T20:09:16.175765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:26.218201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.365016+0000","last_change":"2024-04-26T20:07:31.711968+0000","last_active":"2024-04-26T20:09:16.365016+0000","last_peered":"2024-04-26T20:09:16.365016+0000","last_clean":"2024-04-26T20:09:16.365016+0000","last_became_active":"2024-04-26T20:07:31.710982+0000","last_became_peered":"2024-04-26T20:07:31.710982+0000","last_unstale":"2024-04-26T20:09:16.365016+0000","last_undegraded":"2024-04-26T20:09:16.365016+0000","last_fullsized":"2024-04-26T20:09:16.365016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:29.526536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789172+0000","last_change":"2024-04-26T20:07:34.758101+0000","last_active":"2024-04-26T20:09:15.789172+0000","last_peered":"2024-04-26T20:09:15.789172+0000","last_clean":"2024-04-26T20:09:15.789172+0000","last_became_active":"2024-04-26T20:07:34.757747+0000","last_became_peered":"2024-04-26T20:07:34.757747+0000","last_unstale":"2024-04-26T20:09:15.789172+0000","last_undegraded":"2024-04-26T20:09:15.789172+0000","last_fullsized":"2024-04-26T20:09:15.789172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.337040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364569+0000","last_change":"2024-04-26T20:07:39.312976+0000","last_active":"2024-04-26T20:09:16.364569+0000","last_peered":"2024-04-26T20:09:16.364569+0000","last_clean":"2024-04-26T20:09:16.364569+0000","last_became_active":"2024-04-26T20:07:39.312443+0000","last_became_peered":"2024-04-26T20:07:39.312443+0000","last_unstale":"2024-04-26T20:09:16.364569+0000","last_undegraded":"2024-04-26T20:09:16.364569+0000","last_fullsized":"2024-04-26T20:09:16.364569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:23.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.792536+0000","last_change":"2024-04-26T20:08:14.587592+0000","last_active":"2024-04-26T20:09:15.792536+0000","last_peered":"2024-04-26T20:09:15.792536+0000","last_clean":"2024-04-26T20:09:15.792536+0000","last_became_active":"2024-04-26T20:08:14.587157+0000","last_became_peered":"2024-04-26T20:08:14.587157+0000","last_unstale":"2024-04-26T20:09:15.792536+0000","last_undegraded":"2024-04-26T20:09:15.792536+0000","last_fullsized":"2024-04-26T20:09:15.792536+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:22.184416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787061+0000","last_change":"2024-04-26T20:07:34.764376+0000","last_active":"2024-04-26T20:09:15.787061+0000","last_peered":"2024-04-26T20:09:15.787061+0000","last_clean":"2024-04-26T20:09:15.787061+0000","last_became_active":"2024-04-26T20:07:34.764040+0000","last_became_peered":"2024-04-26T20:07:34.764040+0000","last_unstale":"2024-04-26T20:09:15.787061+0000","last_undegraded":"2024-04-26T20:09:15.787061+0000","last_fullsized":"2024-04-26T20:09:15.787061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:43.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313757+0000","last_change":"2024-04-26T20:07:38.837075+0000","last_active":"2024-04-26T20:09:16.313757+0000","last_peered":"2024-04-26T20:09:16.313757+0000","last_clean":"2024-04-26T20:09:16.313757+0000","last_became_active":"2024-04-26T20:07:38.835912+0000","last_became_peered":"2024-04-26T20:07:38.835912+0000","last_unstale":"2024-04-26T20:09:16.313757+0000","last_undegraded":"2024-04-26T20:09:16.313757+0000","last_fullsized":"2024-04-26T20:09:16.313757+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.853262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863906+0000","last_change":"2024-04-26T20:07:31.728087+0000","last_active":"2024-04-26T20:09:15.863906+0000","last_peered":"2024-04-26T20:09:15.863906+0000","last_clean":"2024-04-26T20:09:15.863906+0000","last_became_active":"2024-04-26T20:07:31.727688+0000","last_became_peered":"2024-04-26T20:07:31.727688+0000","last_unstale":"2024-04-26T20:09:15.863906+0000","last_undegraded":"2024-04-26T20:09:15.863906+0000","last_fullsized":"2024-04-26T20:09:15.863906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:16.220049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787166+0000","last_change":"2024-04-26T20:07:34.765161+0000","last_active":"2024-04-26T20:09:15.787166+0000","last_peered":"2024-04-26T20:09:15.787166+0000","last_clean":"2024-04-26T20:09:15.787166+0000","last_became_active":"2024-04-26T20:07:34.764789+0000","last_became_peered":"2024-04-26T20:07:34.764789+0000","last_unstale":"2024-04-26T20:09:15.787166+0000","last_undegraded":"2024-04-26T20:09:15.787166+0000","last_fullsized":"2024-04-26T20:09:15.787166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:52.417157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314242+0000","last_change":"2024-04-26T20:07:39.313159+0000","last_active":"2024-04-26T20:09:16.314242+0000","last_peered":"2024-04-26T20:09:16.314242+0000","last_clean":"2024-04-26T20:09:16.314242+0000","last_became_active":"2024-04-26T20:07:39.312637+0000","last_became_peered":"2024-04-26T20:07:39.312637+0000","last_unstale":"2024-04-26T20:09:16.314242+0000","last_undegraded":"2024-04-26T20:09:16.314242+0000","last_fullsized":"2024-04-26T20:09:16.314242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:10.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176663+0000","last_change":"2024-04-26T20:08:14.586352+0000","last_active":"2024-04-26T20:09:16.176663+0000","last_peered":"2024-04-26T20:09:16.176663+0000","last_clean":"2024-04-26T20:09:16.176663+0000","last_became_active":"2024-04-26T20:08:14.585975+0000","last_became_peered":"2024-04-26T20:08:14.585975+0000","last_unstale":"2024-04-26T20:09:16.176663+0000","last_undegraded":"2024-04-26T20:09:16.176663+0000","last_fullsized":"2024-04-26T20:09:16.176663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:27.676683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'4","reported_seq":153,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:41.498997+0000","last_change":"2024-04-26T20:07:31.726272+0000","last_active":"2024-04-26T20:10:41.498997+0000","last_peered":"2024-04-26T20:10:41.498997+0000","last_clean":"2024-04-26T20:10:41.498997+0000","last_became_active":"2024-04-26T20:07:31.725889+0000","last_became_peered":"2024-04-26T20:07:31.725889+0000","last_unstale":"2024-04-26T20:10:41.498997+0000","last_undegraded":"2024-04-26T20:10:41.498997+0000","last_fullsized":"2024-04-26T20:10:41.498997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:03.091138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864225+0000","last_change":"2024-04-26T20:07:38.815171+0000","last_active":"2024-04-26T20:09:15.864225+0000","last_peered":"2024-04-26T20:09:15.864225+0000","last_clean":"2024-04-26T20:09:15.864225+0000","last_became_active":"2024-04-26T20:07:38.814769+0000","last_became_peered":"2024-04-26T20:07:38.814769+0000","last_unstale":"2024-04-26T20:09:15.864225+0000","last_undegraded":"2024-04-26T20:09:15.864225+0000","last_fullsized":"2024-04-26T20:09:15.864225+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789245+0000","last_change":"2024-04-26T20:07:34.766538+0000","last_active":"2024-04-26T20:09:15.789245+0000","last_peered":"2024-04-26T20:09:15.789245+0000","last_clean":"2024-04-26T20:09:15.789245+0000","last_became_active":"2024-04-26T20:07:34.761651+0000","last_became_peered":"2024-04-26T20:07:34.761651+0000","last_unstale":"2024-04-26T20:09:15.789245+0000","last_undegraded":"2024-04-26T20:09:15.789245+0000","last_fullsized":"2024-04-26T20:09:15.789245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:11.213046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087329+0000","last_change":"2024-04-26T20:07:31.720351+0000","last_active":"2024-04-26T20:09:16.087329+0000","last_peered":"2024-04-26T20:09:16.087329+0000","last_clean":"2024-04-26T20:09:16.087329+0000","last_became_active":"2024-04-26T20:07:31.719966+0000","last_became_peered":"2024-04-26T20:07:31.719966+0000","last_unstale":"2024-04-26T20:09:16.087329+0000","last_undegraded":"2024-04-26T20:09:16.087329+0000","last_fullsized":"2024-04-26T20:09:16.087329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:17.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863629+0000","last_change":"2024-04-26T20:05:56.337783+0000","last_active":"2024-04-26T20:09:15.863629+0000","last_peered":"2024-04-26T20:09:15.863629+0000","last_clean":"2024-04-26T20:09:15.863629+0000","last_became_active":"2024-04-26T20:05:56.337301+0000","last_became_peered":"2024-04-26T20:05:56.337301+0000","last_unstale":"2024-04-26T20:09:15.863629+0000","last_undegraded":"2024-04-26T20:09:15.863629+0000","last_fullsized":"2024-04-26T20:09:15.863629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_clean_scrub_stamp":"2024-04-26T20:01:58.332403+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:45.808606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176441+0000","last_change":"2024-04-26T20:07:39.313210+0000","last_active":"2024-04-26T20:09:16.176441+0000","last_peered":"2024-04-26T20:09:16.176441+0000","last_clean":"2024-04-26T20:09:16.176441+0000","last_became_active":"2024-04-26T20:07:39.312575+0000","last_became_peered":"2024-04-26T20:07:39.312575+0000","last_unstale":"2024-04-26T20:09:16.176441+0000","last_undegraded":"2024-04-26T20:09:16.176441+0000","last_fullsized":"2024-04-26T20:09:16.176441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:02.877951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313066+0000","last_change":"2024-04-26T20:07:34.759474+0000","last_active":"2024-04-26T20:09:16.313066+0000","last_peered":"2024-04-26T20:09:16.313066+0000","last_clean":"2024-04-26T20:09:16.313066+0000","last_became_active":"2024-04-26T20:07:34.759127+0000","last_became_peered":"2024-04-26T20:07:34.759127+0000","last_unstale":"2024-04-26T20:09:16.313066+0000","last_undegraded":"2024-04-26T20:09:16.313066+0000","last_fullsized":"2024-04-26T20:09:16.313066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.874866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863782+0000","last_change":"2024-04-26T20:07:31.710633+0000","last_active":"2024-04-26T20:09:15.863782+0000","last_peered":"2024-04-26T20:09:15.863782+0000","last_clean":"2024-04-26T20:09:15.863782+0000","last_became_active":"2024-04-26T20:07:31.710290+0000","last_became_peered":"2024-04-26T20:07:31.710290+0000","last_unstale":"2024-04-26T20:09:15.863782+0000","last_undegraded":"2024-04-26T20:09:15.863782+0000","last_fullsized":"2024-04-26T20:09:15.863782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:18.114770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788979+0000","last_change":"2024-04-26T20:07:38.829768+0000","last_active":"2024-04-26T20:09:15.788979+0000","last_peered":"2024-04-26T20:09:15.788979+0000","last_clean":"2024-04-26T20:09:15.788979+0000","last_became_active":"2024-04-26T20:07:38.829364+0000","last_became_peered":"2024-04-26T20:07:38.829364+0000","last_unstale":"2024-04-26T20:09:15.788979+0000","last_undegraded":"2024-04-26T20:09:15.788979+0000","last_fullsized":"2024-04-26T20:09:15.788979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:56.887062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"95'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364841+0000","last_change":"2024-04-26T20:07:31.720296+0000","last_active":"2024-04-26T20:09:16.364841+0000","last_peered":"2024-04-26T20:09:16.364841+0000","last_clean":"2024-04-26T20:09:16.364841+0000","last_became_active":"2024-04-26T20:07:31.719918+0000","last_became_peered":"2024-04-26T20:07:31.719918+0000","last_unstale":"2024-04-26T20:09:16.364841+0000","last_undegraded":"2024-04-26T20:09:16.364841+0000","last_fullsized":"2024-04-26T20:09:16.364841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:43.783104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314016+0000","last_change":"2024-04-26T20:07:34.770634+0000","last_active":"2024-04-26T20:09:16.314016+0000","last_peered":"2024-04-26T20:09:16.314016+0000","last_clean":"2024-04-26T20:09:16.314016+0000","last_became_active":"2024-04-26T20:07:34.770170+0000","last_became_peered":"2024-04-26T20:07:34.770170+0000","last_unstale":"2024-04-26T20:09:16.314016+0000","last_undegraded":"2024-04-26T20:09:16.314016+0000","last_fullsized":"2024-04-26T20:09:16.314016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:21.528735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.790102+0000","last_change":"2024-04-26T20:07:39.317345+0000","last_active":"2024-04-26T20:09:15.790102+0000","last_peered":"2024-04-26T20:09:15.790102+0000","last_clean":"2024-04-26T20:09:15.790102+0000","last_became_active":"2024-04-26T20:07:39.315998+0000","last_became_peered":"2024-04-26T20:07:39.315998+0000","last_unstale":"2024-04-26T20:09:15.790102+0000","last_undegraded":"2024-04-26T20:09:15.790102+0000","last_fullsized":"2024-04-26T20:09:15.790102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:18.715213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'10","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:40.712959+0000","last_change":"2024-04-26T20:07:31.729195+0000","last_active":"2024-04-26T20:09:40.712959+0000","last_peered":"2024-04-26T20:09:40.712959+0000","last_clean":"2024-04-26T20:09:40.712959+0000","last_became_active":"2024-04-26T20:07:31.728746+0000","last_became_peered":"2024-04-26T20:07:31.728746+0000","last_unstale":"2024-04-26T20:09:40.712959+0000","last_undegraded":"2024-04-26T20:09:40.712959+0000","last_fullsized":"2024-04-26T20:09:40.712959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:34.443091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791405+0000","last_change":"2024-04-26T20:07:34.765013+0000","last_active":"2024-04-26T20:09:15.791405+0000","last_peered":"2024-04-26T20:09:15.791405+0000","last_clean":"2024-04-26T20:09:15.791405+0000","last_became_active":"2024-04-26T20:07:34.764597+0000","last_became_peered":"2024-04-26T20:07:34.764597+0000","last_unstale":"2024-04-26T20:09:15.791405+0000","last_undegraded":"2024-04-26T20:09:15.791405+0000","last_fullsized":"2024-04-26T20:09:15.791405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:51.513026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786785+0000","last_change":"2024-04-26T20:07:38.805910+0000","last_active":"2024-04-26T20:09:15.786785+0000","last_peered":"2024-04-26T20:09:15.786785+0000","last_clean":"2024-04-26T20:09:15.786785+0000","last_became_active":"2024-04-26T20:07:38.804580+0000","last_became_peered":"2024-04-26T20:07:38.804580+0000","last_unstale":"2024-04-26T20:09:15.786785+0000","last_undegraded":"2024-04-26T20:09:15.786785+0000","last_fullsized":"2024-04-26T20:09:15.786785+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:41.321084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"95'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.086791+0000","last_change":"2024-04-26T20:07:31.915771+0000","last_active":"2024-04-26T20:09:16.086791+0000","last_peered":"2024-04-26T20:09:16.086791+0000","last_clean":"2024-04-26T20:09:16.086791+0000","last_became_active":"2024-04-26T20:07:31.915422+0000","last_became_peered":"2024-04-26T20:07:31.915422+0000","last_unstale":"2024-04-26T20:09:16.086791+0000","last_undegraded":"2024-04-26T20:09:16.086791+0000","last_fullsized":"2024-04-26T20:09:16.086791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:33.309270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.090395+0000","last_change":"2024-04-26T20:09:16.089155+0000","last_active":"2024-04-26T20:09:16.090395+0000","last_peered":"2024-04-26T20:09:16.090395+0000","last_clean":"2024-04-26T20:09:16.090395+0000","last_became_active":"2024-04-26T20:09:16.088425+0000","last_became_peered":"2024-04-26T20:09:16.088425+0000","last_unstale":"2024-04-26T20:09:16.090395+0000","last_undegraded":"2024-04-26T20:09:16.090395+0000","last_fullsized":"2024-04-26T20:09:16.090395+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:00.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087281+0000","last_change":"2024-04-26T20:07:39.313079+0000","last_active":"2024-04-26T20:09:16.087281+0000","last_peered":"2024-04-26T20:09:16.087281+0000","last_clean":"2024-04-26T20:09:16.087281+0000","last_became_active":"2024-04-26T20:07:39.312603+0000","last_became_peered":"2024-04-26T20:07:39.312603+0000","last_unstale":"2024-04-26T20:09:16.087281+0000","last_undegraded":"2024-04-26T20:09:16.087281+0000","last_fullsized":"2024-04-26T20:09:16.087281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:45.717375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266559+0000","last_change":"2024-04-26T20:07:31.715373+0000","last_active":"2024-04-26T20:09:16.266559+0000","last_peered":"2024-04-26T20:09:16.266559+0000","last_clean":"2024-04-26T20:09:16.266559+0000","last_became_active":"2024-04-26T20:07:31.714973+0000","last_became_peered":"2024-04-26T20:07:31.714973+0000","last_unstale":"2024-04-26T20:09:16.266559+0000","last_undegraded":"2024-04-26T20:09:16.266559+0000","last_fullsized":"2024-04-26T20:09:16.266559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.772671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364255+0000","last_change":"2024-04-26T20:07:34.753619+0000","last_active":"2024-04-26T20:09:16.364255+0000","last_peered":"2024-04-26T20:09:16.364255+0000","last_clean":"2024-04-26T20:09:16.364255+0000","last_became_active":"2024-04-26T20:07:34.753263+0000","last_became_peered":"2024-04-26T20:07:34.753263+0000","last_unstale":"2024-04-26T20:09:16.364255+0000","last_undegraded":"2024-04-26T20:09:16.364255+0000","last_fullsized":"2024-04-26T20:09:16.364255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.358153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.793451+0000","last_change":"2024-04-26T20:08:15.840243+0000","last_active":"2024-04-26T20:09:15.793451+0000","last_peered":"2024-04-26T20:09:15.793451+0000","last_clean":"2024-04-26T20:09:15.793451+0000","last_became_active":"2024-04-26T20:08:15.838891+0000","last_became_peered":"2024-04-26T20:08:15.838891+0000","last_unstale":"2024-04-26T20:09:15.793451+0000","last_undegraded":"2024-04-26T20:09:15.793451+0000","last_fullsized":"2024-04-26T20:09:15.793451+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:15.018199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788938+0000","last_change":"2024-04-26T20:07:31.722516+0000","last_active":"2024-04-26T20:09:15.788938+0000","last_peered":"2024-04-26T20:09:15.788938+0000","last_clean":"2024-04-26T20:09:15.788938+0000","last_became_active":"2024-04-26T20:07:31.722096+0000","last_became_peered":"2024-04-26T20:07:31.722096+0000","last_unstale":"2024-04-26T20:09:15.788938+0000","last_undegraded":"2024-04-26T20:09:15.788938+0000","last_fullsized":"2024-04-26T20:09:15.788938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:57.693947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788865+0000","last_change":"2024-04-26T20:07:34.768754+0000","last_active":"2024-04-26T20:09:15.788865+0000","last_peered":"2024-04-26T20:09:15.788865+0000","last_clean":"2024-04-26T20:09:15.788865+0000","last_became_active":"2024-04-26T20:07:34.768405+0000","last_became_peered":"2024-04-26T20:07:34.768405+0000","last_unstale":"2024-04-26T20:09:15.788865+0000","last_undegraded":"2024-04-26T20:09:15.788865+0000","last_fullsized":"2024-04-26T20:09:15.788865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:08.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087279+0000","last_change":"2024-04-26T20:07:38.828804+0000","last_active":"2024-04-26T20:09:16.087279+0000","last_peered":"2024-04-26T20:09:16.087279+0000","last_clean":"2024-04-26T20:09:16.087279+0000","last_became_active":"2024-04-26T20:07:38.828452+0000","last_became_peered":"2024-04-26T20:07:38.828452+0000","last_unstale":"2024-04-26T20:09:16.087279+0000","last_undegraded":"2024-04-26T20:09:16.087279+0000","last_fullsized":"2024-04-26T20:09:16.087279+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:29.905102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'1","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787059+0000","last_change":"2024-04-26T20:07:31.710696+0000","last_active":"2024-04-26T20:09:15.787059+0000","last_peered":"2024-04-26T20:09:15.787059+0000","last_clean":"2024-04-26T20:09:15.787059+0000","last_became_active":"2024-04-26T20:07:31.710039+0000","last_became_peered":"2024-04-26T20:07:31.710039+0000","last_unstale":"2024-04-26T20:09:15.787059+0000","last_undegraded":"2024-04-26T20:09:15.787059+0000","last_fullsized":"2024-04-26T20:09:15.787059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:50.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.798031+0000","last_change":"2024-04-26T20:09:15.798031+0000","last_active":"2024-04-26T20:09:15.798031+0000","last_peered":"2024-04-26T20:09:15.798031+0000","last_clean":"2024-04-26T20:09:15.798031+0000","last_became_active":"2024-04-26T20:09:15.797535+0000","last_became_peered":"2024-04-26T20:09:15.797535+0000","last_unstale":"2024-04-26T20:09:15.798031+0000","last_undegraded":"2024-04-26T20:09:15.798031+0000","last_fullsized":"2024-04-26T20:09:15.798031+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:03.059618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.087243+0000","last_change":"2024-04-26T20:07:39.312800+0000","last_active":"2024-04-26T20:09:16.087243+0000","last_peered":"2024-04-26T20:09:16.087243+0000","last_clean":"2024-04-26T20:09:16.087243+0000","last_became_active":"2024-04-26T20:07:39.312309+0000","last_became_peered":"2024-04-26T20:07:39.312309+0000","last_unstale":"2024-04-26T20:09:16.087243+0000","last_undegraded":"2024-04-26T20:09:16.087243+0000","last_fullsized":"2024-04-26T20:09:16.087243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:02.013468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.364648+0000","last_change":"2024-04-26T20:07:31.715509+0000","last_active":"2024-04-26T20:09:16.364648+0000","last_peered":"2024-04-26T20:09:16.364648+0000","last_clean":"2024-04-26T20:09:16.364648+0000","last_became_active":"2024-04-26T20:07:31.714606+0000","last_became_peered":"2024-04-26T20:07:31.714606+0000","last_unstale":"2024-04-26T20:09:16.364648+0000","last_undegraded":"2024-04-26T20:09:16.364648+0000","last_fullsized":"2024-04-26T20:09:16.364648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:03.979681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.863942+0000","last_change":"2024-04-26T20:07:34.763771+0000","last_active":"2024-04-26T20:09:15.863942+0000","last_peered":"2024-04-26T20:09:15.863942+0000","last_clean":"2024-04-26T20:09:15.863942+0000","last_became_active":"2024-04-26T20:07:34.763422+0000","last_became_peered":"2024-04-26T20:07:34.763422+0000","last_unstale":"2024-04-26T20:09:15.863942+0000","last_undegraded":"2024-04-26T20:09:15.863942+0000","last_fullsized":"2024-04-26T20:09:15.863942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:27.224024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266640+0000","last_change":"2024-04-26T20:07:38.830156+0000","last_active":"2024-04-26T20:09:16.266640+0000","last_peered":"2024-04-26T20:09:16.266640+0000","last_clean":"2024-04-26T20:09:16.266640+0000","last_became_active":"2024-04-26T20:07:38.829530+0000","last_became_peered":"2024-04-26T20:07:38.829530+0000","last_unstale":"2024-04-26T20:09:16.266640+0000","last_undegraded":"2024-04-26T20:09:16.266640+0000","last_fullsized":"2024-04-26T20:09:16.266640+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:21.720974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787310+0000","last_change":"2024-04-26T20:07:31.915638+0000","last_active":"2024-04-26T20:09:15.787310+0000","last_peered":"2024-04-26T20:09:15.787310+0000","last_clean":"2024-04-26T20:09:15.787310+0000","last_became_active":"2024-04-26T20:07:31.915266+0000","last_became_peered":"2024-04-26T20:07:31.915266+0000","last_unstale":"2024-04-26T20:09:15.787310+0000","last_undegraded":"2024-04-26T20:09:15.787310+0000","last_fullsized":"2024-04-26T20:09:15.787310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:51.452828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864392+0000","last_change":"2024-04-26T20:07:34.754476+0000","last_active":"2024-04-26T20:09:15.864392+0000","last_peered":"2024-04-26T20:09:15.864392+0000","last_clean":"2024-04-26T20:09:15.864392+0000","last_became_active":"2024-04-26T20:07:34.754054+0000","last_became_peered":"2024-04-26T20:07:34.754054+0000","last_unstale":"2024-04-26T20:09:15.864392+0000","last_undegraded":"2024-04-26T20:09:15.864392+0000","last_fullsized":"2024-04-26T20:09:15.864392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.497083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.799393+0000","last_change":"2024-04-26T20:07:38.835265+0000","last_active":"2024-04-26T20:09:15.799393+0000","last_peered":"2024-04-26T20:09:15.799393+0000","last_clean":"2024-04-26T20:09:15.799393+0000","last_became_active":"2024-04-26T20:07:38.833873+0000","last_became_peered":"2024-04-26T20:07:38.833873+0000","last_unstale":"2024-04-26T20:09:15.799393+0000","last_undegraded":"2024-04-26T20:09:15.799393+0000","last_fullsized":"2024-04-26T20:09:15.799393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:17.850236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176410+0000","last_change":"2024-04-26T20:07:31.725576+0000","last_active":"2024-04-26T20:09:16.176410+0000","last_peered":"2024-04-26T20:09:16.176410+0000","last_clean":"2024-04-26T20:09:16.176410+0000","last_became_active":"2024-04-26T20:07:31.725166+0000","last_became_peered":"2024-04-26T20:07:31.725166+0000","last_unstale":"2024-04-26T20:09:16.176410+0000","last_undegraded":"2024-04-26T20:09:16.176410+0000","last_fullsized":"2024-04-26T20:09:16.176410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:04.539215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864110+0000","last_change":"2024-04-26T20:07:34.748263+0000","last_active":"2024-04-26T20:09:15.864110+0000","last_peered":"2024-04-26T20:09:15.864110+0000","last_clean":"2024-04-26T20:09:15.864110+0000","last_became_active":"2024-04-26T20:07:34.747889+0000","last_became_peered":"2024-04-26T20:07:34.747889+0000","last_unstale":"2024-04-26T20:09:15.864110+0000","last_undegraded":"2024-04-26T20:09:15.864110+0000","last_fullsized":"2024-04-26T20:09:15.864110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.257405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.175579+0000","last_change":"2024-04-26T20:07:38.803999+0000","last_active":"2024-04-26T20:09:16.175579+0000","last_peered":"2024-04-26T20:09:16.175579+0000","last_clean":"2024-04-26T20:09:16.175579+0000","last_became_active":"2024-04-26T20:07:38.803585+0000","last_became_peered":"2024-04-26T20:07:38.803585+0000","last_unstale":"2024-04-26T20:09:16.175579+0000","last_undegraded":"2024-04-26T20:09:16.175579+0000","last_fullsized":"2024-04-26T20:09:16.175579+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:58.863511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314386+0000","last_change":"2024-04-26T20:07:31.730445+0000","last_active":"2024-04-26T20:09:16.314386+0000","last_peered":"2024-04-26T20:09:16.314386+0000","last_clean":"2024-04-26T20:09:16.314386+0000","last_became_active":"2024-04-26T20:07:31.729944+0000","last_became_peered":"2024-04-26T20:07:31.729944+0000","last_unstale":"2024-04-26T20:09:16.314386+0000","last_undegraded":"2024-04-26T20:09:16.314386+0000","last_fullsized":"2024-04-26T20:09:16.314386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:06.274097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.864441+0000","last_change":"2024-04-26T20:07:34.755449+0000","last_active":"2024-04-26T20:09:15.864441+0000","last_peered":"2024-04-26T20:09:15.864441+0000","last_clean":"2024-04-26T20:09:15.864441+0000","last_became_active":"2024-04-26T20:07:34.755100+0000","last_became_peered":"2024-04-26T20:07:34.755100+0000","last_unstale":"2024-04-26T20:09:15.864441+0000","last_undegraded":"2024-04-26T20:09:15.864441+0000","last_fullsized":"2024-04-26T20:09:15.864441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.774751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.178064+0000","last_change":"2024-04-26T20:09:16.177597+0000","last_active":"2024-04-26T20:09:16.178064+0000","last_peered":"2024-04-26T20:09:16.178064+0000","last_clean":"2024-04-26T20:09:16.178064+0000","last_became_active":"2024-04-26T20:09:16.177150+0000","last_became_peered":"2024-04-26T20:09:16.177150+0000","last_unstale":"2024-04-26T20:09:16.178064+0000","last_undegraded":"2024-04-26T20:09:16.178064+0000","last_fullsized":"2024-04-26T20:09:16.178064+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:50.910457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,5],"acting":[11,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787233+0000","last_change":"2024-04-26T20:07:34.747723+0000","last_active":"2024-04-26T20:09:15.787233+0000","last_peered":"2024-04-26T20:09:15.787233+0000","last_clean":"2024-04-26T20:09:15.787233+0000","last_became_active":"2024-04-26T20:07:34.747411+0000","last_became_peered":"2024-04-26T20:07:34.747411+0000","last_unstale":"2024-04-26T20:09:15.787233+0000","last_undegraded":"2024-04-26T20:09:15.787233+0000","last_fullsized":"2024-04-26T20:09:15.787233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.786983+0000","last_change":"2024-04-26T20:07:34.773012+0000","last_active":"2024-04-26T20:09:15.786983+0000","last_peered":"2024-04-26T20:09:15.786983+0000","last_clean":"2024-04-26T20:09:15.786983+0000","last_became_active":"2024-04-26T20:07:34.772735+0000","last_became_peered":"2024-04-26T20:07:34.772735+0000","last_unstale":"2024-04-26T20:09:15.786983+0000","last_undegraded":"2024-04-26T20:09:15.786983+0000","last_fullsized":"2024-04-26T20:09:15.786983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:08.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.799589+0000","last_change":"2024-04-26T20:07:34.762977+0000","last_active":"2024-04-26T20:09:15.799589+0000","last_peered":"2024-04-26T20:09:15.799589+0000","last_clean":"2024-04-26T20:09:15.799589+0000","last_became_active":"2024-04-26T20:07:34.762530+0000","last_became_peered":"2024-04-26T20:07:34.762530+0000","last_unstale":"2024-04-26T20:09:15.799589+0000","last_undegraded":"2024-04-26T20:09:15.799589+0000","last_fullsized":"2024-04-26T20:09:15.799589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:24.811825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.314034+0000","last_change":"2024-04-26T20:07:34.769986+0000","last_active":"2024-04-26T20:09:16.314034+0000","last_peered":"2024-04-26T20:09:16.314034+0000","last_clean":"2024-04-26T20:09:16.314034+0000","last_became_active":"2024-04-26T20:07:34.769591+0000","last_became_peered":"2024-04-26T20:07:34.769591+0000","last_unstale":"2024-04-26T20:09:16.314034+0000","last_undegraded":"2024-04-26T20:09:16.314034+0000","last_fullsized":"2024-04-26T20:09:16.314034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:27.466675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788921+0000","last_change":"2024-04-26T20:07:34.754733+0000","last_active":"2024-04-26T20:09:15.788921+0000","last_peered":"2024-04-26T20:09:15.788921+0000","last_clean":"2024-04-26T20:09:15.788921+0000","last_became_active":"2024-04-26T20:07:34.754374+0000","last_became_peered":"2024-04-26T20:07:34.754374+0000","last_unstale":"2024-04-26T20:09:15.788921+0000","last_undegraded":"2024-04-26T20:09:15.788921+0000","last_fullsized":"2024-04-26T20:09:15.788921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:02.732025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.787681+0000","last_change":"2024-04-26T20:07:34.750329+0000","last_active":"2024-04-26T20:09:15.787681+0000","last_peered":"2024-04-26T20:09:15.787681+0000","last_clean":"2024-04-26T20:09:15.787681+0000","last_became_active":"2024-04-26T20:07:34.749970+0000","last_became_peered":"2024-04-26T20:07:34.749970+0000","last_unstale":"2024-04-26T20:09:15.787681+0000","last_undegraded":"2024-04-26T20:09:15.787681+0000","last_fullsized":"2024-04-26T20:09:15.787681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:50.375128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791609+0000","last_change":"2024-04-26T20:07:34.750623+0000","last_active":"2024-04-26T20:09:15.791609+0000","last_peered":"2024-04-26T20:09:15.791609+0000","last_clean":"2024-04-26T20:09:15.791609+0000","last_became_active":"2024-04-26T20:07:34.750314+0000","last_became_peered":"2024-04-26T20:07:34.750314+0000","last_unstale":"2024-04-26T20:09:15.791609+0000","last_undegraded":"2024-04-26T20:09:15.791609+0000","last_fullsized":"2024-04-26T20:09:15.791609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:16.773986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.176551+0000","last_change":"2024-04-26T20:07:34.755474+0000","last_active":"2024-04-26T20:09:16.176551+0000","last_peered":"2024-04-26T20:09:16.176551+0000","last_clean":"2024-04-26T20:09:16.176551+0000","last_became_active":"2024-04-26T20:07:34.755133+0000","last_became_peered":"2024-04-26T20:07:34.755133+0000","last_unstale":"2024-04-26T20:09:16.176551+0000","last_undegraded":"2024-04-26T20:09:16.176551+0000","last_fullsized":"2024-04-26T20:09:16.176551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:02.861643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.791743+0000","last_change":"2024-04-26T20:07:34.752727+0000","last_active":"2024-04-26T20:09:15.791743+0000","last_peered":"2024-04-26T20:09:15.791743+0000","last_clean":"2024-04-26T20:09:15.791743+0000","last_became_active":"2024-04-26T20:07:34.752420+0000","last_became_peered":"2024-04-26T20:07:34.752420+0000","last_unstale":"2024-04-26T20:09:15.791743+0000","last_undegraded":"2024-04-26T20:09:15.791743+0000","last_fullsized":"2024-04-26T20:09:15.791743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:45.076837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.266861+0000","last_change":"2024-04-26T20:07:34.775116+0000","last_active":"2024-04-26T20:09:16.266861+0000","last_peered":"2024-04-26T20:09:16.266861+0000","last_clean":"2024-04-26T20:09:16.266861+0000","last_became_active":"2024-04-26T20:07:34.774710+0000","last_became_peered":"2024-04-26T20:07:34.774710+0000","last_unstale":"2024-04-26T20:09:16.266861+0000","last_undegraded":"2024-04-26T20:09:16.266861+0000","last_fullsized":"2024-04-26T20:09:16.266861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:53.203918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.313636+0000","last_change":"2024-04-26T20:07:34.773245+0000","last_active":"2024-04-26T20:09:16.313636+0000","last_peered":"2024-04-26T20:09:16.313636+0000","last_clean":"2024-04-26T20:09:16.313636+0000","last_became_active":"2024-04-26T20:07:34.772879+0000","last_became_peered":"2024-04-26T20:07:34.772879+0000","last_unstale":"2024-04-26T20:09:16.313636+0000","last_undegraded":"2024-04-26T20:09:16.313636+0000","last_fullsized":"2024-04-26T20:09:16.313636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:54.450454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.789326+0000","last_change":"2024-04-26T20:07:34.753716+0000","last_active":"2024-04-26T20:09:15.789326+0000","last_peered":"2024-04-26T20:09:15.789326+0000","last_clean":"2024-04-26T20:09:15.789326+0000","last_became_active":"2024-04-26T20:07:34.753378+0000","last_became_peered":"2024-04-26T20:07:34.753378+0000","last_unstale":"2024-04-26T20:09:15.789326+0000","last_undegraded":"2024-04-26T20:09:15.789326+0000","last_fullsized":"2024-04-26T20:09:15.789326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:47.165305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60116,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":687,"num_read_kb":408,"num_write":167,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":180348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383735,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":554234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28772,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716380,"statfs":{"total":95995035648,"available":95965573120,"internally_reserved":0,"allocated":1798144,"data_stored":1197617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":558566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070537,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":554390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332174,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1150299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495700,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":549646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":605671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":551034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411436,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40664,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704488,"statfs":{"total":95995035648,"available":95953395712,"internally_reserved":0,"allocated":1196032,"data_stored":607412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182501,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1122304,"data_stored":549840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920864,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":554296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757338,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1732608,"data_stored":1145887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:10:45.882 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T20:10:45.910 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T20:10:45.930 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T20:10:45.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T20:10:46.405 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:10:46.925 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:46 smithi131 ceph-mon[27919]: pgmap v433: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:46.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:46 smithi157 ceph-mon[32703]: pgmap v433: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:47.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:46 smithi148 ceph-mon[32281]: pgmap v433: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:48.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 -- 172.21.15.131:0/3965724631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5810b640 msgr2=0x7efe5810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:48.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 --2- 172.21.15.131:0/3965724631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5810b640 0x7efe5810ba20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efe48009990 tx=0x7efe48009ca0 comp rx=0 tx=0).stop 2024-04-26T20:10:48.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 -- 172.21.15.131:0/3965724631 shutdown_connections 2024-04-26T20:10:48.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 --2- 172.21.15.131:0/3965724631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe58106000 0x7efe58112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:48.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 --2- 172.21.15.131:0/3965724631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5810b640 0x7efe5810ba20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:48.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 --2- 172.21.15.131:0/3965724631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 0x7efe58105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:48.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 -- 172.21.15.131:0/3965724631 >> 172.21.15.131:0/3965724631 conn(0x7efe58101130 msgr2=0x7efe58103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 -- 172.21.15.131:0/3965724631 shutdown_connections 2024-04-26T20:10:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.754+0000 7efe5f200700 1 -- 172.21.15.131:0/3965724631 wait complete. 2024-04-26T20:10:48.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 Processor -- start 2024-04-26T20:10:48.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 -- start start 2024-04-26T20:10:48.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 0x7efe5819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:48.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe58106000 0x7efe5819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:48.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5810b640 0x7efe581a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:48.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efe58115890 con 0x7efe5810b640 2024-04-26T20:10:48.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe58115710 con 0x7efe58106000 2024-04-26T20:10:49.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.756+0000 7efe5f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efe58115a10 con 0x7efe58105640 2024-04-26T20:10:49.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 0x7efe5819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe5d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5810b640 0x7efe581a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:49.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe58106000 0x7efe5819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:49.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 0x7efe5819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47020/0 (socket says 172.21.15.131:47020) 2024-04-26T20:10:49.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe5d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5810b640 0x7efe581a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44510/0 (socket says 172.21.15.131:44510) 2024-04-26T20:10:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 -- 172.21.15.131:0/3480964100 learned_addr learned my addr 172.21.15.131:0/3480964100 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 -- 172.21.15.131:0/3480964100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe58106000 msgr2=0x7efe5819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:49.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe58106000 0x7efe5819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 -- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5810b640 msgr2=0x7efe581a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:49.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5810b640 0x7efe581a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.757+0000 7efe57e00700 1 -- 172.21.15.131:0/3480964100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe581a0d60 con 0x7efe58105640 2024-04-26T20:10:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.758+0000 7efe57e00700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 0x7efe5819bda0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe48000c00 tx=0x7efe480057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:49.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.758+0000 7efe54c00700 1 -- 172.21.15.131:0/3480964100 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48027070 con 0x7efe58105640 2024-04-26T20:10:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.758+0000 7efe54c00700 1 -- 172.21.15.131:0/3480964100 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe48022470 con 0x7efe58105640 2024-04-26T20:10:49.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.758+0000 7efe54c00700 1 -- 172.21.15.131:0/3480964100 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48017650 con 0x7efe58105640 2024-04-26T20:10:49.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.758+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe581a0ff0 con 0x7efe58105640 2024-04-26T20:10:49.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.758+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a88d0 con 0x7efe58105640 2024-04-26T20:10:49.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.759+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5806a110 con 0x7efe58105640 2024-04-26T20:10:49.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.761+0000 7efe54c00700 1 -- 172.21.15.131:0/3480964100 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efe4802a070 con 0x7efe58105640 2024-04-26T20:10:49.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.762+0000 7efe54c00700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe3c0a3d50 0x7efe3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:49.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.763+0000 7efe54c00700 1 -- 172.21.15.131:0/3480964100 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7efe480d1610 con 0x7efe58105640 2024-04-26T20:10:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.763+0000 7efe57400700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe3c0a3d50 0x7efe3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.763+0000 7efe57400700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe3c0a3d50 0x7efe3c0a6210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe5810acb0 tx=0x7efe44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:48.765+0000 7efe54c00700 1 -- 172.21.15.131:0/3480964100 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efe48013070 con 0x7efe58105640 2024-04-26T20:10:49.608 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:48 smithi148 ceph-mon[32281]: pgmap v434: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:49.622 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:48 smithi157 ceph-mon[32703]: pgmap v434: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:49.628 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:48 smithi131 ceph-mon[27919]: pgmap v434: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:49.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.041+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efe5802e490 con 0x7efe58105640 2024-04-26T20:10:49.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.042+0000 7efe54c00700 1 -- 172.21.15.131:0/3480964100 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efe4809dad0 con 0x7efe58105640 2024-04-26T20:10:49.649 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:10:49.655 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T20:10:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.044+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe3c0a3d50 msgr2=0x7efe3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:49.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.044+0000 7efe5f200700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe3c0a3d50 0x7efe3c0a6210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe5810acb0 tx=0x7efe44009260 comp rx=0 tx=0).stop 2024-04-26T20:10:49.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 msgr2=0x7efe5819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:49.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 0x7efe5819bda0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe48000c00 tx=0x7efe480057c0 comp rx=0 tx=0).stop 2024-04-26T20:10:49.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 shutdown_connections 2024-04-26T20:10:49.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5810b640 0x7efe581a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:49.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe58105640 0x7efe5819bda0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe3c0a3d50 0x7efe3c0a6210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:49.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 --2- 172.21.15.131:0/3480964100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe58106000 0x7efe5819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 >> 172.21.15.131:0/3480964100 conn(0x7efe58101130 msgr2=0x7efe58101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:49.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 shutdown_connections 2024-04-26T20:10:49.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:49.045+0000 7efe5f200700 1 -- 172.21.15.131:0/3480964100 wait complete. 2024-04-26T20:10:50.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:49 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3480964100' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T20:10:50.014 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T20:10:50.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:10:50.030 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:49 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3480964100' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T20:10:50.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:49 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3480964100' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T20:10:50.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.251+0000 7f0bb0000640 1 -- 172.21.15.131:0/711910744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba8103490 msgr2=0x7f0ba8105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:50.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.251+0000 7f0bb0000640 1 --2- 172.21.15.131:0/711910744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba8103490 0x7f0ba8105880 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009900 tx=0x7f0b9c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:10:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.251+0000 7f0bb0000640 1 -- 172.21.15.131:0/711910744 shutdown_connections 2024-04-26T20:10:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.251+0000 7f0bb0000640 1 --2- 172.21.15.131:0/711910744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba81086f0 0x7f0ba810aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.251+0000 7f0bb0000640 1 --2- 172.21.15.131:0/711910744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8105dc0 0x7f0ba81081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:50.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.251+0000 7f0bb0000640 1 --2- 172.21.15.131:0/711910744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba8103490 0x7f0ba8105880 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.251+0000 7f0bb0000640 1 -- 172.21.15.131:0/711910744 >> 172.21.15.131:0/711910744 conn(0x7f0ba80fcdc0 msgr2=0x7f0ba80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:50.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.252+0000 7f0bb0000640 1 -- 172.21.15.131:0/711910744 shutdown_connections 2024-04-26T20:10:50.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.252+0000 7f0bb0000640 1 -- 172.21.15.131:0/711910744 wait complete. 2024-04-26T20:10:50.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 Processor -- start 2024-04-26T20:10:50.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 -- start start 2024-04-26T20:10:50.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8103490 0x7f0ba8102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:50.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba8105dc0 0x7f0ba8101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:50.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba81086f0 0x7f0ba8101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:50.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba810e4d0 con 0x7f0ba81086f0 2024-04-26T20:10:50.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba810e350 con 0x7f0ba8103490 2024-04-26T20:10:50.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bb0000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba810e650 con 0x7f0ba8105dc0 2024-04-26T20:10:50.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bad600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8103490 0x7f0ba8102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:50.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bad600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8103490 0x7f0ba8102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:45912/0 (socket says 172.21.15.131:45912) 2024-04-26T20:10:50.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.253+0000 7f0bad600640 1 -- 172.21.15.131:0/361461454 learned_addr learned my addr 172.21.15.131:0/361461454 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:50.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bacc00640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba8105dc0 0x7f0ba8101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bae000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba81086f0 0x7f0ba8101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bad600640 1 -- 172.21.15.131:0/361461454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba8105dc0 msgr2=0x7f0ba8101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:50.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bad600640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba8105dc0 0x7f0ba8101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:50.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bad600640 1 -- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba81086f0 msgr2=0x7f0ba8101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bad600640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba81086f0 0x7f0ba8101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:50.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bad600640 1 -- 172.21.15.131:0/361461454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba8102210 con 0x7f0ba8103490 2024-04-26T20:10:50.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bacc00640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba8105dc0 0x7f0ba8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0bad600640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8103490 0x7f0ba8102e00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c005ab0 tx=0x7f0b9c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:50.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.254+0000 7f0ba6000640 1 -- 172.21.15.131:0/361461454 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c03f070 con 0x7f0ba8103490 2024-04-26T20:10:50.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.255+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba81a2cc0 con 0x7f0ba8103490 2024-04-26T20:10:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.255+0000 7f0ba6000640 1 -- 172.21.15.131:0/361461454 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b9c031c50 con 0x7f0ba8103490 2024-04-26T20:10:50.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.255+0000 7f0ba6000640 1 -- 172.21.15.131:0/361461454 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c035030 con 0x7f0ba8103490 2024-04-26T20:10:50.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.255+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba81a31b0 con 0x7f0ba8103490 2024-04-26T20:10:50.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.256+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba80fe8b0 con 0x7f0ba8103490 2024-04-26T20:10:51.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.258+0000 7f0ba6000640 1 -- 172.21.15.131:0/361461454 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0b9c042070 con 0x7f0ba8103490 2024-04-26T20:10:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.259+0000 7f0ba6000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0b900a3cc0 0x7f0b900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:51.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.259+0000 7f0ba6000640 1 -- 172.21.15.131:0/361461454 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0b9c0f0fe0 con 0x7f0ba8103490 2024-04-26T20:10:51.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.259+0000 7f0bacc00640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0b900a3cc0 0x7f0b900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.260+0000 7f0bacc00640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0b900a3cc0 0x7f0b900a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b940095f0 tx=0x7f0b94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:51.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.264+0000 7f0ba6000640 1 -- 172.21.15.131:0/361461454 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0b9c0bce20 con 0x7f0ba8103490 2024-04-26T20:10:51.221 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:50 smithi148 ceph-mon[32281]: pgmap v435: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:51.227 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:50 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/361461454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:10:51.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:50 smithi157 ceph-mon[32703]: pgmap v435: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:51.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:50 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/361461454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:10:51.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.471+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ba802dcf0 con 0x7f0ba8103490 2024-04-26T20:10:51.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.474+0000 7f0ba6000640 1 -- 172.21.15.131:0/361461454 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7f0b9c0c1120 con 0x7f0ba8103490 2024-04-26T20:10:51.351 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:10:51.359 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:10:51.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0b900a3cc0 msgr2=0x7f0b900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:51.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0b900a3cc0 0x7f0b900a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b940095f0 tx=0x7f0b94005f90 comp rx=0 tx=0).stop 2024-04-26T20:10:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8103490 msgr2=0x7f0ba8102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:51.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8103490 0x7f0ba8102e00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c005ab0 tx=0x7f0b9c003b90 comp rx=0 tx=0).stop 2024-04-26T20:10:51.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 shutdown_connections 2024-04-26T20:10:51.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba81086f0 0x7f0ba8101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:51.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0b900a3cc0 0x7f0b900a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:51.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba8105dc0 0x7f0ba8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:51.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 --2- 172.21.15.131:0/361461454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ba8103490 0x7f0ba8102e00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:51.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.476+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 >> 172.21.15.131:0/361461454 conn(0x7f0ba80fcdc0 msgr2=0x7f0ba8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.477+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 shutdown_connections 2024-04-26T20:10:51.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:50.477+0000 7f0bb0000640 1 -- 172.21.15.131:0/361461454 wait complete. 2024-04-26T20:10:51.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:10:51.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:10:51.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:50 smithi131 ceph-mon[27919]: pgmap v435: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:10:51.692 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:50 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/361461454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:10:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.918+0000 7f1a5fa00640 1 -- 172.21.15.131:0/3846850061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a58075390 msgr2=0x7f1a58075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:51.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.918+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/3846850061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a58075390 0x7f1a58075810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402a410 comp rx=0 tx=0).stop 2024-04-26T20:10:51.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.919+0000 7f1a5fa00640 1 -- 172.21.15.131:0/3846850061 shutdown_connections 2024-04-26T20:10:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.919+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/3846850061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a5810a830 0x7f1a5810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.919+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/3846850061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a58075390 0x7f1a58075810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.919+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/3846850061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58076970 0x7f1a58074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:52.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.919+0000 7f1a5fa00640 1 -- 172.21.15.131:0/3846850061 >> 172.21.15.131:0/3846850061 conn(0x7f1a580fdee0 msgr2=0x7f1a58100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:52.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.919+0000 7f1a5fa00640 1 -- 172.21.15.131:0/3846850061 shutdown_connections 2024-04-26T20:10:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.919+0000 7f1a5fa00640 1 -- 172.21.15.131:0/3846850061 wait complete. 2024-04-26T20:10:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 Processor -- start 2024-04-26T20:10:52.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 -- start start 2024-04-26T20:10:52.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58075390 0x7f1a5819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:52.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58076970 0x7f1a5819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:52.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a5810a830 0x7f1a581a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:52.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58110680 con 0x7f1a5810a830 2024-04-26T20:10:52.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58110500 con 0x7f1a58075390 2024-04-26T20:10:52.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58110800 con 0x7f1a58076970 2024-04-26T20:10:52.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58075390 0x7f1a5819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58075390 0x7f1a5819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:45932/0 (socket says 172.21.15.131:45932) 2024-04-26T20:10:52.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.920+0000 7f1a5d000640 1 -- 172.21.15.131:0/912906534 learned_addr learned my addr 172.21.15.131:0/912906534 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:52.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a5da00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a5810a830 0x7f1a581a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:52.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a57e00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58076970 0x7f1a5819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:52.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a57e00640 1 -- 172.21.15.131:0/912906534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58075390 msgr2=0x7f1a5819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:52.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a57e00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58075390 0x7f1a5819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:52.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a57e00640 1 -- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a5810a830 msgr2=0x7f1a581a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:52.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a57e00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a5810a830 0x7f1a581a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:52.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a57e00640 1 -- 172.21.15.131:0/912906534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a581a2a30 con 0x7f1a58076970 2024-04-26T20:10:52.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a5d000640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58075390 0x7f1a5819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:52.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.921+0000 7f1a5da00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a5810a830 0x7f1a581a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:10:52.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.922+0000 7f1a57e00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58076970 0x7f1a5819b350 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a440042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.922+0000 7f1a55600640 1 -- 172.21.15.131:0/912906534 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4403f030 con 0x7f1a58076970 2024-04-26T20:10:52.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.922+0000 7f1a55600640 1 -- 172.21.15.131:0/912906534 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a44031910 con 0x7f1a58076970 2024-04-26T20:10:52.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.922+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a581a2cc0 con 0x7f1a58076970 2024-04-26T20:10:52.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.922+0000 7f1a55600640 1 -- 172.21.15.131:0/912906534 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44035070 con 0x7f1a58076970 2024-04-26T20:10:52.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.922+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a581a31b0 con 0x7f1a58076970 2024-04-26T20:10:52.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.923+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a58109560 con 0x7f1a58076970 2024-04-26T20:10:52.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.925+0000 7f1a55600640 1 -- 172.21.15.131:0/912906534 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1a44042030 con 0x7f1a58076970 2024-04-26T20:10:52.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.926+0000 7f1a55600640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a400a3cc0 0x7f1a400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:52.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.926+0000 7f1a55600640 1 -- 172.21.15.131:0/912906534 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f1a440f0ea0 con 0x7f1a58076970 2024-04-26T20:10:52.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.927+0000 7f1a5d000640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a400a3cc0 0x7f1a400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.927+0000 7f1a5d000640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a400a3cc0 0x7f1a400a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c011550 tx=0x7f1a4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:51.930+0000 7f1a55600640 1 -- 172.21.15.131:0/912906534 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a440bcc80 con 0x7f1a58076970 2024-04-26T20:10:53.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.132+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a5802dc80 con 0x7f1a58076970 2024-04-26T20:10:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.133+0000 7f1a55600640 1 -- 172.21.15.131:0/912906534 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f1a440c0f80 con 0x7f1a58076970 2024-04-26T20:10:53.097 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:10:53.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a400a3cc0 msgr2=0x7f1a400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a400a3cc0 0x7f1a400a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c011550 tx=0x7f1a4c005f90 comp rx=0 tx=0).stop 2024-04-26T20:10:53.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58076970 msgr2=0x7f1a5819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:53.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58076970 0x7f1a5819b350 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a440042f0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 shutdown_connections 2024-04-26T20:10:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a5810a830 0x7f1a581a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a400a3cc0 0x7f1a400a6180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58076970 0x7f1a5819b350 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 --2- 172.21.15.131:0/912906534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a58075390 0x7f1a5819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 >> 172.21.15.131:0/912906534 conn(0x7f1a580fdee0 msgr2=0x7f1a58106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:53.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 shutdown_connections 2024-04-26T20:10:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:52.136+0000 7f1a5fa00640 1 -- 172.21.15.131:0/912906534 wait complete. 2024-04-26T20:10:53.375 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:10:53.448 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:52 smithi131 ceph-mon[27919]: pgmap v436: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:53.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:52 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/912906534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:10:53.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:52 smithi157 ceph-mon[32703]: pgmap v436: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:53.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:52 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/912906534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:10:53.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:52 smithi148 ceph-mon[32281]: pgmap v436: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:53.573 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:52 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/912906534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:10:53.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 -- 172.21.15.131:0/239329502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 msgr2=0x7fe9ec101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:53.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 --2- 172.21.15.131:0/239329502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 0x7fe9ec101510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009900 tx=0x7fe9dc02a400 comp rx=0 tx=0).stop 2024-04-26T20:10:53.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 -- 172.21.15.131:0/239329502 shutdown_connections 2024-04-26T20:10:53.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 --2- 172.21.15.131:0/239329502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec1023f0 0x7fe9ec102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 --2- 172.21.15.131:0/239329502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9ec101a50 0x7fe9ec101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 --2- 172.21.15.131:0/239329502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 0x7fe9ec101510 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:53.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 -- 172.21.15.131:0/239329502 >> 172.21.15.131:0/239329502 conn(0x7fe9ec0fccd0 msgr2=0x7fe9ec0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:53.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 -- 172.21.15.131:0/239329502 shutdown_connections 2024-04-26T20:10:53.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.771+0000 7fe9eb200640 1 -- 172.21.15.131:0/239329502 wait complete. 2024-04-26T20:10:54.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.772+0000 7fe9eb200640 1 Processor -- start 2024-04-26T20:10:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.772+0000 7fe9eb200640 1 -- start start 2024-04-26T20:10:54.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.772+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 0x7fe9ec19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:54.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.772+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec101a50 0x7fe9ec19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:54.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.772+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9ec1023f0 0x7fe9ec1a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:54.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9eb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec110590 con 0x7fe9ec1010f0 2024-04-26T20:10:54.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9eb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec110410 con 0x7fe9ec1023f0 2024-04-26T20:10:54.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9eb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec110710 con 0x7fe9ec101a50 2024-04-26T20:10:54.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 0x7fe9ec19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:54.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec101a50 0x7fe9ec19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec101a50 0x7fe9ec19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47066/0 (socket says 172.21.15.131:47066) 2024-04-26T20:10:54.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 -- 172.21.15.131:0/4060918088 learned_addr learned my addr 172.21.15.131:0/4060918088 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:54.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9ea800640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9ec1023f0 0x7fe9ec1a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 -- 172.21.15.131:0/4060918088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9ec1023f0 msgr2=0x7fe9ec1a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9ec1023f0 0x7fe9ec1a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:54.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 -- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 msgr2=0x7fe9ec19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:54.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 0x7fe9ec19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:54.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.773+0000 7fe9e3e00640 1 -- 172.21.15.131:0/4060918088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec1a2890 con 0x7fe9ec101a50 2024-04-26T20:10:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.774+0000 7fe9e3e00640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec101a50 0x7fe9ec19b1c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9f0 tx=0x7fe9d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:54.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.774+0000 7fe9e2a00640 1 -- 172.21.15.131:0/4060918088 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400cd80 con 0x7fe9ec101a50 2024-04-26T20:10:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.774+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec1a2b80 con 0x7fe9ec101a50 2024-04-26T20:10:54.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.774+0000 7fe9e2a00640 1 -- 172.21.15.131:0/4060918088 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d40108e0 con 0x7fe9ec101a50 2024-04-26T20:10:54.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.774+0000 7fe9e2a00640 1 -- 172.21.15.131:0/4060918088 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4018980 con 0x7fe9ec101a50 2024-04-26T20:10:54.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.774+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec1a30f0 con 0x7fe9ec101a50 2024-04-26T20:10:54.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.775+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec10be40 con 0x7fe9ec101a50 2024-04-26T20:10:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.777+0000 7fe9e2a00640 1 -- 172.21.15.131:0/4060918088 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe9d4010430 con 0x7fe9ec101a50 2024-04-26T20:10:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.780+0000 7fe9e2a00640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9d00a3c40 0x7fe9d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:54.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.780+0000 7fe9e2a00640 1 -- 172.21.15.131:0/4060918088 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe9d40d1000 con 0x7fe9ec101a50 2024-04-26T20:10:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.780+0000 7fe9e9e00640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9d00a3c40 0x7fe9d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.780+0000 7fe9e9e00640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9d00a3c40 0x7fe9d00a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc031500 tx=0x7fe9dc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:54.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.781+0000 7fe9e2a00640 1 -- 172.21.15.131:0/4060918088 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe9d409cde0 con 0x7fe9ec101a50 2024-04-26T20:10:54.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.983+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe9ec02dad0 con 0x7fe9ec101a50 2024-04-26T20:10:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.985+0000 7fe9e2a00640 1 -- 172.21.15.131:0/4060918088 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fe9d40a10e0 con 0x7fe9ec101a50 2024-04-26T20:10:54.703 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:10:54.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9d00a3c40 msgr2=0x7fe9d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:54.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9d00a3c40 0x7fe9d00a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc031500 tx=0x7fe9dc005a20 comp rx=0 tx=0).stop 2024-04-26T20:10:54.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec101a50 msgr2=0x7fe9ec19b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:54.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec101a50 0x7fe9ec19b1c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9f0 tx=0x7fe9d400eeb0 comp rx=0 tx=0).stop 2024-04-26T20:10:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 shutdown_connections 2024-04-26T20:10:54.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9ec1023f0 0x7fe9ec1a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:54.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9d00a3c40 0x7fe9d00a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:54.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec101a50 0x7fe9ec19b1c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 --2- 172.21.15.131:0/4060918088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ec1010f0 0x7fe9ec19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:54.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.987+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 >> 172.21.15.131:0/4060918088 conn(0x7fe9ec0fccd0 msgr2=0x7fe9ec100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:54.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.988+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 shutdown_connections 2024-04-26T20:10:54.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:53.988+0000 7fe9eb200640 1 -- 172.21.15.131:0/4060918088 wait complete. 2024-04-26T20:10:54.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:10:55.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:54 smithi131 ceph-mon[27919]: pgmap v437: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:55.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:54 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4060918088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:10:55.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:54 smithi148 ceph-mon[32281]: pgmap v437: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:55.147 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:54 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4060918088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:10:55.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.214+0000 7f45d5400640 1 -- 172.21.15.131:0/2350511029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d010a180 msgr2=0x7f45d010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 --2- 172.21.15.131:0/2350511029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d010a180 0x7f45d010a600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009900 tx=0x7f45b802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:10:55.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 -- 172.21.15.131:0/2350511029 shutdown_connections 2024-04-26T20:10:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 --2- 172.21.15.131:0/2350511029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010ab40 0x7f45d01113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:55.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 --2- 172.21.15.131:0/2350511029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d010a180 0x7f45d010a600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 --2- 172.21.15.131:0/2350511029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d0108f90 0x7f45d01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:55.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 -- 172.21.15.131:0/2350511029 >> 172.21.15.131:0/2350511029 conn(0x7f45d0077f30 msgr2=0x7f45d007a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:55.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 -- 172.21.15.131:0/2350511029 shutdown_connections 2024-04-26T20:10:55.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.215+0000 7f45d5400640 1 -- 172.21.15.131:0/2350511029 wait complete. 2024-04-26T20:10:55.370 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:54 smithi157 ceph-mon[32703]: pgmap v437: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:10:55.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:54 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4060918088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:10:55.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.216+0000 7f45d5400640 1 Processor -- start 2024-04-26T20:10:55.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.216+0000 7f45d5400640 1 -- start start 2024-04-26T20:10:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45d5400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0108f90 0x7f45d019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:55.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45d5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010a180 0x7f45d019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45d5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d010ab40 0x7f45d01a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:55.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45d5400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0114ad0 con 0x7f45d010ab40 2024-04-26T20:10:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45d5400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0114950 con 0x7f45d0108f90 2024-04-26T20:10:55.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45d5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0114c50 con 0x7f45d010a180 2024-04-26T20:10:55.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45cf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d010ab40 0x7f45d01a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45c5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010a180 0x7f45d019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:55.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45ce800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0108f90 0x7f45d019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:55.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45c5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010a180 0x7f45d019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57482/0 (socket says 172.21.15.131:57482) 2024-04-26T20:10:55.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.217+0000 7f45c5e00640 1 -- 172.21.15.131:0/1743863192 learned_addr learned my addr 172.21.15.131:0/1743863192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:55.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c5e00640 1 -- 172.21.15.131:0/1743863192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0108f90 msgr2=0x7f45d019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:55.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c5e00640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0108f90 0x7f45d019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c5e00640 1 -- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d010ab40 msgr2=0x7f45d01a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:55.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c5e00640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d010ab40 0x7f45d01a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:55.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c5e00640 1 -- 172.21.15.131:0/1743863192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d01a6da0 con 0x7f45d010a180 2024-04-26T20:10:55.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c5e00640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010a180 0x7f45d019f6c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005940 tx=0x7f45b80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:55.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c7e00640 1 -- 172.21.15.131:0/1743863192 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b803f030 con 0x7f45d010a180 2024-04-26T20:10:55.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d01a7030 con 0x7f45d010a180 2024-04-26T20:10:55.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c7e00640 1 -- 172.21.15.131:0/1743863192 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b8003cb0 con 0x7f45d010a180 2024-04-26T20:10:55.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.218+0000 7f45c7e00640 1 -- 172.21.15.131:0/1743863192 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8035070 con 0x7f45d010a180 2024-04-26T20:10:55.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.219+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01a7520 con 0x7f45d010a180 2024-04-26T20:10:55.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.220+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d0110330 con 0x7f45d010a180 2024-04-26T20:10:55.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.222+0000 7f45c7e00640 1 -- 172.21.15.131:0/1743863192 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f45b8042030 con 0x7f45d010a180 2024-04-26T20:10:55.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.223+0000 7f45c7e00640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f45b40a3c40 0x7f45b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.224+0000 7f45c7e00640 1 -- 172.21.15.131:0/1743863192 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f45b80f13e0 con 0x7f45d010a180 2024-04-26T20:10:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.224+0000 7f45ce800640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f45b40a3c40 0x7f45b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:56.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.224+0000 7f45ce800640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f45b40a3c40 0x7f45b40a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45d0109fe0 tx=0x7f45c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.226+0000 7f45c7e00640 1 -- 172.21.15.131:0/1743863192 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f45b8003e60 con 0x7f45d010a180 2024-04-26T20:10:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.433+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45d002dd20 con 0x7f45d010a180 2024-04-26T20:10:56.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.434+0000 7f45c7e00640 1 -- 172.21.15.131:0/1743863192 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f45b80bd1c0 con 0x7f45d010a180 2024-04-26T20:10:56.052 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:10:56.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.437+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f45b40a3c40 msgr2=0x7f45b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:56.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.437+0000 7f45d5400640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f45b40a3c40 0x7f45b40a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45d0109fe0 tx=0x7f45c0005f90 comp rx=0 tx=0).stop 2024-04-26T20:10:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.437+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010a180 msgr2=0x7f45d019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:56.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.437+0000 7f45d5400640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010a180 0x7f45d019f6c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005940 tx=0x7f45b80042f0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 shutdown_connections 2024-04-26T20:10:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d010ab40 0x7f45d01a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f45b40a3c40 0x7f45b40a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d010a180 0x7f45d019f6c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 --2- 172.21.15.131:0/1743863192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0108f90 0x7f45d019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 >> 172.21.15.131:0/1743863192 conn(0x7f45d0077f30 msgr2=0x7f45d010d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:56.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 shutdown_connections 2024-04-26T20:10:56.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:55.438+0000 7f45d5400640 1 -- 172.21.15.131:0/1743863192 wait complete. 2024-04-26T20:10:56.329 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:10:56.337 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:55 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1743863192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:10:56.345 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:55 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1743863192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:10:56.351 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:55 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1743863192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:10:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 -- 172.21.15.131:0/4090586427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 msgr2=0x7f8d6c074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 --2- 172.21.15.131:0/4090586427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 0x7f8d6c074dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009be0 tx=0x7f8d5c02a6c0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 -- 172.21.15.131:0/4090586427 shutdown_connections 2024-04-26T20:10:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 --2- 172.21.15.131:0/4090586427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 0x7f8d6c111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 --2- 172.21.15.131:0/4090586427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6c075390 0x7f8d6c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 --2- 172.21.15.131:0/4090586427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 0x7f8d6c074dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:56.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 -- 172.21.15.131:0/4090586427 >> 172.21.15.131:0/4090586427 conn(0x7f8d6c0fdee0 msgr2=0x7f8d6c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 -- 172.21.15.131:0/4090586427 shutdown_connections 2024-04-26T20:10:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.558+0000 7f8d71a00640 1 -- 172.21.15.131:0/4090586427 wait complete. 2024-04-26T20:10:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 Processor -- start 2024-04-26T20:10:56.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 -- start start 2024-04-26T20:10:56.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6c075390 0x7f8d6c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:56.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 0x7f8d6c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:56.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 0x7f8d6c1a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c114a70 con 0x7f8d6c10ef40 2024-04-26T20:10:56.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1148f0 con 0x7f8d6c075390 2024-04-26T20:10:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d71a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c114bf0 con 0x7f8d6c076970 2024-04-26T20:10:56.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.560+0000 7f8d6b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 0x7f8d6c1a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:56.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 0x7f8d6c1a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39200/0 (socket says 172.21.15.131:39200) 2024-04-26T20:10:57.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 -- 172.21.15.131:0/107974656 learned_addr learned my addr 172.21.15.131:0/107974656 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d62000640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 0x7f8d6c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6aa00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6c075390 0x7f8d6c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:57.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 -- 172.21.15.131:0/107974656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 msgr2=0x7f8d6c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:57.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 0x7f8d6c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:57.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 -- 172.21.15.131:0/107974656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6c075390 msgr2=0x7f8d6c19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6c075390 0x7f8d6c19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:57.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 -- 172.21.15.131:0/107974656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c1a6e20 con 0x7f8d6c10ef40 2024-04-26T20:10:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.561+0000 7f8d6b400640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 0x7f8d6c1a67b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e970 tx=0x7f8d6400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:57.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.562+0000 7f8d63e00640 1 -- 172.21.15.131:0/107974656 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400ccc0 con 0x7f8d6c10ef40 2024-04-26T20:10:57.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.562+0000 7f8d63e00640 1 -- 172.21.15.131:0/107974656 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d6400ce60 con 0x7f8d6c10ef40 2024-04-26T20:10:57.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.562+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c1a7110 con 0x7f8d6c10ef40 2024-04-26T20:10:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.562+0000 7f8d63e00640 1 -- 172.21.15.131:0/107974656 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64010430 con 0x7f8d6c10ef40 2024-04-26T20:10:57.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.562+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c1a7610 con 0x7f8d6c10ef40 2024-04-26T20:10:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.563+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6c105110 con 0x7f8d6c10ef40 2024-04-26T20:10:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.565+0000 7f8d63e00640 1 -- 172.21.15.131:0/107974656 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8d64004ff0 con 0x7f8d6c10ef40 2024-04-26T20:10:57.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.566+0000 7f8d63e00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d500a3c40 0x7f8d500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.566+0000 7f8d63e00640 1 -- 172.21.15.131:0/107974656 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8d640d0dd0 con 0x7f8d6c10ef40 2024-04-26T20:10:57.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.567+0000 7f8d6aa00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d500a3c40 0x7f8d500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.567+0000 7f8d6aa00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d500a3c40 0x7f8d500a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006010 tx=0x7f8d5c030960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:57.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.571+0000 7f8d63e00640 1 -- 172.21.15.131:0/107974656 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8d6409d200 con 0x7f8d6c10ef40 2024-04-26T20:10:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.773+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8d6c063530 con 0x7f8d6c10ef40 2024-04-26T20:10:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.774+0000 7f8d63e00640 1 -- 172.21.15.131:0/107974656 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f8d6409cb00 con 0x7f8d6c10ef40 2024-04-26T20:10:57.787 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:10:57.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d500a3c40 msgr2=0x7f8d500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:57.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d500a3c40 0x7f8d500a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006010 tx=0x7f8d5c030960 comp rx=0 tx=0).stop 2024-04-26T20:10:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 msgr2=0x7f8d6c1a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:57.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 0x7f8d6c1a67b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e970 tx=0x7f8d6400ed70 comp rx=0 tx=0).stop 2024-04-26T20:10:57.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 shutdown_connections 2024-04-26T20:10:57.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c10ef40 0x7f8d6c1a67b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:57.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8d500a3c40 0x7f8d500a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:57.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d6c076970 0x7f8d6c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:57.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 --2- 172.21.15.131:0/107974656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6c075390 0x7f8d6c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:57.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 >> 172.21.15.131:0/107974656 conn(0x7f8d6c0fdee0 msgr2=0x7f8d6c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:57.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 shutdown_connections 2024-04-26T20:10:57.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:56.777+0000 7f8d71a00640 1 -- 172.21.15.131:0/107974656 wait complete. 2024-04-26T20:10:57.966 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:10:57.974 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:56 smithi131 ceph-mon[27919]: pgmap v438: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:58.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:56 smithi157 ceph-mon[32703]: pgmap v438: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:58.071 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:56 smithi148 ceph-mon[32281]: pgmap v438: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:10:58.150 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:57 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/107974656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:10:58.166 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:57 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/107974656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:10:58.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:57 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/107974656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:10:58.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 -- 172.21.15.131:0/2454257826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c105b50 msgr2=0x7fa65c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:58.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 --2- 172.21.15.131:0/2454257826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c105b50 0x7fa65c105fd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa648009900 tx=0x7fa64802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:10:58.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 -- 172.21.15.131:0/2454257826 shutdown_connections 2024-04-26T20:10:58.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 --2- 172.21.15.131:0/2454257826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65c106510 0x7fa65c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:58.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 --2- 172.21.15.131:0/2454257826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c105b50 0x7fa65c105fd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:58.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 --2- 172.21.15.131:0/2454257826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c104960 0x7fa65c104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:58.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 -- 172.21.15.131:0/2454257826 >> 172.21.15.131:0/2454257826 conn(0x7fa65c0ffeb0 msgr2=0x7fa65c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:58.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 -- 172.21.15.131:0/2454257826 shutdown_connections 2024-04-26T20:10:58.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.254+0000 7fa660a00640 1 -- 172.21.15.131:0/2454257826 wait complete. 2024-04-26T20:10:58.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 Processor -- start 2024-04-26T20:10:58.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 -- start start 2024-04-26T20:10:58.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65c104960 0x7fa65c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:58.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c105b50 0x7fa65c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:58.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c106510 0x7fa65c1a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:58.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1104e0 con 0x7fa65c105b50 2024-04-26T20:10:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c110360 con 0x7fa65c106510 2024-04-26T20:10:58.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.255+0000 7fa660a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c110660 con 0x7fa65c104960 2024-04-26T20:10:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c105b50 0x7fa65c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c105b50 0x7fa65c19b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39206/0 (socket says 172.21.15.131:39206) 2024-04-26T20:10:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 -- 172.21.15.131:0/3835271126 learned_addr learned my addr 172.21.15.131:0/3835271126 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65be00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c106510 0x7fa65c1a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:58.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65b400640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65c104960 0x7fa65c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:58.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 -- 172.21.15.131:0/3835271126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65c104960 msgr2=0x7fa65c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:58.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65c104960 0x7fa65c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 -- 172.21.15.131:0/3835271126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c106510 msgr2=0x7fa65c1a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c106510 0x7fa65c1a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 -- 172.21.15.131:0/3835271126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c1a2780 con 0x7fa65c105b50 2024-04-26T20:10:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.256+0000 7fa65aa00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c105b50 0x7fa65c19b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa648005ab0 tx=0x7fa648003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:58.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.257+0000 7fa653e00640 1 -- 172.21.15.131:0/3835271126 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64803f070 con 0x7fa65c105b50 2024-04-26T20:10:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.257+0000 7fa653e00640 1 -- 172.21.15.131:0/3835271126 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648031c50 con 0x7fa65c105b50 2024-04-26T20:10:58.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.257+0000 7fa653e00640 1 -- 172.21.15.131:0/3835271126 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648035030 con 0x7fa65c105b50 2024-04-26T20:10:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.257+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c1a2a10 con 0x7fa65c105b50 2024-04-26T20:10:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.257+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c1a2f00 con 0x7fa65c105b50 2024-04-26T20:10:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.258+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65c10bd50 con 0x7fa65c105b50 2024-04-26T20:10:58.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.260+0000 7fa653e00640 1 -- 172.21.15.131:0/3835271126 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa648042070 con 0x7fa65c105b50 2024-04-26T20:10:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.261+0000 7fa653e00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6400a3c40 0x7fa6400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:58.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.261+0000 7fa653e00640 1 -- 172.21.15.131:0/3835271126 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa6480f1120 con 0x7fa65c105b50 2024-04-26T20:10:58.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.262+0000 7fa65b400640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6400a3c40 0x7fa6400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:58.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.262+0000 7fa65b400640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6400a3c40 0x7fa6400a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa65c1059b0 tx=0x7fa64c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:58.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.266+0000 7fa653e00640 1 -- 172.21.15.131:0/3835271126 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa6480bcf00 con 0x7fa65c105b50 2024-04-26T20:10:58.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.510+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa65c02dd20 con 0x7fa65c105b50 2024-04-26T20:10:58.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.511+0000 7fa653e00640 1 -- 172.21.15.131:0/3835271126 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11895 (secure 0 0 0) 0x7fa6480c1200 con 0x7fa65c105b50 2024-04-26T20:10:58.943 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:10:58.951 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:08:57.196498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24557,24571]},"id":1}]} 2024-04-26T20:10:58.959 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-04-26T20:10:58.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6400a3c40 msgr2=0x7fa6400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6400a3c40 0x7fa6400a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa65c1059b0 tx=0x7fa64c005f90 comp rx=0 tx=0).stop 2024-04-26T20:10:59.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c105b50 msgr2=0x7fa65c19b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:59.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c105b50 0x7fa65c19b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa648005ab0 tx=0x7fa648003b90 comp rx=0 tx=0).stop 2024-04-26T20:10:59.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 shutdown_connections 2024-04-26T20:10:59.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65c106510 0x7fa65c1a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65c105b50 0x7fa65c19b0a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa6400a3c40 0x7fa6400a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 --2- 172.21.15.131:0/3835271126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa65c104960 0x7fa65c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 >> 172.21.15.131:0/3835271126 conn(0x7fa65c0ffeb0 msgr2=0x7fa65c108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.514+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 shutdown_connections 2024-04-26T20:10:59.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:58.515+0000 7fa660a00640 1 -- 172.21.15.131:0/3835271126 wait complete. 2024-04-26T20:10:59.260 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:10:59.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:58 smithi131 ceph-mon[27919]: pgmap v439: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:58 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3835271126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:10:59.280 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:58 smithi157 ceph-mon[32703]: pgmap v439: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:59.286 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:58 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3835271126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:10:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:58 smithi148 ceph-mon[32281]: pgmap v439: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:10:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:58 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3835271126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:10:59.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.488+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3267054169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f41022d0 msgr2=0x7fb3f4102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:59.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.488+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3267054169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f41022d0 0x7fb3f4102750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009bd0 tx=0x7fb3ec02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.488+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3267054169 shutdown_connections 2024-04-26T20:10:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.488+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3267054169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f41022d0 0x7fb3f4102750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.488+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3267054169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f4101930 0x7fb3f4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.488+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3267054169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4100fd0 0x7fb3f41013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.488+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3267054169 >> 172.21.15.131:0/3267054169 conn(0x7fb3f40fcbf0 msgr2=0x7fb3f40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:10:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.489+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3267054169 shutdown_connections 2024-04-26T20:10:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.489+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3267054169 wait complete. 2024-04-26T20:10:59.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.489+0000 7fb3f3e00640 1 Processor -- start 2024-04-26T20:10:59.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3e00640 1 -- start start 2024-04-26T20:10:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100fd0 0x7fb3f4106b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f4101930 0x7fb3f4107090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f41022d0 0x7fb3f41036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:10:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4110480 con 0x7fb3f4100fd0 2024-04-26T20:10:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4110300 con 0x7fb3f4101930 2024-04-26T20:10:59.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4110600 con 0x7fb3f41022d0 2024-04-26T20:10:59.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100fd0 0x7fb3f4106b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100fd0 0x7fb3f4106b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39236/0 (socket says 172.21.15.131:39236) 2024-04-26T20:10:59.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f2a00640 1 -- 172.21.15.131:0/3713118567 learned_addr learned my addr 172.21.15.131:0/3713118567 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:10:59.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f3400640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f41022d0 0x7fb3f41036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:59.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.490+0000 7fb3f2000640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f4101930 0x7fb3f4107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:10:59.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3f2a00640 1 -- 172.21.15.131:0/3713118567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f41022d0 msgr2=0x7fb3f41036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3f2a00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f41022d0 0x7fb3f41036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3f2a00640 1 -- 172.21.15.131:0/3713118567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f4101930 msgr2=0x7fb3f4107090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:10:59.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3f2a00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f4101930 0x7fb3f4107090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:10:59.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3f2a00640 1 -- 172.21.15.131:0/3713118567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f4103f80 con 0x7fb3f4100fd0 2024-04-26T20:10:59.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3f2a00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100fd0 0x7fb3f4106b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400e9f0 tx=0x7fb3e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:10:59.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3eb400640 1 -- 172.21.15.131:0/3713118567 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400cd80 con 0x7fb3f4100fd0 2024-04-26T20:10:59.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3eb400640 1 -- 172.21.15.131:0/3713118567 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e40054d0 con 0x7fb3f4100fd0 2024-04-26T20:10:59.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3eb400640 1 -- 172.21.15.131:0/3713118567 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4010690 con 0x7fb3f4100fd0 2024-04-26T20:10:59.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.491+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f41a6fb0 con 0x7fb3f4100fd0 2024-04-26T20:10:59.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.492+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f41a7360 con 0x7fb3f4100fd0 2024-04-26T20:11:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.493+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f4069ac0 con 0x7fb3f4100fd0 2024-04-26T20:11:00.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.495+0000 7fb3eb400640 1 -- 172.21.15.131:0/3713118567 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb3e4004e90 con 0x7fb3f4100fd0 2024-04-26T20:11:00.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.496+0000 7fb3eb400640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb3d80a3cc0 0x7fb3d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.496+0000 7fb3eb400640 1 -- 172.21.15.131:0/3713118567 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb3e40d0d70 con 0x7fb3f4100fd0 2024-04-26T20:11:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.497+0000 7fb3f2000640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb3d80a3cc0 0x7fb3d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.497+0000 7fb3f2000640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb3d80a3cc0 0x7fb3d80a6180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4104ad0 tx=0x7fb3dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:00.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.501+0000 7fb3eb400640 1 -- 172.21.15.131:0/3713118567 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb3e409d1a0 con 0x7fb3f4100fd0 2024-04-26T20:11:00.244 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:10:59 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3713118567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:00.251 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:10:59 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3713118567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:00.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.707+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3f402db50 con 0x7fb3f4100fd0 2024-04-26T20:11:00.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.710+0000 7fb3eb400640 1 -- 172.21.15.131:0/3713118567 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7fb3e409caa0 con 0x7fb3f4100fd0 2024-04-26T20:11:00.275 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:00.286 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:11:00.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.712+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb3d80a3cc0 msgr2=0x7fb3d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:00.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.712+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb3d80a3cc0 0x7fb3d80a6180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4104ad0 tx=0x7fb3dc009260 comp rx=0 tx=0).stop 2024-04-26T20:11:00.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.712+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100fd0 msgr2=0x7fb3f4106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:00.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.712+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100fd0 0x7fb3f4106b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400e9f0 tx=0x7fb3e400edf0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 shutdown_connections 2024-04-26T20:11:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb3d80a3cc0 0x7fb3d80a6180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f41022d0 0x7fb3f41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f4101930 0x7fb3f4107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 --2- 172.21.15.131:0/3713118567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100fd0 0x7fb3f4106b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 >> 172.21.15.131:0/3713118567 conn(0x7fb3f40fcbf0 msgr2=0x7fb3f40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 shutdown_connections 2024-04-26T20:11:00.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:10:59.713+0000 7fb3f3e00640 1 -- 172.21.15.131:0/3713118567 wait complete. 2024-04-26T20:11:00.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:11:00.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:11:00.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:10:59 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3713118567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:00.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 -- 172.21.15.131:0/403435024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c109fc0 msgr2=0x7f703c10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 --2- 172.21.15.131:0/403435024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c109fc0 0x7f703c10a440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7024009900 tx=0x7f702402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 -- 172.21.15.131:0/403435024 shutdown_connections 2024-04-26T20:11:00.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 --2- 172.21.15.131:0/403435024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c10a980 0x7f703c1112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 --2- 172.21.15.131:0/403435024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c109fc0 0x7f703c10a440 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:00.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 --2- 172.21.15.131:0/403435024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f703c108e70 0x7f703c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 -- 172.21.15.131:0/403435024 >> 172.21.15.131:0/403435024 conn(0x7f703c077e00 msgr2=0x7f703c07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:01.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 -- 172.21.15.131:0/403435024 shutdown_connections 2024-04-26T20:11:01.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.881+0000 7f7040a00640 1 -- 172.21.15.131:0/403435024 wait complete. 2024-04-26T20:11:01.207 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:00 smithi148 ceph-mon[32281]: pgmap v440: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 Processor -- start 2024-04-26T20:11:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 -- start start 2024-04-26T20:11:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c108e70 0x7f703c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:01.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f703c109fc0 0x7f703c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:01.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c10a980 0x7f703c1a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:01.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1149d0 con 0x7f703c10a980 2024-04-26T20:11:01.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f703c114850 con 0x7f703c109fc0 2024-04-26T20:11:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f7040a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f703c114b50 con 0x7f703c108e70 2024-04-26T20:11:01.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.882+0000 7f703be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c10a980 0x7f703c1a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c108e70 0x7f703c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c108e70 0x7f703c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57560/0 (socket says 172.21.15.131:57560) 2024-04-26T20:11:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 -- 172.21.15.131:0/4209499883 learned_addr learned my addr 172.21.15.131:0/4209499883 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f7033e00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f703c109fc0 0x7f703c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 -- 172.21.15.131:0/4209499883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f703c109fc0 msgr2=0x7f703c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f703c109fc0 0x7f703c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:01.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 -- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c10a980 msgr2=0x7f703c1a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c10a980 0x7f703c1a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:01.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 -- 172.21.15.131:0/4209499883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c1a6cf0 con 0x7f703c108e70 2024-04-26T20:11:01.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703be00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c10a980 0x7f703c1a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.883+0000 7f703b400640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c108e70 0x7f703c19f0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f702c00e9f0 tx=0x7f702c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.884+0000 7f7038c00640 1 -- 172.21.15.131:0/4209499883 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c00cd80 con 0x7f703c108e70 2024-04-26T20:11:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.884+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c1a6fe0 con 0x7f703c108e70 2024-04-26T20:11:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.884+0000 7f7038c00640 1 -- 172.21.15.131:0/4209499883 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702c0108e0 con 0x7f703c108e70 2024-04-26T20:11:01.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.884+0000 7f7038c00640 1 -- 172.21.15.131:0/4209499883 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c018980 con 0x7f703c108e70 2024-04-26T20:11:01.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.884+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c1a7450 con 0x7f703c108e70 2024-04-26T20:11:01.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.885+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c1101f0 con 0x7f703c108e70 2024-04-26T20:11:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.887+0000 7f7038c00640 1 -- 172.21.15.131:0/4209499883 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f702c010430 con 0x7f703c108e70 2024-04-26T20:11:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.888+0000 7f7038c00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f701c0a3c40 0x7f701c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.888+0000 7f7038c00640 1 -- 172.21.15.131:0/4209499883 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f702c0d0f40 con 0x7f703c108e70 2024-04-26T20:11:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.888+0000 7f7033e00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f701c0a3c40 0x7f701c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:01.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.889+0000 7f7033e00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f701c0a3c40 0x7f701c0a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7024009230 tx=0x7f70240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:01.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:00.891+0000 7f7038c00640 1 -- 172.21.15.131:0/4209499883 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f702c09cd20 con 0x7f703c108e70 2024-04-26T20:11:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:00 smithi131 ceph-mon[27919]: pgmap v440: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:01.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:00 smithi157 ceph-mon[32703]: pgmap v440: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:01.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.095+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f703c02dad0 con 0x7f703c108e70 2024-04-26T20:11:01.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.096+0000 7f7038c00640 1 -- 172.21.15.131:0/4209499883 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f702c0a1020 con 0x7f703c108e70 2024-04-26T20:11:01.944 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:11:01.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f701c0a3c40 msgr2=0x7f701c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:01.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f701c0a3c40 0x7f701c0a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7024009230 tx=0x7f70240314a0 comp rx=0 tx=0).stop 2024-04-26T20:11:01.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c108e70 msgr2=0x7f703c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c108e70 0x7f703c19f0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f702c00e9f0 tx=0x7f702c00eeb0 comp rx=0 tx=0).stop 2024-04-26T20:11:01.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 shutdown_connections 2024-04-26T20:11:01.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f703c10a980 0x7f703c1a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:01.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f703c109fc0 0x7f703c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f701c0a3c40 0x7f701c0a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 --2- 172.21.15.131:0/4209499883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c108e70 0x7f703c19f0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 >> 172.21.15.131:0/4209499883 conn(0x7f703c077e00 msgr2=0x7f703c10d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:02.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.099+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 shutdown_connections 2024-04-26T20:11:02.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:01.100+0000 7f7040a00640 1 -- 172.21.15.131:0/4209499883 wait complete. 2024-04-26T20:11:02.193 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:11:02.213 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:01 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4209499883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:11:02.245 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:01 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4209499883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:11:02.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:01 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4209499883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:11:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 -- 172.21.15.131:0/1634445942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206410a980 msgr2=0x7f20641112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:02.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 --2- 172.21.15.131:0/1634445942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206410a980 0x7f20641112a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f205c009be0 tx=0x7f205c02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 -- 172.21.15.131:0/1634445942 shutdown_connections 2024-04-26T20:11:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 --2- 172.21.15.131:0/1634445942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206410a980 0x7f20641112a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 --2- 172.21.15.131:0/1634445942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2064109fc0 0x7f206410a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 --2- 172.21.15.131:0/1634445942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 0x7f2064109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 -- 172.21.15.131:0/1634445942 >> 172.21.15.131:0/1634445942 conn(0x7f2064077e00 msgr2=0x7f206407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:02.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.427+0000 7f2063200640 1 -- 172.21.15.131:0/1634445942 shutdown_connections 2024-04-26T20:11:02.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.428+0000 7f2063200640 1 -- 172.21.15.131:0/1634445942 wait complete. 2024-04-26T20:11:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.428+0000 7f2063200640 1 Processor -- start 2024-04-26T20:11:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2063200640 1 -- start start 2024-04-26T20:11:02.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2063200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 0x7f206419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:02.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2063200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2064109fc0 0x7f206419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2063200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206410a980 0x7f20641a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2063200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20641149a0 con 0x7f2064108e70 2024-04-26T20:11:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2063200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2064114820 con 0x7f206410a980 2024-04-26T20:11:02.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2063200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2064114b20 con 0x7f2064109fc0 2024-04-26T20:11:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2061e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 0x7f206419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:02.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2061e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 0x7f206419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39260/0 (socket says 172.21.15.131:39260) 2024-04-26T20:11:02.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2061e00640 1 -- 172.21.15.131:0/223475636 learned_addr learned my addr 172.21.15.131:0/223475636 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f205be00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2064109fc0 0x7f206419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:02.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.429+0000 7f2062800640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206410a980 0x7f20641a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:02.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f2061e00640 1 -- 172.21.15.131:0/223475636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2064109fc0 msgr2=0x7f206419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:02.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f2061e00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2064109fc0 0x7f206419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f2061e00640 1 -- 172.21.15.131:0/223475636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206410a980 msgr2=0x7f20641a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:02.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f2061e00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206410a980 0x7f20641a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:02.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f2061e00640 1 -- 172.21.15.131:0/223475636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20641a6c60 con 0x7f2064108e70 2024-04-26T20:11:02.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f205be00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2064109fc0 0x7f206419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:03.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f2061e00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 0x7f206419f040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f205400e9c0 tx=0x7f205400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f205aa00640 1 -- 172.21.15.131:0/223475636 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205400ccc0 con 0x7f2064108e70 2024-04-26T20:11:03.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f205aa00640 1 -- 172.21.15.131:0/223475636 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20540054d0 con 0x7f2064108e70 2024-04-26T20:11:03.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20641a6f50 con 0x7f2064108e70 2024-04-26T20:11:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.430+0000 7f205aa00640 1 -- 172.21.15.131:0/223475636 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054010430 con 0x7f2064108e70 2024-04-26T20:11:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.431+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20641a73b0 con 0x7f2064108e70 2024-04-26T20:11:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.432+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2064069af0 con 0x7f2064108e70 2024-04-26T20:11:03.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.434+0000 7f205aa00640 1 -- 172.21.15.131:0/223475636 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2054004f80 con 0x7f2064108e70 2024-04-26T20:11:03.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.435+0000 7f205aa00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f20440a3c40 0x7f20440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:03.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.435+0000 7f205aa00640 1 -- 172.21.15.131:0/223475636 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f20540d0de0 con 0x7f2064108e70 2024-04-26T20:11:03.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.435+0000 7f205be00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f20440a3c40 0x7f20440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:03.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.436+0000 7f205be00640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f20440a3c40 0x7f20440a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20641a0550 tx=0x7f204c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:03.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.440+0000 7f205aa00640 1 -- 172.21.15.131:0/223475636 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2064069af0 con 0x7f2064108e70 2024-04-26T20:11:03.328 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:02 smithi157 ceph-mon[32703]: pgmap v441: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:03.335 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:02 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/223475636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:11:03.344 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:02 smithi148 ceph-mon[32281]: pgmap v441: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:03.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:02 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/223475636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:11:03.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.643+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f206402dba0 con 0x7f2064108e70 2024-04-26T20:11:03.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.644+0000 7f205aa00640 1 -- 172.21.15.131:0/223475636 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f205409d210 con 0x7f2064108e70 2024-04-26T20:11:03.539 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:11:03.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f20440a3c40 msgr2=0x7f20440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f20440a3c40 0x7f20440a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20641a0550 tx=0x7f204c005f90 comp rx=0 tx=0).stop 2024-04-26T20:11:03.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 msgr2=0x7f206419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 0x7f206419f040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f205400e9c0 tx=0x7f205400ee80 comp rx=0 tx=0).stop 2024-04-26T20:11:03.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 shutdown_connections 2024-04-26T20:11:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206410a980 0x7f20641a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:03.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f20440a3c40 0x7f20440a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2064109fc0 0x7f206419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 --2- 172.21.15.131:0/223475636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2064108e70 0x7f206419f040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:03.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.647+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 >> 172.21.15.131:0/223475636 conn(0x7f2064077e00 msgr2=0x7f206410d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:03.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.648+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 shutdown_connections 2024-04-26T20:11:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:02.648+0000 7f2063200640 1 -- 172.21.15.131:0/223475636 wait complete. 2024-04-26T20:11:03.831 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:11:03.842 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:02 smithi131 ceph-mon[27919]: pgmap v441: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:03.916 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:02 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/223475636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:11:04.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.161+0000 7fd8ef200640 1 -- 172.21.15.131:0/58163683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0104990 msgr2=0x7fd8f0104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.161+0000 7fd8ef200640 1 --2- 172.21.15.131:0/58163683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0104990 0x7fd8f0104db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009860 tx=0x7fd8e802a340 comp rx=0 tx=0).stop 2024-04-26T20:11:04.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.161+0000 7fd8ef200640 1 -- 172.21.15.131:0/58163683 shutdown_connections 2024-04-26T20:11:04.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.161+0000 7fd8ef200640 1 --2- 172.21.15.131:0/58163683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0106540 0x7fd8f010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:04.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.161+0000 7fd8ef200640 1 --2- 172.21.15.131:0/58163683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f0105b80 0x7fd8f0106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.161+0000 7fd8ef200640 1 --2- 172.21.15.131:0/58163683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0104990 0x7fd8f0104db0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:04.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.162+0000 7fd8ef200640 1 -- 172.21.15.131:0/58163683 >> 172.21.15.131:0/58163683 conn(0x7fd8f00fff00 msgr2=0x7fd8f0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:04.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.162+0000 7fd8ef200640 1 -- 172.21.15.131:0/58163683 shutdown_connections 2024-04-26T20:11:04.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.162+0000 7fd8ef200640 1 -- 172.21.15.131:0/58163683 wait complete. 2024-04-26T20:11:04.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 Processor -- start 2024-04-26T20:11:04.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 -- start start 2024-04-26T20:11:04.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0104990 0x7fd8f019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:04.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0105b80 0x7fd8f019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f0106540 0x7fd8f01a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:04.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01104b0 con 0x7fd8f0104990 2024-04-26T20:11:04.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0110330 con 0x7fd8f0106540 2024-04-26T20:11:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ef200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0110630 con 0x7fd8f0105b80 2024-04-26T20:11:04.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0104990 0x7fd8f019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0104990 0x7fd8f019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40074/0 (socket says 172.21.15.131:40074) 2024-04-26T20:11:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ede00640 1 -- 172.21.15.131:0/98335348 learned_addr learned my addr 172.21.15.131:0/98335348 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ee800640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f0106540 0x7fd8f01a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:04.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.163+0000 7fd8ed400640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0105b80 0x7fd8f019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:04.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8ede00640 1 -- 172.21.15.131:0/98335348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0105b80 msgr2=0x7fd8f019b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8ede00640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0105b80 0x7fd8f019b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:04.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8ede00640 1 -- 172.21.15.131:0/98335348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f0106540 msgr2=0x7fd8f01a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:04.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8ede00640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f0106540 0x7fd8f01a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:04.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8ede00640 1 -- 172.21.15.131:0/98335348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f01a2800 con 0x7fd8f0104990 2024-04-26T20:11:04.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8ee800640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f0106540 0x7fd8f01a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:04.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8ede00640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0104990 0x7fd8f019abe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80057c0 tx=0x7fd8e8032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:04.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.164+0000 7fd8e6a00640 1 -- 172.21.15.131:0/98335348 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8037040 con 0x7fd8f0104990 2024-04-26T20:11:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.165+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f01a2a90 con 0x7fd8f0104990 2024-04-26T20:11:04.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.165+0000 7fd8e6a00640 1 -- 172.21.15.131:0/98335348 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e8030bd0 con 0x7fd8f0104990 2024-04-26T20:11:04.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.165+0000 7fd8e6a00640 1 -- 172.21.15.131:0/98335348 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8005280 con 0x7fd8f0104990 2024-04-26T20:11:04.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.165+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f01a2f70 con 0x7fd8f0104990 2024-04-26T20:11:05.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.166+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f010bd80 con 0x7fd8f0104990 2024-04-26T20:11:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.168+0000 7fd8e6a00640 1 -- 172.21.15.131:0/98335348 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd8e8043020 con 0x7fd8f0104990 2024-04-26T20:11:05.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.169+0000 7fd8e6a00640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd8d40a3c40 0x7fd8d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.169+0000 7fd8e6a00640 1 -- 172.21.15.131:0/98335348 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd8e80efe80 con 0x7fd8f0104990 2024-04-26T20:11:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.169+0000 7fd8ed400640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd8d40a3c40 0x7fd8d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:05.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.170+0000 7fd8ed400640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd8d40a3c40 0x7fd8d40a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd8f019c0f0 tx=0x7fd8d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:05.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.173+0000 7fd8e6a00640 1 -- 172.21.15.131:0/98335348 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd8e80bc3c0 con 0x7fd8f0104990 2024-04-26T20:11:05.258 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:04 smithi157 ceph-mon[32703]: pgmap v442: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:05.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:04 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/98335348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:11:05.273 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:04 smithi148 ceph-mon[32281]: pgmap v442: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:05.278 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:04 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/98335348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:11:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.376+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8f0063530 con 0x7fd8f0104990 2024-04-26T20:11:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.377+0000 7fd8e6a00640 1 -- 172.21.15.131:0/98335348 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fd8e80bbcc0 con 0x7fd8f0104990 2024-04-26T20:11:05.298 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:11:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.379+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd8d40a3c40 msgr2=0x7fd8d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:05.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.379+0000 7fd8ef200640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd8d40a3c40 0x7fd8d40a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd8f019c0f0 tx=0x7fd8d8009500 comp rx=0 tx=0).stop 2024-04-26T20:11:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.379+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0104990 msgr2=0x7fd8f019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:05.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.379+0000 7fd8ef200640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0104990 0x7fd8f019abe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80057c0 tx=0x7fd8e8032da0 comp rx=0 tx=0).stop 2024-04-26T20:11:05.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 shutdown_connections 2024-04-26T20:11:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f0106540 0x7fd8f01a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:05.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd8d40a3c40 0x7fd8d40a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f0105b80 0x7fd8f019b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:05.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 --2- 172.21.15.131:0/98335348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f0104990 0x7fd8f019abe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:05.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 >> 172.21.15.131:0/98335348 conn(0x7fd8f00fff00 msgr2=0x7fd8f0108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:05.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 shutdown_connections 2024-04-26T20:11:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:04.380+0000 7fd8ef200640 1 -- 172.21.15.131:0/98335348 wait complete. 2024-04-26T20:11:05.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:11:05.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:04 smithi131 ceph-mon[27919]: pgmap v442: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:05.690 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:04 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/98335348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:11:05.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 -- 172.21.15.131:0/1588778994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f745010a980 msgr2=0x7f74501112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:05.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 --2- 172.21.15.131:0/1588778994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f745010a980 0x7f74501112a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7448009860 tx=0x7f744802a340 comp rx=0 tx=0).stop 2024-04-26T20:11:05.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 -- 172.21.15.131:0/1588778994 shutdown_connections 2024-04-26T20:11:05.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 --2- 172.21.15.131:0/1588778994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f745010a980 0x7f74501112a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:06.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 --2- 172.21.15.131:0/1588778994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7450109fc0 0x7f745010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:06.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 --2- 172.21.15.131:0/1588778994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7450108e70 0x7f7450109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:06.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 -- 172.21.15.131:0/1588778994 >> 172.21.15.131:0/1588778994 conn(0x7f7450077e00 msgr2=0x7f745007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:06.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 -- 172.21.15.131:0/1588778994 shutdown_connections 2024-04-26T20:11:06.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.920+0000 7f7455c00640 1 -- 172.21.15.131:0/1588778994 wait complete. 2024-04-26T20:11:06.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 Processor -- start 2024-04-26T20:11:06.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 -- start start 2024-04-26T20:11:06.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7450108e70 0x7f745019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7450109fc0 0x7f745019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:06.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f745010a980 0x7f74501a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:06.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74501149c0 con 0x7f7450109fc0 2024-04-26T20:11:06.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7450114840 con 0x7f7450108e70 2024-04-26T20:11:06.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.921+0000 7f7455c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7450114b40 con 0x7f745010a980 2024-04-26T20:11:06.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7450108e70 0x7f745019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:06.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f7447400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7450109fc0 0x7f745019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:06.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7450108e70 0x7f745019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:59104/0 (socket says 172.21.15.131:59104) 2024-04-26T20:11:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 -- 172.21.15.131:0/3784928595 learned_addr learned my addr 172.21.15.131:0/3784928595 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f7455200640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f745010a980 0x7f74501a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:06.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 -- 172.21.15.131:0/3784928595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f745010a980 msgr2=0x7f74501a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:06.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f745010a980 0x7f74501a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:06.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 -- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7450109fc0 msgr2=0x7f745019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7450109fc0 0x7f745019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:06.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 -- 172.21.15.131:0/3784928595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74501a6c50 con 0x7f7450108e70 2024-04-26T20:11:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.922+0000 7f744fe00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7450108e70 0x7f745019f030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f744000eb90 tx=0x7f744000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:06.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.923+0000 7f744d600640 1 -- 172.21.15.131:0/3784928595 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440004b20 con 0x7f7450108e70 2024-04-26T20:11:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.923+0000 7f744d600640 1 -- 172.21.15.131:0/3784928595 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7440004cc0 con 0x7f7450108e70 2024-04-26T20:11:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.923+0000 7f744d600640 1 -- 172.21.15.131:0/3784928595 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440010690 con 0x7f7450108e70 2024-04-26T20:11:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.923+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74501a6f40 con 0x7f7450108e70 2024-04-26T20:11:06.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.923+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a7350 con 0x7f7450108e70 2024-04-26T20:11:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.924+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7450069af0 con 0x7f7450108e70 2024-04-26T20:11:06.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.926+0000 7f744d600640 1 -- 172.21.15.131:0/3784928595 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7440005020 con 0x7f7450108e70 2024-04-26T20:11:06.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.927+0000 7f744d600640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74340a3c40 0x7f74340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.928+0000 7f744d600640 1 -- 172.21.15.131:0/3784928595 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f74400d1080 con 0x7f7450108e70 2024-04-26T20:11:06.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.928+0000 7f7447400640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74340a3c40 0x7f74340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:06.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.928+0000 7f7447400640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74340a3c40 0x7f74340a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74501a0540 tx=0x7f7438009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:06.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:05.932+0000 7f744d600640 1 -- 172.21.15.131:0/3784928595 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7450069af0 con 0x7f7450108e70 2024-04-26T20:11:06.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.136+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f745002dba0 con 0x7f7450108e70 2024-04-26T20:11:06.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.137+0000 7f744d600640 1 -- 172.21.15.131:0/3784928595 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f744009d4b0 con 0x7f7450108e70 2024-04-26T20:11:06.938 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:11:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.139+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74340a3c40 msgr2=0x7f74340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.139+0000 7f7455c00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74340a3c40 0x7f74340a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74501a0540 tx=0x7f7438009260 comp rx=0 tx=0).stop 2024-04-26T20:11:07.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7450108e70 msgr2=0x7f745019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:07.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7450108e70 0x7f745019f030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f744000eb90 tx=0x7f744000ef90 comp rx=0 tx=0).stop 2024-04-26T20:11:07.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 shutdown_connections 2024-04-26T20:11:07.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f74340a3c40 0x7f74340a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f745010a980 0x7f74501a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7450109fc0 0x7f745019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 --2- 172.21.15.131:0/3784928595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7450108e70 0x7f745019f030 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 >> 172.21.15.131:0/3784928595 conn(0x7f7450077e00 msgr2=0x7f745010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:07.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 shutdown_connections 2024-04-26T20:11:07.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:06.140+0000 7f7455c00640 1 -- 172.21.15.131:0/3784928595 wait complete. 2024-04-26T20:11:07.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:11:07.153 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:06 smithi148 ceph-mon[32281]: pgmap v443: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:07.161 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:06 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3784928595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:11:07.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:06 smithi157 ceph-mon[32703]: pgmap v443: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:07.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:06 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3784928595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:11:07.245 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:06 smithi131 ceph-mon[27919]: pgmap v443: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:07.304 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:06 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3784928595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:11:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 -- 172.21.15.131:0/3051724023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 msgr2=0x7f222410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:07.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 --2- 172.21.15.131:0/3051724023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222410a600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f220c009900 tx=0x7f220c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 -- 172.21.15.131:0/3051724023 shutdown_connections 2024-04-26T20:11:07.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 --2- 172.21.15.131:0/3051724023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f222410ab40 0x7f22241113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 --2- 172.21.15.131:0/3051724023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222410a600 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 --2- 172.21.15.131:0/3051724023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224108f90 0x7f22241093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 -- 172.21.15.131:0/3051724023 >> 172.21.15.131:0/3051724023 conn(0x7f2224077f40 msgr2=0x7f222407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.385+0000 7f222a400640 1 -- 172.21.15.131:0/3051724023 shutdown_connections 2024-04-26T20:11:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.386+0000 7f222a400640 1 -- 172.21.15.131:0/3051724023 wait complete. 2024-04-26T20:11:07.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.386+0000 7f222a400640 1 Processor -- start 2024-04-26T20:11:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f222a400640 1 -- start start 2024-04-26T20:11:07.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f222a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2224108f90 0x7f222419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f222a400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f222a400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f222410ab40 0x7f22241a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:07.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f222a400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2224114ad0 con 0x7f222410a180 2024-04-26T20:11:07.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f222a400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2224114950 con 0x7f222410ab40 2024-04-26T20:11:07.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f222a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2224114c50 con 0x7f2224108f90 2024-04-26T20:11:07.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f221be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:07.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f221be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40114/0 (socket says 172.21.15.131:40114) 2024-04-26T20:11:07.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f221be00640 1 -- 172.21.15.131:0/2443895120 learned_addr learned my addr 172.21.15.131:0/2443895120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f2223400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2224108f90 0x7f222419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:07.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.387+0000 7f2223e00640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f222410ab40 0x7f22241a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f221be00640 1 -- 172.21.15.131:0/2443895120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2224108f90 msgr2=0x7f222419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:07.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f221be00640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2224108f90 0x7f222419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f221be00640 1 -- 172.21.15.131:0/2443895120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f222410ab40 msgr2=0x7f22241a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f221be00640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f222410ab40 0x7f22241a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f221be00640 1 -- 172.21.15.131:0/2443895120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22241a6dc0 con 0x7f222410a180 2024-04-26T20:11:08.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f221be00640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222419f6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f220c005940 tx=0x7f220c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f2220c00640 1 -- 172.21.15.131:0/2443895120 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c03f030 con 0x7f222410a180 2024-04-26T20:11:08.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22241a7050 con 0x7f222410a180 2024-04-26T20:11:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f2220c00640 1 -- 172.21.15.131:0/2443895120 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220c003cb0 con 0x7f222410a180 2024-04-26T20:11:08.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.388+0000 7f2220c00640 1 -- 172.21.15.131:0/2443895120 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c035070 con 0x7f222410a180 2024-04-26T20:11:08.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.389+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241a7590 con 0x7f222410a180 2024-04-26T20:11:08.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.390+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2224110300 con 0x7f222410a180 2024-04-26T20:11:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.392+0000 7f2220c00640 1 -- 172.21.15.131:0/2443895120 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f220c042030 con 0x7f222410a180 2024-04-26T20:11:08.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.394+0000 7f2220c00640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22080a3cc0 0x7f22080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:08.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.394+0000 7f2220c00640 1 -- 172.21.15.131:0/2443895120 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f220c0f0d30 con 0x7f222410a180 2024-04-26T20:11:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.394+0000 7f2223400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22080a3cc0 0x7f22080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:08.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.396+0000 7f2223400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22080a3cc0 0x7f22080a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2224109fe0 tx=0x7f2214005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:08.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.398+0000 7f2220c00640 1 -- 172.21.15.131:0/2443895120 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f220c003e60 con 0x7f222410a180 2024-04-26T20:11:08.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:07 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2443895120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:11:08.268 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:07 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2443895120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:11:08.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.641+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f222402dd20 con 0x7f222410a180 2024-04-26T20:11:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.643+0000 7f2220c00640 1 -- 172.21.15.131:0/2443895120 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11895 (secure 0 0 0) 0x7f220c0bcb10 con 0x7f222410a180 2024-04-26T20:11:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:08.350 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:08:57.196498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24557,24571]},"id":1}]} 2024-04-26T20:11:08.359 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-04-26T20:11:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22080a3cc0 msgr2=0x7f22080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22080a3cc0 0x7f22080a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2224109fe0 tx=0x7f2214005f90 comp rx=0 tx=0).stop 2024-04-26T20:11:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 msgr2=0x7f222419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:08.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222419f6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f220c005940 tx=0x7f220c0042f0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 shutdown_connections 2024-04-26T20:11:08.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f222410ab40 0x7f22241a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f222410a180 0x7f222419f6e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22080a3cc0 0x7f22080a6180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 --2- 172.21.15.131:0/2443895120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2224108f90 0x7f222419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.646+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 >> 172.21.15.131:0/2443895120 conn(0x7f2224077f40 msgr2=0x7f222410d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.647+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 shutdown_connections 2024-04-26T20:11:08.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:07.647+0000 7f222a400640 1 -- 172.21.15.131:0/2443895120 wait complete. 2024-04-26T20:11:08.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:11:08.526 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:07 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2443895120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:11:08.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 -- 172.21.15.131:0/3326027127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c108d90 msgr2=0x7fec1c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:08.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 --2- 172.21.15.131:0/3326027127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c108d90 0x7fec1c1091b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009900 tx=0x7fec0c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 -- 172.21.15.131:0/3326027127 shutdown_connections 2024-04-26T20:11:08.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 --2- 172.21.15.131:0/3326027127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c10a940 0x7fec1c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 --2- 172.21.15.131:0/3326027127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c109f80 0x7fec1c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 --2- 172.21.15.131:0/3326027127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c108d90 0x7fec1c1091b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 -- 172.21.15.131:0/3326027127 >> 172.21.15.131:0/3326027127 conn(0x7fec1c077d30 msgr2=0x7fec1c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:08.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 -- 172.21.15.131:0/3326027127 shutdown_connections 2024-04-26T20:11:08.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.790+0000 7fec21c00640 1 -- 172.21.15.131:0/3326027127 wait complete. 2024-04-26T20:11:08.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.791+0000 7fec21c00640 1 Processor -- start 2024-04-26T20:11:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec21c00640 1 -- start start 2024-04-26T20:11:08.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec21c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c108d90 0x7fec1c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec21c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c109f80 0x7fec1c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec21c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c10a940 0x7fec1c1a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:08.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec21c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1148c0 con 0x7fec1c109f80 2024-04-26T20:11:09.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec21c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c114740 con 0x7fec1c10a940 2024-04-26T20:11:09.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec21c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c114a40 con 0x7fec1c108d90 2024-04-26T20:11:09.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec13400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c109f80 0x7fec1c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:09.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec1be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c108d90 0x7fec1c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.792+0000 7fec13400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c109f80 0x7fec1c19f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40120/0 (socket says 172.21.15.131:40120) 2024-04-26T20:11:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec21200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c10a940 0x7fec1c1a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec13400640 1 -- 172.21.15.131:0/1416136047 learned_addr learned my addr 172.21.15.131:0/1416136047 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec1be00640 1 -- 172.21.15.131:0/1416136047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c10a940 msgr2=0x7fec1c1a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:09.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec1be00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c10a940 0x7fec1c1a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec1be00640 1 -- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c109f80 msgr2=0x7fec1c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec1be00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c109f80 0x7fec1c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:09.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec1be00640 1 -- 172.21.15.131:0/1416136047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c1a6bb0 con 0x7fec1c108d90 2024-04-26T20:11:09.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec21200640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c10a940 0x7fec1c1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:09.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.793+0000 7fec1be00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c108d90 0x7fec1c19ef90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec0c006010 tx=0x7fec0c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.794+0000 7fec19600640 1 -- 172.21.15.131:0/1416136047 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c03f030 con 0x7fec1c108d90 2024-04-26T20:11:09.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.794+0000 7fec19600640 1 -- 172.21.15.131:0/1416136047 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec0c031910 con 0x7fec1c108d90 2024-04-26T20:11:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.794+0000 7fec19600640 1 -- 172.21.15.131:0/1416136047 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c035070 con 0x7fec1c108d90 2024-04-26T20:11:09.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.794+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c1a6e40 con 0x7fec1c108d90 2024-04-26T20:11:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.794+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c1a7250 con 0x7fec1c108d90 2024-04-26T20:11:09.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.795+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c110180 con 0x7fec1c108d90 2024-04-26T20:11:09.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.797+0000 7fec19600640 1 -- 172.21.15.131:0/1416136047 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fec0c042030 con 0x7fec1c108d90 2024-04-26T20:11:09.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.799+0000 7fec19600640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fec000a3cc0 0x7fec000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:09.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.799+0000 7fec19600640 1 -- 172.21.15.131:0/1416136047 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fec0c0f0ff0 con 0x7fec1c108d90 2024-04-26T20:11:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.799+0000 7fec13400640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fec000a3cc0 0x7fec000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.800+0000 7fec13400640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fec000a3cc0 0x7fec000a6180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1a04a0 tx=0x7fec04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:09.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:08.801+0000 7fec19600640 1 -- 172.21.15.131:0/1416136047 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fec0c0bceb0 con 0x7fec1c108d90 2024-04-26T20:11:09.612 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:08 smithi157 ceph-mon[32703]: pgmap v444: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:09.623 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:08 smithi148 ceph-mon[32281]: pgmap v444: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:09.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.004+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec1c02dca0 con 0x7fec1c108d90 2024-04-26T20:11:09.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:08 smithi131 ceph-mon[27919]: pgmap v444: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.006+0000 7fec19600640 1 -- 172.21.15.131:0/1416136047 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7fec0c0c11b0 con 0x7fec1c108d90 2024-04-26T20:11:09.657 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:09.666 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:11:09.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fec000a3cc0 msgr2=0x7fec000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:09.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fec000a3cc0 0x7fec000a6180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1a04a0 tx=0x7fec04009500 comp rx=0 tx=0).stop 2024-04-26T20:11:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c108d90 msgr2=0x7fec1c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c108d90 0x7fec1c19ef90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec0c006010 tx=0x7fec0c0042f0 comp rx=0 tx=0).stop 2024-04-26T20:11:09.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 shutdown_connections 2024-04-26T20:11:09.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec1c10a940 0x7fec1c1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:09.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c109f80 0x7fec1c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:09.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fec000a3cc0 0x7fec000a6180 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 --2- 172.21.15.131:0/1416136047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c108d90 0x7fec1c19ef90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:09.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 >> 172.21.15.131:0/1416136047 conn(0x7fec1c077d30 msgr2=0x7fec1c10d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:09.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 shutdown_connections 2024-04-26T20:11:09.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:09.009+0000 7fec21c00640 1 -- 172.21.15.131:0/1416136047 wait complete. 2024-04-26T20:11:09.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:11:10.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 -- 172.21.15.131:0/1910935963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38102980 msgr2=0x7f5e38102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:10.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/1910935963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38102980 0x7f5e38102da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009860 tx=0x7f5e2802a340 comp rx=0 tx=0).stop 2024-04-26T20:11:10.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 -- 172.21.15.131:0/1910935963 shutdown_connections 2024-04-26T20:11:10.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/1910935963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38104530 0x7f5e3810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/1910935963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e38103b70 0x7f5e38103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:10.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/1910935963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38102980 0x7f5e38102da0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:10.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 -- 172.21.15.131:0/1910935963 >> 172.21.15.131:0/1910935963 conn(0x7f5e380fdf10 msgr2=0x7f5e38100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:10.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 -- 172.21.15.131:0/1910935963 shutdown_connections 2024-04-26T20:11:10.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.109+0000 7f5e3ea00640 1 -- 172.21.15.131:0/1910935963 wait complete. 2024-04-26T20:11:10.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:09 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1416136047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:10.383 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:09 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1416136047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:10.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 Processor -- start 2024-04-26T20:11:10.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 -- start start 2024-04-26T20:11:10.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38102980 0x7f5e3819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:10.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38103b70 0x7f5e3819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e38104530 0x7f5e381a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:10.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810e4b0 con 0x7f5e38104530 2024-04-26T20:11:10.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810e330 con 0x7f5e38102980 2024-04-26T20:11:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.110+0000 7f5e3ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810e630 con 0x7f5e38103b70 2024-04-26T20:11:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38102980 0x7f5e3819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:10.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38103b70 0x7f5e3819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:10.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38102980 0x7f5e3819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:59142/0 (socket says 172.21.15.131:59142) 2024-04-26T20:11:10.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37e00640 1 -- 172.21.15.131:0/3232994650 learned_addr learned my addr 172.21.15.131:0/3232994650 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:10.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e3ca00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e38104530 0x7f5e381a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:10.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37400640 1 -- 172.21.15.131:0/3232994650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38102980 msgr2=0x7f5e3819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:10.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37400640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38102980 0x7f5e3819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37400640 1 -- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e38104530 msgr2=0x7f5e381a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37400640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e38104530 0x7f5e381a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37400640 1 -- 172.21.15.131:0/3232994650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e381a29e0 con 0x7f5e38103b70 2024-04-26T20:11:10.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37e00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38102980 0x7f5e3819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.111+0000 7f5e37400640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38103b70 0x7f5e3819b300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e790 tx=0x7f5e2400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.112+0000 7f5e34c00640 1 -- 172.21.15.131:0/3232994650 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400eea0 con 0x7f5e38103b70 2024-04-26T20:11:10.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.112+0000 7f5e34c00640 1 -- 172.21.15.131:0/3232994650 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e2400ccc0 con 0x7f5e38103b70 2024-04-26T20:11:10.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.112+0000 7f5e34c00640 1 -- 172.21.15.131:0/3232994650 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24018680 con 0x7f5e38103b70 2024-04-26T20:11:10.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.112+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e381a2cd0 con 0x7f5e38103b70 2024-04-26T20:11:10.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.112+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e381a31d0 con 0x7f5e38103b70 2024-04-26T20:11:10.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.113+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e381a33a0 con 0x7f5e38103b70 2024-04-26T20:11:10.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.116+0000 7f5e34c00640 1 -- 172.21.15.131:0/3232994650 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5e24010430 con 0x7f5e38103b70 2024-04-26T20:11:10.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.117+0000 7f5e34c00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5e1c0a3c40 0x7f5e1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:10.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.117+0000 7f5e34c00640 1 -- 172.21.15.131:0/3232994650 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5e240d0b60 con 0x7f5e38103b70 2024-04-26T20:11:10.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.117+0000 7f5e37e00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5e1c0a3c40 0x7f5e1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:10.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.118+0000 7f5e37e00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5e1c0a3c40 0x7f5e1c0a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e280095f0 tx=0x7f5e28030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:10.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.121+0000 7f5e34c00640 1 -- 172.21.15.131:0/3232994650 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e2409c940 con 0x7f5e38103b70 2024-04-26T20:11:10.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:09 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1416136047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:11.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.373+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e381a3610 con 0x7f5e38103b70 2024-04-26T20:11:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.375+0000 7f5e34c00640 1 -- 172.21.15.131:0/3232994650 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11895 (secure 0 0 0) 0x7f5e240a0c40 con 0x7f5e38103b70 2024-04-26T20:11:11.077 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:11.084 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:08:57.196498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24557,24571]},"id":1}]} 2024-04-26T20:11:11.092 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-04-26T20:11:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5e1c0a3c40 msgr2=0x7f5e1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:11.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5e1c0a3c40 0x7f5e1c0a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e280095f0 tx=0x7f5e28030450 comp rx=0 tx=0).stop 2024-04-26T20:11:11.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38103b70 msgr2=0x7f5e3819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:11.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38103b70 0x7f5e3819b300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e790 tx=0x7f5e2400ec50 comp rx=0 tx=0).stop 2024-04-26T20:11:11.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 shutdown_connections 2024-04-26T20:11:11.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e38104530 0x7f5e381a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5e1c0a3c40 0x7f5e1c0a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e38103b70 0x7f5e3819b300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 --2- 172.21.15.131:0/3232994650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e38102980 0x7f5e3819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.377+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 >> 172.21.15.131:0/3232994650 conn(0x7f5e380fdf10 msgr2=0x7f5e38109310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:11.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.378+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 shutdown_connections 2024-04-26T20:11:11.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:10.378+0000 7f5e3ea00640 1 -- 172.21.15.131:0/3232994650 wait complete. 2024-04-26T20:11:11.238 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T20:07:34.194239+0000', 'modified': '2024-04-26T20:08:57.196498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24575, 'mds_1': 24571, 'mds_2': 24557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/3509306790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 3509306790}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 3509306790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6837/1158905219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1158905219}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1158905219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6839/1960857281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1960857281}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1960857281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'g', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.131:6839/2686418281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 2686418281}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 2686418281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24571': {'gid': 24571, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.131:6841/1288851219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6840', 'nonce': 1288851219}, {'type': 'v1', 'addr': '172.21.15.131:6841', 'nonce': 1288851219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6839/4047119723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 4047119723}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 4047119723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24575, 'qdb_cluster': [24575, 24557, 24571]} 2024-04-26T20:11:11.286 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T20:11:11.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-26T20:11:11.321 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:10 smithi131 ceph-mon[27919]: pgmap v445: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:11.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:10 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3232994650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:11:11.338 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:10 smithi148 ceph-mon[32281]: pgmap v445: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:11.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:10 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3232994650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:11:11.353 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:10 smithi157 ceph-mon[32703]: pgmap v445: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:11.363 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:10 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3232994650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:11:11.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 -- 172.21.15.131:0/3392028105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 msgr2=0x7fd7f410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:11.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/3392028105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f410a400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009860 tx=0x7fd7dc02a340 comp rx=0 tx=0).stop 2024-04-26T20:11:11.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 -- 172.21.15.131:0/3392028105 shutdown_connections 2024-04-26T20:11:11.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/3392028105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f410a940 0x7fd7f41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/3392028105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f410a400 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/3392028105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4108d90 0x7fd7f41091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 -- 172.21.15.131:0/3392028105 >> 172.21.15.131:0/3392028105 conn(0x7fd7f4077d30 msgr2=0x7fd7f407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:11.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 -- 172.21.15.131:0/3392028105 shutdown_connections 2024-04-26T20:11:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.522+0000 7fd7f9a00640 1 -- 172.21.15.131:0/3392028105 wait complete. 2024-04-26T20:11:11.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.523+0000 7fd7f9a00640 1 Processor -- start 2024-04-26T20:11:11.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.523+0000 7fd7f9a00640 1 -- start start 2024-04-26T20:11:11.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.523+0000 7fd7f9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4108d90 0x7fd7f419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:11.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.523+0000 7fd7f9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:11.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.523+0000 7fd7f9a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f410a940 0x7fd7f41a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7f9a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f41148c0 con 0x7fd7f4109f80 2024-04-26T20:11:11.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7f9a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4114740 con 0x7fd7f410a940 2024-04-26T20:11:11.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7f9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4114a40 con 0x7fd7f4108d90 2024-04-26T20:11:11.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:11.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7f9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f410a940 0x7fd7f41a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:11.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f419f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40156/0 (socket says 172.21.15.131:40156) 2024-04-26T20:11:11.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 -- 172.21.15.131:0/1277019387 learned_addr learned my addr 172.21.15.131:0/1277019387 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:11.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4108d90 0x7fd7f419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 -- 172.21.15.131:0/1277019387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4108d90 msgr2=0x7fd7f419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:11.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4108d90 0x7fd7f419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:11.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 -- 172.21.15.131:0/1277019387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f410a940 msgr2=0x7fd7f41a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:11.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f410a940 0x7fd7f41a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7eb400640 1 -- 172.21.15.131:0/1277019387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f41a6be0 con 0x7fd7f4109f80 2024-04-26T20:11:12.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.524+0000 7fd7f9000640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f410a940 0x7fd7f41a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:12.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.525+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f419f500 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc005940 tx=0x7fd7dc02ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.525+0000 7fd7f1600640 1 -- 172.21.15.131:0/1277019387 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc040020 con 0x7fd7f4109f80 2024-04-26T20:11:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.525+0000 7fd7f1600640 1 -- 172.21.15.131:0/1277019387 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc030e50 con 0x7fd7f4109f80 2024-04-26T20:11:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.525+0000 7fd7f1600640 1 -- 172.21.15.131:0/1277019387 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc037040 con 0x7fd7f4109f80 2024-04-26T20:11:12.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.525+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f41a6e70 con 0x7fd7f4109f80 2024-04-26T20:11:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.525+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f41a7280 con 0x7fd7f4109f80 2024-04-26T20:11:12.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 Processor -- start 2024-04-26T20:11:12.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 -- start start 2024-04-26T20:11:12.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7f1600640 1 -- 172.21.15.131:0/1277019387 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd7dc043020 con 0x7fd7f4109f80 2024-04-26T20:11:12.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf50a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:12.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf50f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:12.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf5122b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:12.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf46e600 con 0x55bbcf50acb0 2024-04-26T20:11:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf40b100 con 0x55bbcf50a2c0 2024-04-26T20:11:12.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.528+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf46a5c0 con 0x55bbcf50fe20 2024-04-26T20:11:12.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf50f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:12.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf50f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40170/0 (socket says 172.21.15.131:40170) 2024-04-26T20:11:12.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf50a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:12.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1559105116 learned_addr learned my addr 172.21.15.131:0/1559105116 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:12.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf5122b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:12.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1559105116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 msgr2=0x55bbcf5122b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:12.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf5122b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:12.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1559105116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 msgr2=0x55bbcf50a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:12.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf50a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:12.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1559105116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcf1802d0 con 0x55bbcf50acb0 2024-04-26T20:11:12.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf5122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:11:12.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.529+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf50f7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e550 tx=0x7fd7e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d03ad6d24d144b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:12.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7e9600640 1 -- 172.21.15.131:0/1559105116 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4003970 con 0x55bbcf50acb0 2024-04-26T20:11:12.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7e9600640 1 -- 172.21.15.131:0/1559105116 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e400d7d0 con 0x55bbcf50acb0 2024-04-26T20:11:12.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7e9600640 1 -- 172.21.15.131:0/1559105116 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4004460 con 0x55bbcf50acb0 2024-04-26T20:11:12.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7f1600640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7d80a3cc0 0x7fd7d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:12.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7f1600640 1 -- 172.21.15.131:0/1277019387 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd7dc0f0530 con 0x7fd7f4109f80 2024-04-26T20:11:12.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7f9000640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7d80a3cc0 0x7fd7d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:12.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1559105116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 msgr2=0x55bbcf50f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:12.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.530+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf50f7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e550 tx=0x7fd7e40095c0 comp rx=0 tx=0).stop 2024-04-26T20:11:12.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1559105116 shutdown_connections 2024-04-26T20:11:12.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf5122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:12.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf50f7b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:12.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1559105116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf50a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:12.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1559105116 >> 172.21.15.131:0/1559105116 conn(0x55bbcf49ed10 msgr2=0x55bbcf505d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:12.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1559105116 shutdown_connections 2024-04-26T20:11:12.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7f9000640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7d80a3cc0 0x7fd7d80a6180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55bbcf50b510 tx=0x7fd7ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1559105116 wait complete. 2024-04-26T20:11:12.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 -- start start 2024-04-26T20:11:12.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.531+0000 7fd7fdfe2740 1 Processor -- start 2024-04-26T20:11:12.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:12.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf514c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:12.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf512dd0 con 0x55bbcf50acb0 2024-04-26T20:11:12.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf3f5f70 con 0x55bbcf50a2c0 2024-04-26T20:11:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf512f50 con 0x55bbcf50fe20 2024-04-26T20:11:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7fdfe2740 10 client.0 _ll_register_callbacks cb 0x55bbcf464480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:11:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf514c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:13.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:13.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf514c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40184/0 (socket says 172.21.15.131:40184) 2024-04-26T20:11:13.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 -- 172.21.15.131:0/2207094175 learned_addr learned my addr 172.21.15.131:0/2207094175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:13.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7f9000640 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:13.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 -- 172.21.15.131:0/2207094175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 msgr2=0x55bbcf51bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 -- 172.21.15.131:0/2207094175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 msgr2=0x55bbcf514720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7eb400640 1 -- 172.21.15.131:0/2207094175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcf24a6b0 con 0x55bbcf50acb0 2024-04-26T20:11:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.532+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:13.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7eb400640 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf514c60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0c4ab0 tx=0x7fd7dc0f3eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd2da197a90e6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:13.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7e2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:11:13.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7e2000640 1 -- 172.21.15.131:0/2207094175 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0bbeb0 con 0x55bbcf50acb0 2024-04-26T20:11:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7e2000640 1 -- 172.21.15.131:0/2207094175 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd7dc0040d0 con 0x55bbcf50acb0 2024-04-26T20:11:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7e400d7d0 con 0x55bbcf50acb0 2024-04-26T20:11:13.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7e2000640 1 -- 172.21.15.131:0/2207094175 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7dc0bbcd0 con 0x55bbcf50acb0 2024-04-26T20:11:13.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7fdfe2740 10 client.15024 fetch_fsmap learned FSMap version 26 2024-04-26T20:11:13.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbcf51ec80 con 0x55bbcf50acb0 2024-04-26T20:11:13.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.533+0000 7fd7e2000640 1 -- 172.21.15.131:0/2207094175 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e400d7d0 con 0x55bbcf50acb0 2024-04-26T20:11:13.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7e2000640 1 -- 172.21.15.131:0/2207094175 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd7dc04d060 con 0x55bbcf50acb0 2024-04-26T20:11:13.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7fdfe2740 10 client.15024 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:11:13.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7fdfe2740 10 client.15024 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T20:11:13.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7fdfe2740 20 client.15024 populate_metadata read hostname 'smithi131' 2024-04-26T20:11:13.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf521410 0x55bbcf522880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:13.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7fdfe2740 4 client.15024 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:11:13.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbcf47c210 con 0x55bbcf521410 2024-04-26T20:11:13.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf521410 0x55bbcf522880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:13.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7e2000640 10 client.15024 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:11:13.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.534+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf521410 0x55bbcf522880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a14a802fbf4730c server_cookie=278ac52f7108d2e in_seq=0 out_seq=0 2024-04-26T20:11:13.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.535+0000 7fd7e2000640 1 -- 172.21.15.131:0/2207094175 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbcf47c210 con 0x55bbcf521410 2024-04-26T20:11:13.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.535+0000 7fd7e2000640 10 client.15024 handle_command_reply: tid=0 2024-04-26T20:11:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.535+0000 7fd7fdfe2740 1 client.15024 shutdown 2024-04-26T20:11:13.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 msgr2=0x55bbcf514c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:13.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf514c60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0c4ab0 tx=0x7fd7dc0f3eb0 comp rx=0 tx=0).stop 2024-04-26T20:11:13.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 shutdown_connections 2024-04-26T20:11:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf521410 0x55bbcf522880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:13.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:13.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf514c60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:13.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7e2000640 0 client.15024 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:11:13.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/2207094175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:13.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 >> 172.21.15.131:0/2207094175 conn(0x55bbcf49ed10 msgr2=0x55bbcf504f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:13.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 shutdown_connections 2024-04-26T20:11:13.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/2207094175 wait complete. 2024-04-26T20:11:13.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.536+0000 7fd7fdfe2740 20 client.15024 trim_cache size 0 max 16384 2024-04-26T20:11:13.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.555+0000 7fd7fdfe2740 1 Processor -- start 2024-04-26T20:11:13.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.555+0000 7fd7fdfe2740 1 -- start start 2024-04-26T20:11:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.555+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf51bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:13.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50fe20 0x55bbcf514870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:13.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf520bc0 con 0x55bbcf50acb0 2024-04-26T20:11:13.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf521290 con 0x55bbcf50fe20 2024-04-26T20:11:13.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf521960 con 0x55bbcf50a2c0 2024-04-26T20:11:13.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf51bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:13.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf51bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40200/0 (socket says 172.21.15.131:40200) 2024-04-26T20:11:14.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 -- 172.21.15.131:0/1856498406 learned_addr learned my addr 172.21.15.131:0/1856498406 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:14.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:14.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7f9000640 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50fe20 0x55bbcf514870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:14.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 -- 172.21.15.131:0/1856498406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50a2c0 msgr2=0x55bbcf514330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:14.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:14.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 -- 172.21.15.131:0/1856498406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50fe20 msgr2=0x55bbcf514870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:14.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50fe20 0x55bbcf514870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:14.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7eb400640 1 -- 172.21.15.131:0/1856498406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcf50b540 con 0x55bbcf50acb0 2024-04-26T20:11:14.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.556+0000 7fd7f9000640 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50fe20 0x55bbcf514870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:11:14.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf51bc60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009830 tx=0x7fd7dc0c5140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5280c2603cb0ec9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:14.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7e8c00640 1 -- 172.21.15.131:0/1856498406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0bbeb0 con 0x55bbcf50acb0 2024-04-26T20:11:14.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7e8c00640 1 -- 172.21.15.131:0/1856498406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc0bca70 con 0x55bbcf50acb0 2024-04-26T20:11:14.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7e8c00640 1 -- 172.21.15.131:0/1856498406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0f01a0 con 0x55bbcf50acb0 2024-04-26T20:11:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1856498406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 msgr2=0x55bbcf51bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:14.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf51bc60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009830 tx=0x7fd7dc0c5140 comp rx=0 tx=0).stop 2024-04-26T20:11:14.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1856498406 shutdown_connections 2024-04-26T20:11:14.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50fe20 0x55bbcf514870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:14.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50acb0 0x55bbcf51bc60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.557+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1856498406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50a2c0 0x55bbcf514330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:14.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.558+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1856498406 >> 172.21.15.131:0/1856498406 conn(0x55bbcf49ed10 msgr2=0x55bbcf51ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:14.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.558+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1856498406 shutdown_connections 2024-04-26T20:11:14.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.558+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1856498406 wait complete. 2024-04-26T20:11:14.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.558+0000 7fd7fdfe2740 1 -- start start 2024-04-26T20:11:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.558+0000 7fd7fdfe2740 1 Processor -- start 2024-04-26T20:11:14.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50a2c0 0x55bbcf517830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:14.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50acb0 0x55bbcf517d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:14.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7fdfe2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:14.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf522700 con 0x55bbcf50a2c0 2024-04-26T20:11:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf522030 con 0x55bbcf50acb0 2024-04-26T20:11:14.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7fdfe2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bbcf522dd0 con 0x55bbcf50fe20 2024-04-26T20:11:14.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7fdfe2740 10 client.0 _ll_register_callbacks cb 0x7fd7dc005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:11:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50a2c0 0x55bbcf517830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:14.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50a2c0 0x55bbcf517830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40210/0 (socket says 172.21.15.131:40210) 2024-04-26T20:11:14.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1522533620 learned_addr learned my addr 172.21.15.131:0/1522533620 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:14.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f9000640 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50acb0 0x55bbcf517d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:14.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1522533620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 msgr2=0x55bbcf51ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:14.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1522533620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50acb0 msgr2=0x55bbcf517d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:14.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50acb0 0x55bbcf517d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:14.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 -- 172.21.15.131:0/1522533620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcf51b330 con 0x55bbcf50a2c0 2024-04-26T20:11:14.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7f3e00640 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50a2c0 0x55bbcf517830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e550 tx=0x7fd7e4009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7938e247a63e0567 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:14.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.559+0000 7fd7e2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:11:14.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7e2000640 1 -- 172.21.15.131:0/1522533620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400dc40 con 0x55bbcf50a2c0 2024-04-26T20:11:14.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7e2000640 1 -- 172.21.15.131:0/1522533620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd7e4003970 con 0x55bbcf50a2c0 2024-04-26T20:11:14.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7dc0bca70 con 0x55bbcf50a2c0 2024-04-26T20:11:14.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7e2000640 1 -- 172.21.15.131:0/1522533620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7e4003b70 con 0x55bbcf50a2c0 2024-04-26T20:11:14.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 10 client.15030 fetch_fsmap learned FSMap version 26 2024-04-26T20:11:14.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbcf587800 con 0x55bbcf50a2c0 2024-04-26T20:11:14.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7e2000640 1 -- 172.21.15.131:0/1522533620 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc0bca70 con 0x55bbcf50a2c0 2024-04-26T20:11:14.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7e2000640 1 -- 172.21.15.131:0/1522533620 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd7e400b040 con 0x55bbcf50a2c0 2024-04-26T20:11:14.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 10 client.15030 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:11:14.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 10 client.15030 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T20:11:14.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 20 client.15030 populate_metadata read hostname 'smithi131' 2024-04-26T20:11:14.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf58c060 0x55bbcf58ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:14.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 4 client.15030 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:11:14.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.560+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "status"}) -- 0x55bbcf476080 con 0x55bbcf58c060 2024-04-26T20:11:14.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf58c060 0x55bbcf58ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:15.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7e2000640 10 client.15030 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:11:15.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7eb400640 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf58c060 0x55bbcf58ad70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3b4f79b4d078f19 server_cookie=eda0b9db6f94d1d5 in_seq=0 out_seq=0 2024-04-26T20:11:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7e2000640 1 -- 172.21.15.131:0/1522533620 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bbcf476080 con 0x55bbcf58c060 2024-04-26T20:11:15.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7e2000640 10 client.15030 handle_command_reply: tid=0 2024-04-26T20:11:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7fdfe2740 1 client.15030 shutdown 2024-04-26T20:11:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50a2c0 msgr2=0x55bbcf517830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50a2c0 0x55bbcf517830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e550 tx=0x7fd7e4009ad0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.561+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 shutdown_connections 2024-04-26T20:11:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55bbcf58c060 0x55bbcf58ad70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbcf50fe20 0x55bbcf51ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bbcf50acb0 0x55bbcf517d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 1 --2- 172.21.15.131:0/1522533620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bbcf50a2c0 0x55bbcf517830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 >> 172.21.15.131:0/1522533620 conn(0x55bbcf49ed10 msgr2=0x55bbcf514330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:15.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7e2000640 0 client.15030 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:11:15.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 shutdown_connections 2024-04-26T20:11:15.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 1 -- 172.21.15.131:0/1522533620 wait complete. 2024-04-26T20:11:15.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7fdfe2740 20 client.15030 trim_cache size 0 max 16384 2024-04-26T20:11:15.326 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:11:15.337 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T20:11:15.344 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-04-26T20:11:15.383 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24557, 2024-04-26T20:11:15.392 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-26T20:11:15.402 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-26T20:11:15.409 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-26T20:11:15.416 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 141.61898058099999, 2024-04-26T20:11:15.464 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-04-26T20:11:15.470 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 97, 2024-04-26T20:11:15.479 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T20:11:15.487 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 246.56188120900001 2024-04-26T20:11:15.496 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:11:15.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7d80a3cc0 msgr2=0x7fd7d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:15.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7d80a3cc0 0x7fd7d80a6180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55bbcf50b510 tx=0x7fd7ec005f90 comp rx=0 tx=0).stop 2024-04-26T20:11:15.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 msgr2=0x7fd7f419f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:15.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f419f500 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc005940 tx=0x7fd7dc02ae20 comp rx=0 tx=0).stop 2024-04-26T20:11:15.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 shutdown_connections 2024-04-26T20:11:15.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f410a940 0x7fd7f41a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4109f80 0x7fd7f419f500 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7d80a3cc0 0x7fd7d80a6180 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 --2- 172.21.15.131:0/1277019387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4108d90 0x7fd7f419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:15.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.562+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 >> 172.21.15.131:0/1277019387 conn(0x7fd7f4077d30 msgr2=0x7fd7f410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.563+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 shutdown_connections 2024-04-26T20:11:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:11.563+0000 7fd7f9a00640 1 -- 172.21.15.131:0/1277019387 wait complete. 2024-04-26T20:11:15.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:11:15.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:12 smithi131 ceph-mon[27919]: pgmap v446: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:15.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:14 smithi131 ceph-mon[27919]: pgmap v447: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:15.735 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:12 smithi148 ceph-mon[32281]: pgmap v446: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:15.743 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:14 smithi148 ceph-mon[32281]: pgmap v447: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:15.752 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:12 smithi157 ceph-mon[32703]: pgmap v446: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:15.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:14 smithi157 ceph-mon[32703]: pgmap v447: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:15.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 -- 172.21.15.131:0/739049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4103bd0 msgr2=0x7f97a4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:15.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 --2- 172.21.15.131:0/739049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4103bd0 0x7f97a4104050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f978c009900 tx=0x7f978c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:11:16.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 -- 172.21.15.131:0/739049075 shutdown_connections 2024-04-26T20:11:16.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 --2- 172.21.15.131:0/739049075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4104590 0x7f97a410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 --2- 172.21.15.131:0/739049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4103bd0 0x7f97a4104050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 --2- 172.21.15.131:0/739049075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41029e0 0x7f97a4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 -- 172.21.15.131:0/739049075 >> 172.21.15.131:0/739049075 conn(0x7f97a40fdf70 msgr2=0x7f97a4100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 -- 172.21.15.131:0/739049075 shutdown_connections 2024-04-26T20:11:16.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.944+0000 7f97a9000640 1 -- 172.21.15.131:0/739049075 wait complete. 2024-04-26T20:11:16.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.945+0000 7f97a9000640 1 Processor -- start 2024-04-26T20:11:16.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a9000640 1 -- start start 2024-04-26T20:11:16.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41029e0 0x7f97a419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:16.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4103bd0 0x7f97a419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:16.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4104590 0x7f97a41a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:16.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a9000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f97a410e4f0 con 0x7f97a41029e0 2024-04-26T20:11:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a9000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97a410e370 con 0x7f97a4104590 2024-04-26T20:11:16.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97a410e670 con 0x7f97a4103bd0 2024-04-26T20:11:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41029e0 0x7f97a419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:16.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41029e0 0x7f97a419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45666/0 (socket says 172.21.15.131:45666) 2024-04-26T20:11:16.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a1e00640 1 -- 172.21.15.131:0/1883908065 learned_addr learned my addr 172.21.15.131:0/1883908065 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:16.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.946+0000 7f97a2800640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4104590 0x7f97a41a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:16.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f9799400640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4103bd0 0x7f97a419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f97a2800640 1 -- 172.21.15.131:0/1883908065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4103bd0 msgr2=0x7f97a419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:16.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f97a2800640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4103bd0 0x7f97a419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f97a2800640 1 -- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41029e0 msgr2=0x7f97a419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f97a2800640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41029e0 0x7f97a419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:16.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f97a2800640 1 -- 172.21.15.131:0/1883908065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a41a29f0 con 0x7f97a4104590 2024-04-26T20:11:16.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f97a2800640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4104590 0x7f97a41a2380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f979c00e970 tx=0x7f979c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f979b400640 1 -- 172.21.15.131:0/1883908065 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c00ccc0 con 0x7f97a4104590 2024-04-26T20:11:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f979b400640 1 -- 172.21.15.131:0/1883908065 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f979c005020 con 0x7f97a4104590 2024-04-26T20:11:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a41a2ce0 con 0x7f97a4104590 2024-04-26T20:11:16.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.947+0000 7f979b400640 1 -- 172.21.15.131:0/1883908065 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c004a00 con 0x7f97a4104590 2024-04-26T20:11:16.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.948+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a41a31b0 con 0x7f97a4104590 2024-04-26T20:11:16.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 Processor -- start 2024-04-26T20:11:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f979b400640 1 -- 172.21.15.131:0/1883908065 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f979c004ba0 con 0x7f97a4104590 2024-04-26T20:11:16.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 -- start start 2024-04-26T20:11:16.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e4950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:16.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56015e3f1600 con 0x56015e48d0f0 2024-04-26T20:11:16.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e38e100 con 0x56015e492c50 2024-04-26T20:11:16.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.951+0000 7f97abd8f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56015e3ed5c0 con 0x56015e48dae0 2024-04-26T20:11:16.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f9799400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:16.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f9799400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45672/0 (socket says 172.21.15.131:45672) 2024-04-26T20:11:16.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f97a1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:16.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f9799400640 1 -- 172.21.15.131:0/4084813303 learned_addr learned my addr 172.21.15.131:0/4084813303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:16.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f9799400640 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e4950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f97a1e00640 1 -- 172.21.15.131:0/4084813303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 msgr2=0x56015e4950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:16.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f97a1e00640 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e4950e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f97a1e00640 1 -- 172.21.15.131:0/4084813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 msgr2=0x56015e48d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:17.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f97a1e00640 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:17.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f97a1e00640 1 -- 172.21.15.131:0/4084813303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015e1032d0 con 0x56015e48dae0 2024-04-26T20:11:17.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f9799400640 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:17.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f979b400640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97880a3c40 0x7f97880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:17.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.952+0000 7f979b400640 1 -- 172.21.15.131:0/1883908065 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f979c0d08c0 con 0x7f97a4104590 2024-04-26T20:11:17.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.953+0000 7f97a1e00640 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4925e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9794011440 tx=0x7f9794011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b27a33721faf20d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:17.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.953+0000 7f9799400640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97880a3c40 0x7f97880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:17.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.953+0000 7f9793400640 1 -- 172.21.15.131:0/4084813303 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794003970 con 0x56015e48dae0 2024-04-26T20:11:17.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.953+0000 7f9793400640 1 -- 172.21.15.131:0/4084813303 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9794004020 con 0x56015e48dae0 2024-04-26T20:11:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.953+0000 7f9793400640 1 -- 172.21.15.131:0/4084813303 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979400f840 con 0x56015e48dae0 2024-04-26T20:11:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 -- 172.21.15.131:0/4084813303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 msgr2=0x56015e4925e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:17.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f9799400640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97880a3c40 0x7f97880a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56015e48e340 tx=0x7f978c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4925e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9794011440 tx=0x7f9794011900 comp rx=0 tx=0).stop 2024-04-26T20:11:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 -- 172.21.15.131:0/4084813303 shutdown_connections 2024-04-26T20:11:17.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e4950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:17.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4925e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:17.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 --2- 172.21.15.131:0/4084813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:17.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 -- 172.21.15.131:0/4084813303 >> 172.21.15.131:0/4084813303 conn(0x56015e421d10 msgr2=0x56015e488b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:17.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 -- 172.21.15.131:0/4084813303 shutdown_connections 2024-04-26T20:11:17.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 -- 172.21.15.131:0/4084813303 wait complete. 2024-04-26T20:11:17.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.954+0000 7f97abd8f740 1 -- start start 2024-04-26T20:11:17.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 1 Processor -- start 2024-04-26T20:11:17.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e4974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:17.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e49ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:17.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56015e495c00 con 0x56015e48d0f0 2024-04-26T20:11:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e378f70 con 0x56015e492c50 2024-04-26T20:11:17.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56015e495d80 con 0x56015e48dae0 2024-04-26T20:11:17.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97abd8f740 10 client.0 _ll_register_callbacks cb 0x56015e3e7480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:11:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97a1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e4974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:17.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f9799400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:17.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97a1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e4974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45676/0 (socket says 172.21.15.131:45676) 2024-04-26T20:11:17.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f9799400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37952/0 (socket says 172.21.15.131:37952) 2024-04-26T20:11:17.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97a1e00640 1 -- 172.21.15.131:0/1402429072 learned_addr learned my addr 172.21.15.131:0/1402429072 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:17.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.955+0000 7f97a2800640 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e49ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:17.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f97a1e00640 1 -- 172.21.15.131:0/1402429072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 msgr2=0x56015e4979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:17.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f97a1e00640 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:17.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f97a1e00640 1 -- 172.21.15.131:0/1402429072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 msgr2=0x56015e49ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:17.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f97a1e00640 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e49ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:17.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f97a1e00640 1 -- 172.21.15.131:0/1402429072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015e1cd6b0 con 0x56015e48d0f0 2024-04-26T20:11:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f9799400640 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:18.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f97a1e00640 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e4974b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f979401b040 tx=0x7f9794004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faba9d41786dccc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f9787e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:11:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f9787e00640 1 -- 172.21.15.131:0/1402429072 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794027020 con 0x56015e48d0f0 2024-04-26T20:11:18.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f9787e00640 1 -- 172.21.15.131:0/1402429072 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9794004de0 con 0x56015e48d0f0 2024-04-26T20:11:18.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.956+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9794004020 con 0x56015e48d0f0 2024-04-26T20:11:18.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f9787e00640 1 -- 172.21.15.131:0/1402429072 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9794032050 con 0x56015e48d0f0 2024-04-26T20:11:18.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 10 client.15042 fetch_fsmap learned FSMap version 26 2024-04-26T20:11:18.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56015e4a1a10 con 0x56015e48d0f0 2024-04-26T20:11:18.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f9787e00640 1 -- 172.21.15.131:0/1402429072 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9794004020 con 0x56015e48d0f0 2024-04-26T20:11:18.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f9787e00640 1 -- 172.21.15.131:0/1402429072 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f979400fcc0 con 0x56015e48d0f0 2024-04-26T20:11:18.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 10 client.15042 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:11:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 10 client.15042 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:11:18.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 20 client.15042 populate_metadata read hostname 'smithi131' 2024-04-26T20:11:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e4a3ed0 0x56015e4a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:18.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 4 client.15042 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:11:18.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.957+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56015e3ff210 con 0x56015e4a3ed0 2024-04-26T20:11:18.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.958+0000 7f9799400640 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e4a3ed0 0x56015e4a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.958+0000 7f9799400640 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e4a3ed0 0x56015e4a5340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49a7c05ca1ee4263 server_cookie=64b12246bfbfab68 in_seq=0 out_seq=0 2024-04-26T20:11:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.958+0000 7f9787e00640 10 client.15042 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:11:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f9787e00640 1 -- 172.21.15.131:0/1402429072 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56015e3ff210 con 0x56015e4a3ed0 2024-04-26T20:11:18.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f9787e00640 10 client.15042 handle_command_reply: tid=0 2024-04-26T20:11:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f97abd8f740 1 client.15042 shutdown 2024-04-26T20:11:18.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 msgr2=0x56015e4974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:18.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e4974b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f979401b040 tx=0x7f9794004c70 comp rx=0 tx=0).stop 2024-04-26T20:11:18.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 shutdown_connections 2024-04-26T20:11:18.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e4a3ed0 0x56015e4a5340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.959+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e492c50 0x56015e49ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:18.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.960+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e48dae0 0x56015e4979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:18.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.960+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1402429072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e4974b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.960+0000 7f9787e00640 0 client.15042 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:11:18.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.960+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 >> 172.21.15.131:0/1402429072 conn(0x56015e421d10 msgr2=0x56015e487eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:18.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.960+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 shutdown_connections 2024-04-26T20:11:18.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.960+0000 7f97abd8f740 1 -- 172.21.15.131:0/1402429072 wait complete. 2024-04-26T20:11:18.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.960+0000 7f97abd8f740 20 client.15042 trim_cache size 0 max 16384 2024-04-26T20:11:18.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 Processor -- start 2024-04-26T20:11:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 -- start start 2024-04-26T20:11:18.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:18.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48dae0 0x56015e49e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:18.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e487620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:18.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56015e4a3680 con 0x56015e48d0f0 2024-04-26T20:11:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e4a3d50 con 0x56015e48dae0 2024-04-26T20:11:18.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97abd8f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56015e4a4420 con 0x56015e492c50 2024-04-26T20:11:18.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97a1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97a1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45678/0 (socket says 172.21.15.131:45678) 2024-04-26T20:11:18.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f97a1e00640 1 -- 172.21.15.131:0/1009664402 learned_addr learned my addr 172.21.15.131:0/1009664402 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:18.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.980+0000 7f9799400640 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48dae0 0x56015e49e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f97a2800640 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e487620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f97a1e00640 1 -- 172.21.15.131:0/1009664402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 msgr2=0x56015e487620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f97a1e00640 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e487620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f97a1e00640 1 -- 172.21.15.131:0/1009664402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48dae0 msgr2=0x56015e49e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f97a1e00640 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48dae0 0x56015e49e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f97a1e00640 1 -- 172.21.15.131:0/1009664402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015e48dec0 con 0x56015e48d0f0 2024-04-26T20:11:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f97a1e00640 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48e4e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9794011440 tx=0x7f9794009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4968928cbb399402 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:19.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f9792a00640 1 -- 172.21.15.131:0/1009664402 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794027070 con 0x56015e48d0f0 2024-04-26T20:11:19.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f9792a00640 1 -- 172.21.15.131:0/1009664402 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9794009500 con 0x56015e48d0f0 2024-04-26T20:11:19.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.981+0000 7f9792a00640 1 -- 172.21.15.131:0/1009664402 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979401d040 con 0x56015e48d0f0 2024-04-26T20:11:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 -- 172.21.15.131:0/1009664402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 msgr2=0x56015e48e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:19.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48e4e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9794011440 tx=0x7f9794009ed0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 -- 172.21.15.131:0/1009664402 shutdown_connections 2024-04-26T20:11:19.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e487620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48dae0 0x56015e49e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 --2- 172.21.15.131:0/1009664402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48d0f0 0x56015e48e4e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 -- 172.21.15.131:0/1009664402 >> 172.21.15.131:0/1009664402 conn(0x56015e421d10 msgr2=0x56015e4a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 -- 172.21.15.131:0/1009664402 shutdown_connections 2024-04-26T20:11:19.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 -- 172.21.15.131:0/1009664402 wait complete. 2024-04-26T20:11:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.982+0000 7f97abd8f740 1 -- start start 2024-04-26T20:11:19.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 1 Processor -- start 2024-04-26T20:11:19.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48d0f0 0x56015e49a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:19.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48dae0 0x56015e49a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e49d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56015e4a51c0 con 0x56015e48dae0 2024-04-26T20:11:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e4a4af0 con 0x56015e48d0f0 2024-04-26T20:11:19.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56015e4a5890 con 0x56015e492c50 2024-04-26T20:11:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f97abd8f740 10 client.0 _ll_register_callbacks cb 0x7f9794004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:11:19.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f9799400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48dae0 0x56015e49a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:19.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f9799400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48dae0 0x56015e49a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45690/0 (socket says 172.21.15.131:45690) 2024-04-26T20:11:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.983+0000 7f9799400640 1 -- 172.21.15.131:0/437208431 learned_addr learned my addr 172.21.15.131:0/437208431 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:19.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f97a1e00640 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48d0f0 0x56015e49a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:19.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f97a2800640 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e49d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f9799400640 1 -- 172.21.15.131:0/437208431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 msgr2=0x56015e49d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:19.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f9799400640 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e49d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f9799400640 1 -- 172.21.15.131:0/437208431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48d0f0 msgr2=0x56015e49a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f9799400640 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48d0f0 0x56015e49a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f9799400640 1 -- 172.21.15.131:0/437208431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015e49dec0 con 0x56015e48dae0 2024-04-26T20:11:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f9799400640 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48dae0 0x56015e49a8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f978c005ab0 tx=0x7f978c03d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e7533a7478ccc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.984+0000 7f9787e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:11:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f9787e00640 1 -- 172.21.15.131:0/437208431 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c035050 con 0x56015e48dae0 2024-04-26T20:11:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56015e3f9080 con 0x56015e48dae0 2024-04-26T20:11:20.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f9787e00640 1 -- 172.21.15.131:0/437208431 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f978c0541a0 con 0x56015e48dae0 2024-04-26T20:11:20.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f9787e00640 1 -- 172.21.15.131:0/437208431 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f978c063050 con 0x56015e48dae0 2024-04-26T20:11:20.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 10 client.15048 fetch_fsmap learned FSMap version 26 2024-04-26T20:11:20.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56015e50a0c0 con 0x56015e48dae0 2024-04-26T20:11:20.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f9787e00640 1 -- 172.21.15.131:0/437208431 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56015e3f9080 con 0x56015e48dae0 2024-04-26T20:11:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f9787e00640 1 -- 172.21.15.131:0/437208431 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f978c003b00 con 0x56015e48dae0 2024-04-26T20:11:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 10 client.15048 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:11:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 10 client.15048 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:11:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 20 client.15048 populate_metadata read hostname 'smithi131' 2024-04-26T20:11:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e50ed30 0x56015e50da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:20.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 4 client.15048 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:11:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.985+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "status"}) -- 0x56015e374860 con 0x56015e50ed30 2024-04-26T20:11:20.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.986+0000 7f97a1e00640 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e50ed30 0x56015e50da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:20.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.986+0000 7f97a1e00640 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e50ed30 0x56015e50da40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7be9a6bf61dff934 server_cookie=cb7372c9631fabed in_seq=0 out_seq=0 2024-04-26T20:11:20.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.986+0000 7f9787e00640 10 client.15048 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:11:20.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f9787e00640 1 -- 172.21.15.131:0/437208431 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56015e374860 con 0x56015e50ed30 2024-04-26T20:11:20.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f9787e00640 10 client.15048 handle_command_reply: tid=0 2024-04-26T20:11:20.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 client.15048 shutdown 2024-04-26T20:11:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48dae0 msgr2=0x56015e49a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:20.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48dae0 0x56015e49a8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f978c005ab0 tx=0x7f978c03d070 comp rx=0 tx=0).stop 2024-04-26T20:11:20.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 shutdown_connections 2024-04-26T20:11:20.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x56015e50ed30 0x56015e50da40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56015e492c50 0x56015e49d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:20.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56015e48dae0 0x56015e49a8b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 --2- 172.21.15.131:0/437208431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e48d0f0 0x56015e49a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:20.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f9787e00640 0 client.15048 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:11:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 >> 172.21.15.131:0/437208431 conn(0x56015e421d10 msgr2=0x56015e487620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 shutdown_connections 2024-04-26T20:11:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 1 -- 172.21.15.131:0/437208431 wait complete. 2024-04-26T20:11:20.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.987+0000 7f97abd8f740 20 client.15048 trim_cache size 0 max 16384 2024-04-26T20:11:20.743 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:11:20.813 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T20:11:20.822 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-04-26T20:11:20.832 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24571, 2024-04-26T20:11:20.843 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-26T20:11:20.850 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-26T20:11:20.857 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-26T20:11:20.864 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 147.059046903, 2024-04-26T20:11:20.873 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-04-26T20:11:20.884 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 97, 2024-04-26T20:11:20.892 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T20:11:20.900 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 246.78189727099999 2024-04-26T20:11:20.907 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:11:20.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97880a3c40 msgr2=0x7f97880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:21.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97880a3c40 0x7f97880a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56015e48e340 tx=0x7f978c005d50 comp rx=0 tx=0).stop 2024-04-26T20:11:21.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4104590 msgr2=0x7f97a41a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4104590 0x7f97a41a2380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f979c00e970 tx=0x7f979c00ee30 comp rx=0 tx=0).stop 2024-04-26T20:11:21.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 shutdown_connections 2024-04-26T20:11:21.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a4104590 0x7f97a41a2380 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f97880a3c40 0x7f97880a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4103bd0 0x7f97a419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 --2- 172.21.15.131:0/1883908065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41029e0 0x7f97a419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 >> 172.21.15.131:0/1883908065 conn(0x7f97a40fdf70 msgr2=0x7f97a4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:21.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 shutdown_connections 2024-04-26T20:11:21.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:15.988+0000 7f97a9000640 1 -- 172.21.15.131:0/1883908065 wait complete. 2024-04-26T20:11:21.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T20:11:21.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:16 smithi131 ceph-mon[27919]: pgmap v448: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:18 smithi131 ceph-mon[27919]: pgmap v449: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:21.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:20 smithi131 ceph-mon[27919]: pgmap v450: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:21.226 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:16 smithi148 ceph-mon[32281]: pgmap v448: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:21.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:18 smithi148 ceph-mon[32281]: pgmap v449: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:21.338 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:16 smithi157 ceph-mon[32703]: pgmap v448: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:21.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:18 smithi157 ceph-mon[32703]: pgmap v449: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:21.371 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:20 smithi148 ceph-mon[32281]: pgmap v450: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:21.404 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:20 smithi157 ceph-mon[32703]: pgmap v450: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 -- 172.21.15.131:0/2103719009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1640768f0 msgr2=0x7ff164074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:21.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 --2- 172.21.15.131:0/2103719009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1640768f0 0x7ff164074d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff154009ca0 tx=0x7ff15402a950 comp rx=0 tx=0).stop 2024-04-26T20:11:21.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 -- 172.21.15.131:0/2103719009 shutdown_connections 2024-04-26T20:11:21.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 --2- 172.21.15.131:0/2103719009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff16410a820 0x7ff16410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 --2- 172.21.15.131:0/2103719009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff164075310 0x7ff164075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 --2- 172.21.15.131:0/2103719009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1640768f0 0x7ff164074d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 -- 172.21.15.131:0/2103719009 >> 172.21.15.131:0/2103719009 conn(0x7ff1640fdee0 msgr2=0x7ff164100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:21.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 -- 172.21.15.131:0/2103719009 shutdown_connections 2024-04-26T20:11:21.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.441+0000 7ff16b200640 1 -- 172.21.15.131:0/2103719009 wait complete. 2024-04-26T20:11:21.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.442+0000 7ff16b200640 1 Processor -- start 2024-04-26T20:11:21.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff16b200640 1 -- start start 2024-04-26T20:11:21.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff16b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff164075310 0x7ff16419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:21.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff16b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1640768f0 0x7ff16419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff16b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff16410a820 0x7ff1641a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:21.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff16b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641106f0 con 0x7ff164075310 2024-04-26T20:11:21.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff16b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff164110570 con 0x7ff16410a820 2024-04-26T20:11:21.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff16b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff164110870 con 0x7ff1640768f0 2024-04-26T20:11:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff164075310 0x7ff16419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:21.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff164075310 0x7ff16419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45706/0 (socket says 172.21.15.131:45706) 2024-04-26T20:11:21.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff163e00640 1 -- 172.21.15.131:0/674360873 learned_addr learned my addr 172.21.15.131:0/674360873 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:21.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff15b400640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1640768f0 0x7ff16419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:21.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.443+0000 7ff169200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff16410a820 0x7ff1641a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff163e00640 1 -- 172.21.15.131:0/674360873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1640768f0 msgr2=0x7ff16419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:21.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff163e00640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1640768f0 0x7ff16419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff163e00640 1 -- 172.21.15.131:0/674360873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff16410a820 msgr2=0x7ff1641a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:21.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff163e00640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff16410a820 0x7ff1641a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:21.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff163e00640 1 -- 172.21.15.131:0/674360873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1641a29d0 con 0x7ff164075310 2024-04-26T20:11:21.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff169200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff16410a820 0x7ff1641a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:11:21.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff163e00640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff164075310 0x7ff16419adb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff15402f7a0 tx=0x7ff15402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:21.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff161600640 1 -- 172.21.15.131:0/674360873 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154034040 con 0x7ff164075310 2024-04-26T20:11:21.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1641a2c60 con 0x7ff164075310 2024-04-26T20:11:21.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff161600640 1 -- 172.21.15.131:0/674360873 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff15402fe20 con 0x7ff164075310 2024-04-26T20:11:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff161600640 1 -- 172.21.15.131:0/674360873 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154004700 con 0x7ff164075310 2024-04-26T20:11:21.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.444+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641a31a0 con 0x7ff164075310 2024-04-26T20:11:22.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.448+0000 7ff16dfc7740 1 Processor -- start 2024-04-26T20:11:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.448+0000 7ff161600640 1 -- 172.21.15.131:0/674360873 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff154037020 con 0x7ff164075310 2024-04-26T20:11:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.448+0000 7ff16dfc7740 1 -- start start 2024-04-26T20:11:22.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.448+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254f0f0 0x55c81254f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:22.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.448+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:22.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.448+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c8124b3600 con 0x55c81254fae0 2024-04-26T20:11:22.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c812450100 con 0x55c812554c50 2024-04-26T20:11:22.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c8124af5c0 con 0x55c81254f0f0 2024-04-26T20:11:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:22.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff169200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254f0f0 0x55c81254f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:22.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff15b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:22.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff169200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254f0f0 0x55c81254f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37980/0 (socket says 172.21.15.131:37980) 2024-04-26T20:11:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff15b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:56362/0 (socket says 172.21.15.131:56362) 2024-04-26T20:11:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff169200640 1 -- 172.21.15.131:0/1443598487 learned_addr learned my addr 172.21.15.131:0/1443598487 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff163e00640 1 -- 172.21.15.131:0/1443598487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254f0f0 msgr2=0x55c81254f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff163e00640 1 --2- 172.21.15.131:0/1443598487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254f0f0 0x55c81254f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff163e00640 1 -- 172.21.15.131:0/1443598487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 msgr2=0x55c8125570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff163e00640 1 --2- 172.21.15.131:0/1443598487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.449+0000 7ff163e00640 1 -- 172.21.15.131:0/1443598487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8121c52d0 con 0x55c81254fae0 2024-04-26T20:11:22.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.450+0000 7ff163e00640 1 --2- 172.21.15.131:0/1443598487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125545e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1540eef10 tx=0x7ff1540f7cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655162beef0ff862 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:22.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.450+0000 7ff161600640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1480a3c40 0x7ff1480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.450+0000 7ff159600640 1 -- 172.21.15.131:0/1443598487 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154100070 con 0x55c81254fae0 2024-04-26T20:11:22.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.450+0000 7ff161600640 1 -- 172.21.15.131:0/674360873 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff1540f1840 con 0x7ff164075310 2024-04-26T20:11:22.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.450+0000 7ff159600640 1 -- 172.21.15.131:0/1443598487 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1540bdde0 con 0x55c81254fae0 2024-04-26T20:11:22.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.450+0000 7ff159600640 1 -- 172.21.15.131:0/1443598487 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1540b9980 con 0x55c81254fae0 2024-04-26T20:11:22.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.450+0000 7ff169200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1480a3c40 0x7ff1480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1443598487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 msgr2=0x55c8125545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff169200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1480a3c40 0x7ff1480a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c812550340 tx=0x7ff15c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1443598487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125545e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1540eef10 tx=0x7ff1540f7cb0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1443598487 shutdown_connections 2024-04-26T20:11:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1443598487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1443598487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125545e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1443598487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254f0f0 0x55c81254f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1443598487 >> 172.21.15.131:0/1443598487 conn(0x55c8124e3d10 msgr2=0x55c81254ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:22.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1443598487 shutdown_connections 2024-04-26T20:11:22.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.451+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1443598487 wait complete. 2024-04-26T20:11:22.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 -- start start 2024-04-26T20:11:22.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 Processor -- start 2024-04-26T20:11:22.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c812559260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:22.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254fae0 0x55c8125597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:22.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c8125579c0 con 0x55c81254f0f0 2024-04-26T20:11:22.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c81243af70 con 0x55c812554c50 2024-04-26T20:11:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.452+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c812557b40 con 0x55c81254fae0 2024-04-26T20:11:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff16dfc7740 10 client.0 _ll_register_callbacks cb 0x55c8124a9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:11:22.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c812559260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c812559260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45722/0 (socket says 172.21.15.131:45722) 2024-04-26T20:11:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 -- 172.21.15.131:0/3389242329 learned_addr learned my addr 172.21.15.131:0/3389242329 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff15b400640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254fae0 0x55c8125597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff169200640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:22.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 -- 172.21.15.131:0/3389242329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254fae0 msgr2=0x55c8125597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:22.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254fae0 0x55c8125597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 -- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 msgr2=0x55c8125607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:22.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 -- 172.21.15.131:0/3389242329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81228f6b0 con 0x55c81254f0f0 2024-04-26T20:11:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff15b400640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254fae0 0x55c8125597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:22.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff163e00640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c812559260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff1540c5e20 tx=0x7ff154005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866628bfa0641b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:22.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.453+0000 7ff152000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:11:22.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff152000640 1 -- 172.21.15.131:0/3389242329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154100070 con 0x55c81254f0f0 2024-04-26T20:11:22.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff152000640 1 -- 172.21.15.131:0/3389242329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff1540b9d50 con 0x55c81254f0f0 2024-04-26T20:11:22.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1540bdde0 con 0x55c81254f0f0 2024-04-26T20:11:22.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff152000640 1 -- 172.21.15.131:0/3389242329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff154037310 con 0x55c81254f0f0 2024-04-26T20:11:22.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff16dfc7740 10 client.15060 fetch_fsmap learned FSMap version 26 2024-04-26T20:11:22.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8125579c0 con 0x55c81254f0f0 2024-04-26T20:11:23.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff152000640 1 -- 172.21.15.131:0/3389242329 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1540d3070 con 0x55c81254f0f0 2024-04-26T20:11:23.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff152000640 1 -- 172.21.15.131:0/3389242329 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff15404d030 con 0x55c81254f0f0 2024-04-26T20:11:23.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff16dfc7740 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:11:23.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.454+0000 7ff16dfc7740 10 client.15060 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:11:23.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.455+0000 7ff16dfc7740 20 client.15060 populate_metadata read hostname 'smithi131' 2024-04-26T20:11:23.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.455+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c812565c10 0x55c812567080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:23.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.455+0000 7ff16dfc7740 4 client.15060 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:11:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.455+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8124c1210 con 0x55c812565c10 2024-04-26T20:11:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.455+0000 7ff15b400640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c812565c10 0x55c812567080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.455+0000 7ff15b400640 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c812565c10 0x55c812567080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d9f2726c10c0f60 server_cookie=8e4f2831a4755053 in_seq=0 out_seq=0 2024-04-26T20:11:23.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.455+0000 7ff152000640 10 client.15060 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:11:23.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.456+0000 7ff152000640 1 -- 172.21.15.131:0/3389242329 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8124c1210 con 0x55c812565c10 2024-04-26T20:11:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.456+0000 7ff152000640 10 client.15060 handle_command_reply: tid=0 2024-04-26T20:11:23.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.456+0000 7ff16dfc7740 1 client.15060 shutdown 2024-04-26T20:11:23.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 msgr2=0x55c812559260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:23.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c812559260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff1540c5e20 tx=0x7ff154005f50 comp rx=0 tx=0).stop 2024-04-26T20:11:23.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 shutdown_connections 2024-04-26T20:11:23.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c812565c10 0x55c812567080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:23.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c812554c50 0x55c8125607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:23.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c81254fae0 0x55c8125597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:23.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/3389242329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c812559260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:23.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff152000640 0 client.15060 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:11:23.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 >> 172.21.15.131:0/3389242329 conn(0x55c8124e3d10 msgr2=0x55c812549eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:23.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 shutdown_connections 2024-04-26T20:11:23.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 1 -- 172.21.15.131:0/3389242329 wait complete. 2024-04-26T20:11:23.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.457+0000 7ff16dfc7740 20 client.15060 trim_cache size 0 max 16384 2024-04-26T20:11:23.585 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:22 smithi148 ceph-mon[32281]: pgmap v451: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:23.591 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:22 smithi157 ceph-mon[32703]: pgmap v451: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:23.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.476+0000 7ff16dfc7740 1 Processor -- start 2024-04-26T20:11:23.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.476+0000 7ff16dfc7740 1 -- start start 2024-04-26T20:11:23.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254f0f0 0x55c812559580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:23.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:23.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c812557e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:23.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c8125653c0 con 0x55c81254fae0 2024-04-26T20:11:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c812565a90 con 0x55c81254f0f0 2024-04-26T20:11:23.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c812566160 con 0x55c812554c50 2024-04-26T20:11:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff15b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:23.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff15b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45726/0 (socket says 172.21.15.131:45726) 2024-04-26T20:11:23.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff15b400640 1 -- 172.21.15.131:0/1480542365 learned_addr learned my addr 172.21.15.131:0/1480542365 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:23.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff163e00640 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254f0f0 0x55c812559580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:23.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff169200640 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c812557e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:23.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff15b400640 1 -- 172.21.15.131:0/1480542365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 msgr2=0x55c812557e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff15b400640 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c812557e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:23.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.477+0000 7ff15b400640 1 -- 172.21.15.131:0/1480542365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254f0f0 msgr2=0x55c812559580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:23.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.478+0000 7ff15b400640 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254f0f0 0x55c812559580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:23.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.478+0000 7ff15b400640 1 -- 172.21.15.131:0/1480542365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8125651f0 con 0x55c81254fae0 2024-04-26T20:11:23.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.478+0000 7ff163e00640 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254f0f0 0x55c812559580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:23.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.478+0000 7ff15b400640 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125607a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55c81254ba20 tx=0x7ff14c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab087876c61d049 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:23.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.478+0000 7ff158c00640 1 -- 172.21.15.131:0/1480542365 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c005220 con 0x55c81254fae0 2024-04-26T20:11:23.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.478+0000 7ff158c00640 1 -- 172.21.15.131:0/1480542365 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff14c004c50 con 0x55c81254fae0 2024-04-26T20:11:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.478+0000 7ff158c00640 1 -- 172.21.15.131:0/1480542365 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c020640 con 0x55c81254fae0 2024-04-26T20:11:24.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1480542365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 msgr2=0x55c8125607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125607a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55c81254ba20 tx=0x7ff14c005f90 comp rx=0 tx=0).stop 2024-04-26T20:11:24.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1480542365 shutdown_connections 2024-04-26T20:11:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c812557e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:24.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254fae0 0x55c8125607a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:24.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/1480542365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254f0f0 0x55c812559580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:24.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1480542365 >> 172.21.15.131:0/1480542365 conn(0x55c8124e3d10 msgr2=0x55c812558e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:24.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1480542365 shutdown_connections 2024-04-26T20:11:24.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 -- 172.21.15.131:0/1480542365 wait complete. 2024-04-26T20:11:24.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 -- start start 2024-04-26T20:11:24.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.479+0000 7ff16dfc7740 1 Processor -- start 2024-04-26T20:11:24.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c81255bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254fae0 0x55c81255c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff16dfc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c81255f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:24.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c812566f00 con 0x55c81254f0f0 2024-04-26T20:11:24.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c812566830 con 0x55c81254fae0 2024-04-26T20:11:24.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff16dfc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c8125675d0 con 0x55c812554c50 2024-04-26T20:11:24.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff16dfc7740 10 client.0 _ll_register_callbacks cb 0x7ff14c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:11:24.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c81255bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c81255bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45728/0 (socket says 172.21.15.131:45728) 2024-04-26T20:11:24.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff169200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c81255f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:24.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff163e00640 1 -- 172.21.15.131:0/468174718 learned_addr learned my addr 172.21.15.131:0/468174718 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:24.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.480+0000 7ff15b400640 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254fae0 0x55c81255c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:24.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff169200640 1 -- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254fae0 msgr2=0x55c81255c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:24.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff169200640 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254fae0 0x55c81255c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff169200640 1 -- 172.21.15.131:0/468174718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 msgr2=0x55c81255bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:11:24.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff169200640 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c81255bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff169200640 1 -- 172.21.15.131:0/468174718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81255fae0 con 0x55c812554c50 2024-04-26T20:11:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff163e00640 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c81255bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff169200640 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c81255f4f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009530 tx=0x7ff15c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7ad037180c8cc7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff152000640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:11:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff152000640 1 -- 172.21.15.131:0/468174718 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c01e6e0 con 0x55c812554c50 2024-04-26T20:11:24.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff14c004c50 con 0x55c812554c50 2024-04-26T20:11:24.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.481+0000 7ff152000640 1 -- 172.21.15.131:0/468174718 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff15c01e880 con 0x55c812554c50 2024-04-26T20:11:24.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.482+0000 7ff152000640 1 -- 172.21.15.131:0/468174718 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff15c038050 con 0x55c812554c50 2024-04-26T20:11:24.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.482+0000 7ff16dfc7740 10 client.25189 fetch_fsmap learned FSMap version 26 2024-04-26T20:11:24.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.482+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8125cbce0 con 0x55c812554c50 2024-04-26T20:11:24.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.482+0000 7ff152000640 1 -- 172.21.15.131:0/468174718 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff14c004c50 con 0x55c812554c50 2024-04-26T20:11:24.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff152000640 1 -- 172.21.15.131:0/468174718 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff15c02ac40 con 0x55c812554c50 2024-04-26T20:11:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff16dfc7740 10 client.25189 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:11:24.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff16dfc7740 10 client.25189 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:11:24.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff16dfc7740 20 client.25189 populate_metadata read hostname 'smithi131' 2024-04-26T20:11:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c8125d0970 0x55c8125ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:24.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff16dfc7740 4 client.25189 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:11:24.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "status"}) -- 0x55c8124bb080 con 0x55c8125d0970 2024-04-26T20:11:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.483+0000 7ff163e00640 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c8125d0970 0x55c8125ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:24.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff163e00640 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c8125d0970 0x55c8125ccdb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9d72fc175b33f7 server_cookie=d9874466191d671a in_seq=0 out_seq=0 2024-04-26T20:11:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff152000640 10 client.25189 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:11:24.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff152000640 1 -- 172.21.15.131:0/468174718 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c8124bb080 con 0x55c8125d0970 2024-04-26T20:11:24.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff152000640 10 client.25189 handle_command_reply: tid=0 2024-04-26T20:11:24.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 client.25189 shutdown 2024-04-26T20:11:24.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 msgr2=0x55c81255f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c81255f4f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009530 tx=0x7ff15c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 shutdown_connections 2024-04-26T20:11:25.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c8125d0970 0x55c8125ccdb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c812554c50 0x55c81255f4f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c81254fae0 0x55c81255c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff152000640 0 client.25189 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:11:25.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 --2- 172.21.15.131:0/468174718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c81254f0f0 0x55c81255bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 >> 172.21.15.131:0/468174718 conn(0x55c8124e3d10 msgr2=0x55c812557e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:25.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 shutdown_connections 2024-04-26T20:11:25.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 1 -- 172.21.15.131:0/468174718 wait complete. 2024-04-26T20:11:25.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.484+0000 7ff16dfc7740 20 client.25189 trim_cache size 0 max 16384 2024-04-26T20:11:25.108 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:11:25.113 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T20:11:25.121 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-04-26T20:11:25.186 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24575, 2024-04-26T20:11:25.224 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-26T20:11:25.230 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-26T20:11:25.237 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-26T20:11:25.305 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 226.76004238499999, 2024-04-26T20:11:25.331 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-04-26T20:11:25.338 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 97, 2024-04-26T20:11:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T20:11:25.355 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 249.339705931 2024-04-26T20:11:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:11:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1480a3c40 msgr2=0x7ff1480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1480a3c40 0x7ff1480a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c812550340 tx=0x7ff15c005f90 comp rx=0 tx=0).stop 2024-04-26T20:11:25.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff164075310 msgr2=0x7ff16419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff164075310 0x7ff16419adb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff15402f7a0 tx=0x7ff15402f650 comp rx=0 tx=0).stop 2024-04-26T20:11:25.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 shutdown_connections 2024-04-26T20:11:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff16410a820 0x7ff1641a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff1480a3c40 0x7ff1480a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1640768f0 0x7ff16419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 --2- 172.21.15.131:0/674360873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff164075310 0x7ff16419adb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 >> 172.21.15.131:0/674360873 conn(0x7ff1640fdee0 msgr2=0x7ff1641067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:25.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 shutdown_connections 2024-04-26T20:11:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:21.485+0000 7ff16b200640 1 -- 172.21.15.131:0/674360873 wait complete. 2024-04-26T20:11:25.621 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T20:11:25.627 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T20:11:25.641 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:11:25.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T20:11:25.655 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:22 smithi131 ceph-mon[27919]: pgmap v451: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:25.662 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:24 smithi131 ceph-mon[27919]: pgmap v452: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:25.668 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:24 smithi148 ceph-mon[32281]: pgmap v452: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:25.674 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:24 smithi157 ceph-mon[32703]: pgmap v452: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:11:27.154 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:26 smithi131 ceph-mon[27919]: pgmap v453: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:27.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:26 smithi157 ceph-mon[32703]: pgmap v453: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:27.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:26 smithi148 ceph-mon[32281]: pgmap v453: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:28.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 -- 172.21.15.131:0/469285143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4381091d0 msgr2=0x7fd43810b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:28.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 --2- 172.21.15.131:0/469285143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4381091d0 0x7fd43810b5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd43000b0a0 tx=0x7fd43000b3b0 comp rx=0 tx=0).stop 2024-04-26T20:11:28.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 -- 172.21.15.131:0/469285143 shutdown_connections 2024-04-26T20:11:28.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 --2- 172.21.15.131:0/469285143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4381091d0 0x7fd43810b5e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:28.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 --2- 172.21.15.131:0/469285143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd438103f70 0x7fd438106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 --2- 172.21.15.131:0/469285143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 0x7fd438108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:28.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 -- 172.21.15.131:0/469285143 >> 172.21.15.131:0/469285143 conn(0x7fd4380fd920 msgr2=0x7fd4380ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:28.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 -- 172.21.15.131:0/469285143 shutdown_connections 2024-04-26T20:11:28.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.560+0000 7fd43fa00700 1 -- 172.21.15.131:0/469285143 wait complete. 2024-04-26T20:11:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 Processor -- start 2024-04-26T20:11:28.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 -- start start 2024-04-26T20:11:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd438103f70 0x7fd4381038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 0x7fd438101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4381091d0 0x7fd438102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810ef40 con 0x7fd438103f70 2024-04-26T20:11:28.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810edc0 con 0x7fd4381068a0 2024-04-26T20:11:28.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.562+0000 7fd43fa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810f0c0 con 0x7fd4381091d0 2024-04-26T20:11:28.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd43d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd438103f70 0x7fd4381038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 0x7fd438101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:28.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 0x7fd438101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:47078/0 (socket says 172.21.15.131:47078) 2024-04-26T20:11:29.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 -- 172.21.15.131:0/3983556525 learned_addr learned my addr 172.21.15.131:0/3983556525 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:29.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd43da00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4381091d0 0x7fd438102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:29.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 -- 172.21.15.131:0/3983556525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4381091d0 msgr2=0x7fd438102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:29.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4381091d0 0x7fd438102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 -- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd438103f70 msgr2=0x7fd4381038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd438103f70 0x7fd4381038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.563+0000 7fd437e00700 1 -- 172.21.15.131:0/3983556525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd438102d10 con 0x7fd4381068a0 2024-04-26T20:11:29.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.564+0000 7fd437e00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 0x7fd438101f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd42400eb30 tx=0x7fd42400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:29.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.564+0000 7fd435600700 1 -- 172.21.15.131:0/3983556525 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42400cc90 con 0x7fd4381068a0 2024-04-26T20:11:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.564+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4381a3730 con 0x7fd4381068a0 2024-04-26T20:11:29.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.564+0000 7fd435600700 1 -- 172.21.15.131:0/3983556525 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4240044d0 con 0x7fd4381068a0 2024-04-26T20:11:29.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.564+0000 7fd435600700 1 -- 172.21.15.131:0/3983556525 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424010430 con 0x7fd4381068a0 2024-04-26T20:11:29.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.564+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381a3c40 con 0x7fd4381068a0 2024-04-26T20:11:29.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.565+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43806a110 con 0x7fd4381068a0 2024-04-26T20:11:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.567+0000 7fd435600700 1 -- 172.21.15.131:0/3983556525 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd424003ea0 con 0x7fd4381068a0 2024-04-26T20:11:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.569+0000 7fd435600700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4200a3cd0 0x7fd4200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:29.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.569+0000 7fd435600700 1 -- 172.21.15.131:0/3983556525 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd4240d0c50 con 0x7fd4381068a0 2024-04-26T20:11:29.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.569+0000 7fd43d000700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4200a3cd0 0x7fd4200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:29.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.570+0000 7fd43d000700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4200a3cd0 0x7fd4200a6190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009c60 tx=0x7fd42c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:29.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.573+0000 7fd435600700 1 -- 172.21.15.131:0/3983556525 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd42409d080 con 0x7fd4381068a0 2024-04-26T20:11:29.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:28 smithi157 ceph-mon[32703]: pgmap v454: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:29.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.817+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd438063860 con 0x7fd4200a3cd0 2024-04-26T20:11:29.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.819+0000 7fd435600700 1 -- 172.21.15.131:0/3983556525 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd438063860 con 0x7fd4200a3cd0 2024-04-26T20:11:29.323 INFO:teuthology.orchestra.run.smithi131.stdout:Backend: cephadm 2024-04-26T20:11:29.330 INFO:teuthology.orchestra.run.smithi131.stdout:Available: Yes 2024-04-26T20:11:29.338 INFO:teuthology.orchestra.run.smithi131.stdout:Paused: No 2024-04-26T20:11:29.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4200a3cd0 msgr2=0x7fd4200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:29.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4200a3cd0 0x7fd4200a6190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009c60 tx=0x7fd42c009260 comp rx=0 tx=0).stop 2024-04-26T20:11:29.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 msgr2=0x7fd438101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:29.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 0x7fd438101f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd42400eb30 tx=0x7fd42400ee40 comp rx=0 tx=0).stop 2024-04-26T20:11:29.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 shutdown_connections 2024-04-26T20:11:29.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4381091d0 0x7fd438102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd438103f70 0x7fd4381038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4200a3cd0 0x7fd4200a6190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:29.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 --2- 172.21.15.131:0/3983556525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4381068a0 0x7fd438101f20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.822+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 >> 172.21.15.131:0/3983556525 conn(0x7fd4380fd920 msgr2=0x7fd438104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:29.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.823+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 shutdown_connections 2024-04-26T20:11:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:28.823+0000 7fd43fa00700 1 -- 172.21.15.131:0/3983556525 wait complete. 2024-04-26T20:11:29.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:28 smithi131 ceph-mon[27919]: pgmap v454: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:29.624 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:28 smithi148 ceph-mon[32281]: pgmap v454: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:29.816 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T20:11:30.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:29 smithi131 ceph-mon[27919]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:30.231 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:29 smithi157 ceph-mon[32703]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:11:30.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:29 smithi148 ceph-mon[32281]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:31.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:30 smithi157 ceph-mon[32703]: pgmap v455: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:31.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:30 smithi131 ceph-mon[27919]: pgmap v455: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:31.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:30 smithi148 ceph-mon[32281]: pgmap v455: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 -- 172.21.15.131:0/3168975840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 msgr2=0x7fdf0010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 --2- 172.21.15.131:0/3168975840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 0x7fdf0010b8e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdef800b0d0 tx=0x7fdef800b3e0 comp rx=0 tx=0).stop 2024-04-26T20:11:32.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 -- 172.21.15.131:0/3168975840 shutdown_connections 2024-04-26T20:11:32.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 --2- 172.21.15.131:0/3168975840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 0x7fdf0010b8e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:32.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 --2- 172.21.15.131:0/3168975840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf00103490 0x7fdf001038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:32.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 --2- 172.21.15.131:0/3168975840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00104690 0x7fdf00104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 -- 172.21.15.131:0/3168975840 >> 172.21.15.131:0/3168975840 conn(0x7fdf000fe9f0 msgr2=0x7fdf00100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:32.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 -- 172.21.15.131:0/3168975840 shutdown_connections 2024-04-26T20:11:32.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.603+0000 7fdf07a00700 1 -- 172.21.15.131:0/3168975840 wait complete. 2024-04-26T20:11:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.604+0000 7fdf07a00700 1 Processor -- start 2024-04-26T20:11:32.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf07a00700 1 -- start start 2024-04-26T20:11:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf07a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00103490 0x7fdf0019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:32.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf07a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf00104690 0x7fdf0019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf07a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 0x7fdf001a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf07a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0010ef80 con 0x7fdf00104690 2024-04-26T20:11:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf07a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0010ee00 con 0x7fdf00105030 2024-04-26T20:11:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf07a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0010f100 con 0x7fdf00103490 2024-04-26T20:11:32.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdeffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf00104690 0x7fdf0019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:32.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf05000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00103490 0x7fdf0019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdeffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf00104690 0x7fdf0019bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60800/0 (socket says 172.21.15.131:60800) 2024-04-26T20:11:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf05000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00103490 0x7fdf0019b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44716/0 (socket says 172.21.15.131:44716) 2024-04-26T20:11:32.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdeffe00700 1 -- 172.21.15.131:0/692002575 learned_addr learned my addr 172.21.15.131:0/692002575 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.605+0000 7fdf05a00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 0x7fdf001a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:32.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdf05000700 1 -- 172.21.15.131:0/692002575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 msgr2=0x7fdf001a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:32.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdf05000700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 0x7fdf001a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdf05000700 1 -- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf00104690 msgr2=0x7fdf0019bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdf05000700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf00104690 0x7fdf0019bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdf05000700 1 -- 172.21.15.131:0/692002575 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf001a3400 con 0x7fdf00103490 2024-04-26T20:11:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdf05000700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00103490 0x7fdf0019b7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e9f0 tx=0x7fdef400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:33.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdefd600700 1 -- 172.21.15.131:0/692002575 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef400cb70 con 0x7fdf00103490 2024-04-26T20:11:33.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdefd600700 1 -- 172.21.15.131:0/692002575 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef400ee90 con 0x7fdf00103490 2024-04-26T20:11:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf001a36f0 con 0x7fdf00103490 2024-04-26T20:11:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.606+0000 7fdefd600700 1 -- 172.21.15.131:0/692002575 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4018680 con 0x7fdf00103490 2024-04-26T20:11:33.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.607+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf001a3c80 con 0x7fdf00103490 2024-04-26T20:11:33.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.608+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0010f740 con 0x7fdf00103490 2024-04-26T20:11:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.610+0000 7fdefd600700 1 -- 172.21.15.131:0/692002575 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdef401f020 con 0x7fdf00103490 2024-04-26T20:11:33.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.611+0000 7fdefd600700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdee80a3d50 0x7fdee80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:33.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.611+0000 7fdefd600700 1 -- 172.21.15.131:0/692002575 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fdef40d1030 con 0x7fdf00103490 2024-04-26T20:11:33.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.611+0000 7fdeffe00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdee80a3d50 0x7fdee80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:33.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.612+0000 7fdeffe00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdee80a3d50 0x7fdee80a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009d60 tx=0x7fdeec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.613+0000 7fdefd600700 1 -- 172.21.15.131:0/692002575 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdef4026070 con 0x7fdf00103490 2024-04-26T20:11:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.854+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdf00076d70 con 0x7fdee80a3d50 2024-04-26T20:11:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.870+0000 7fdefd600700 1 -- 172.21.15.131:0/692002575 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fdf00076d70 con 0x7fdee80a3d50 2024-04-26T20:11:33.233 INFO:teuthology.orchestra.run.smithi131.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T20:11:33.240 INFO:teuthology.orchestra.run.smithi131.stdout:mds.a smithi131 running (4m) 2m ago 4m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 dd30dc3a6547 2024-04-26T20:11:33.248 INFO:teuthology.orchestra.run.smithi131.stdout:mds.b smithi148 running (4m) 2m ago 4m 25.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 6ad835133a7c 2024-04-26T20:11:33.257 INFO:teuthology.orchestra.run.smithi131.stdout:mds.c smithi157 running (4m) 2m ago 4m 18.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 dafe6d5c0750 2024-04-26T20:11:33.263 INFO:teuthology.orchestra.run.smithi131.stdout:mds.d smithi131 running (4m) 2m ago 4m 12.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 6dfa9a2d47bd 2024-04-26T20:11:33.272 INFO:teuthology.orchestra.run.smithi131.stdout:mds.e smithi148 running (4m) 2m ago 4m 14.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 e6a0e595490d 2024-04-26T20:11:33.281 INFO:teuthology.orchestra.run.smithi131.stdout:mds.f smithi157 running (4m) 2m ago 4m 14.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 270914e0e0f8 2024-04-26T20:11:33.289 INFO:teuthology.orchestra.run.smithi131.stdout:mds.g smithi131 running (4m) 2m ago 4m 16.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 cb8a2959c132 2024-04-26T20:11:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:mds.h smithi148 running (4m) 2m ago 4m 17.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 170567daf9ef 2024-04-26T20:11:33.367 INFO:teuthology.orchestra.run.smithi131.stdout:mds.i smithi157 running (4m) 2m ago 4m 18.7M - 19.0.0-3306-gbc612ffb 55dc54019ad8 7ea454b9e86a 2024-04-26T20:11:33.375 INFO:teuthology.orchestra.run.smithi131.stdout:mds.j smithi131 running (4m) 2m ago 4m 12.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c68ace96567d 2024-04-26T20:11:33.396 INFO:teuthology.orchestra.run.smithi131.stdout:mds.k smithi148 running (4m) 2m ago 4m 14.7M - 19.0.0-3306-gbc612ffb 55dc54019ad8 fbec80a25b19 2024-04-26T20:11:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:mds.l smithi157 running (4m) 2m ago 4m 14.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d6aba73548fb 2024-04-26T20:11:33.408 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.x smithi131 *:9283,8765 running (13m) 2m ago 13m 517M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c98006765fd6 2024-04-26T20:11:33.413 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.y smithi148 *:8443,8765 running (10m) 2m ago 10m 454M - 19.0.0-3306-gbc612ffb 55dc54019ad8 4409d5162bb4 2024-04-26T20:11:33.420 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.z smithi157 *:8443,8765 running (11m) 2m ago 10m 453M - 19.0.0-3306-gbc612ffb 55dc54019ad8 8e82ed36c14d 2024-04-26T20:11:33.428 INFO:teuthology.orchestra.run.smithi131.stdout:mon.a smithi131 running (13m) 2m ago 13m 50.4M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 fc567c89abfa 2024-04-26T20:11:33.436 INFO:teuthology.orchestra.run.smithi131.stdout:mon.b smithi148 running (11m) 2m ago 11m 45.9M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 7a3443425100 2024-04-26T20:11:33.446 INFO:teuthology.orchestra.run.smithi131.stdout:mon.c smithi157 running (11m) 2m ago 11m 43.0M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 1d4e58156150 2024-04-26T20:11:33.452 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 smithi131 running (10m) 2m ago 10m 45.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 002caef914ea 2024-04-26T20:11:33.460 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 smithi148 running (10m) 2m ago 10m 47.0M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 d85ff1de9c70 2024-04-26T20:11:33.468 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 smithi157 running (9m) 2m ago 9m 58.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 bba2c7b9b375 2024-04-26T20:11:33.474 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 smithi131 running (9m) 2m ago 9m 51.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 c24ee85052d2 2024-04-26T20:11:33.480 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 smithi148 running (8m) 2m ago 8m 43.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 9a82d73b1efc 2024-04-26T20:11:33.487 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 smithi157 running (8m) 2m ago 8m 45.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 5691987efdbd 2024-04-26T20:11:33.494 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 smithi131 running (7m) 2m ago 7m 41.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 2c5d73af42d9 2024-04-26T20:11:33.597 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 smithi148 running (7m) 2m ago 7m 44.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 bd92ace124ca 2024-04-26T20:11:33.604 INFO:teuthology.orchestra.run.smithi131.stdout:osd.8 smithi157 running (6m) 2m ago 6m 51.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 485fbecd0471 2024-04-26T20:11:33.656 INFO:teuthology.orchestra.run.smithi131.stdout:osd.9 smithi131 running (6m) 2m ago 6m 49.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 1696fbeb42ac 2024-04-26T20:11:33.664 INFO:teuthology.orchestra.run.smithi131.stdout:osd.10 smithi148 running (5m) 2m ago 5m 44.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 80652a7b488d 2024-04-26T20:11:33.669 INFO:teuthology.orchestra.run.smithi131.stdout:osd.11 smithi157 running (5m) 2m ago 5m 41.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 bdc2b35ce822 2024-04-26T20:11:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdee80a3d50 msgr2=0x7fdee80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:33.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdee80a3d50 0x7fdee80a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009d60 tx=0x7fdeec009260 comp rx=0 tx=0).stop 2024-04-26T20:11:33.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00103490 msgr2=0x7fdf0019b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:33.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00103490 0x7fdf0019b7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e9f0 tx=0x7fdef400ed00 comp rx=0 tx=0).stop 2024-04-26T20:11:33.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 shutdown_connections 2024-04-26T20:11:33.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdee80a3d50 0x7fdee80a6210 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:33.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00105030 0x7fdf001a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:33.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf00103490 0x7fdf0019b7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 --2- 172.21.15.131:0/692002575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf00104690 0x7fdf0019bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:33.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 >> 172.21.15.131:0/692002575 conn(0x7fdf000fe9f0 msgr2=0x7fdf00075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 shutdown_connections 2024-04-26T20:11:33.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:32.874+0000 7fdf07a00700 1 -- 172.21.15.131:0/692002575 wait complete. 2024-04-26T20:11:33.872 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:32 smithi131 ceph-mon[27919]: pgmap v456: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:33.918 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:32 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:11:33.924 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T20:11:33.932 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:32 smithi148 ceph-mon[32281]: pgmap v456: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:33.940 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:32 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:11:33.948 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:32 smithi157 ceph-mon[32703]: pgmap v456: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:33.955 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:32 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:11:34.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:33 smithi157 ceph-mon[32703]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:34.235 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:11:34.243 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:11:34.250 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:33 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:11:34.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:33 smithi131 ceph-mon[27919]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:34.261 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:11:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:11:34.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:33 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:11:34.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:33 smithi148 ceph-mon[32281]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:34.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:11:34.476 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:11:34.505 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:33 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:11:34.514 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:11:35.104 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:34 smithi131 ceph-mon[27919]: pgmap v457: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:35.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:34 smithi157 ceph-mon[32703]: pgmap v457: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:35.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:34 smithi148 ceph-mon[32281]: pgmap v457: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:36.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.765+0000 7fce83a00700 1 -- 172.21.15.131:0/4209480841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103f10 msgr2=0x7fce7c10a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:36.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.765+0000 7fce83a00700 1 --2- 172.21.15.131:0/4209480841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103f10 0x7fce7c10a7c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fce7400b0d0 tx=0x7fce7400b3e0 comp rx=0 tx=0).stop 2024-04-26T20:11:36.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.765+0000 7fce83a00700 1 -- 172.21.15.131:0/4209480841 shutdown_connections 2024-04-26T20:11:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.765+0000 7fce83a00700 1 --2- 172.21.15.131:0/4209480841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103f10 0x7fce7c10a7c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:36.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.765+0000 7fce83a00700 1 --2- 172.21.15.131:0/4209480841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c102370 0x7fce7c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:36.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.765+0000 7fce83a00700 1 --2- 172.21.15.131:0/4209480841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c103570 0x7fce7c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.765+0000 7fce83a00700 1 -- 172.21.15.131:0/4209480841 >> 172.21.15.131:0/4209480841 conn(0x7fce7c0fd910 msgr2=0x7fce7c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.766+0000 7fce83a00700 1 -- 172.21.15.131:0/4209480841 shutdown_connections 2024-04-26T20:11:36.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.766+0000 7fce83a00700 1 -- 172.21.15.131:0/4209480841 wait complete. 2024-04-26T20:11:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.767+0000 7fce83a00700 1 Processor -- start 2024-04-26T20:11:37.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.767+0000 7fce83a00700 1 -- start start 2024-04-26T20:11:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce83a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c102370 0x7fce7c106290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce83a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103570 0x7fce7c104950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce83a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c103f10 0x7fce7c104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:37.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce83a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c10dde0 con 0x7fce7c103570 2024-04-26T20:11:37.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce83a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c10dc60 con 0x7fce7c102370 2024-04-26T20:11:37.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce83a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c10df60 con 0x7fce7c103f10 2024-04-26T20:11:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce81000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c102370 0x7fce7c106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:37.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce81000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c102370 0x7fce7c106290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:53258/0 (socket says 172.21.15.131:53258) 2024-04-26T20:11:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce73e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103570 0x7fce7c104950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:37.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce81000700 1 -- 172.21.15.131:0/3873082543 learned_addr learned my addr 172.21.15.131:0/3873082543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:37.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.768+0000 7fce81a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c103f10 0x7fce7c104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:37.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce81000700 1 -- 172.21.15.131:0/3873082543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c103f10 msgr2=0x7fce7c104e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:37.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce81000700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c103f10 0x7fce7c104e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:37.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce81000700 1 -- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103570 msgr2=0x7fce7c104950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:37.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce81000700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103570 0x7fce7c104950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:37.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce81000700 1 -- 172.21.15.131:0/3873082543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c1056d0 con 0x7fce7c102370 2024-04-26T20:11:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce81a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c103f10 0x7fce7c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:37.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce81000700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c102370 0x7fce7c106290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e970 tx=0x7fce6c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:37.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce7a000700 1 -- 172.21.15.131:0/3873082543 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c004220 con 0x7fce7c102370 2024-04-26T20:11:37.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c1a3660 con 0x7fce7c102370 2024-04-26T20:11:37.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c1a3bf0 con 0x7fce7c102370 2024-04-26T20:11:37.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce7a000700 1 -- 172.21.15.131:0/3873082543 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce6c0043c0 con 0x7fce7c102370 2024-04-26T20:11:37.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.769+0000 7fce7a000700 1 -- 172.21.15.131:0/3873082543 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c003c80 con 0x7fce7c102370 2024-04-26T20:11:37.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.771+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7c06a110 con 0x7fce7c102370 2024-04-26T20:11:37.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.773+0000 7fce7a000700 1 -- 172.21.15.131:0/3873082543 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fce6c004870 con 0x7fce7c102370 2024-04-26T20:11:37.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.774+0000 7fce7a000700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fce600a3d50 0x7fce600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:37.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.774+0000 7fce7a000700 1 -- 172.21.15.131:0/3873082543 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fce6c0d0cf0 con 0x7fce7c102370 2024-04-26T20:11:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.774+0000 7fce73e00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fce600a3d50 0x7fce600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:37.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.775+0000 7fce73e00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fce600a3d50 0x7fce600a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce64009c60 tx=0x7fce64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:37.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:36.777+0000 7fce7a000700 1 -- 172.21.15.131:0/3873082543 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fce6c09d120 con 0x7fce7c102370 2024-04-26T20:11:37.452 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:36 smithi148 ceph-mon[32281]: pgmap v458: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:37.461 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:36 smithi157 ceph-mon[32703]: pgmap v458: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:37.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.017+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fce7c0ffaf0 con 0x7fce600a3d50 2024-04-26T20:11:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.031+0000 7fce7a000700 1 -- 172.21.15.131:0/3873082543 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fce7c0ffaf0 con 0x7fce600a3d50 2024-04-26T20:11:37.651 INFO:teuthology.orchestra.run.smithi131.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T20:11:37.657 INFO:teuthology.orchestra.run.smithi131.stdout:mds.all 12/12 2m ago 5m smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-26T20:11:37.665 INFO:teuthology.orchestra.run.smithi131.stdout:mgr 3/3 2m ago 11m smithi131=x;smithi148=y;smithi157=z;count:3 2024-04-26T20:11:37.671 INFO:teuthology.orchestra.run.smithi131.stdout:mon 3/3 2m ago 11m smithi131:172.21.15.131=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c;count:3 2024-04-26T20:11:37.677 INFO:teuthology.orchestra.run.smithi131.stdout:osd 12 2m ago - 2024-04-26T20:11:37.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.033+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fce600a3d50 msgr2=0x7fce600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:37.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.033+0000 7fce83a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fce600a3d50 0x7fce600a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce64009c60 tx=0x7fce64009260 comp rx=0 tx=0).stop 2024-04-26T20:11:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.033+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c102370 msgr2=0x7fce7c106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:37.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.033+0000 7fce83a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c102370 0x7fce7c106290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e970 tx=0x7fce6c00ec80 comp rx=0 tx=0).stop 2024-04-26T20:11:37.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 shutdown_connections 2024-04-26T20:11:37.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c103f10 0x7fce7c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:37.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fce600a3d50 0x7fce600a6210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce7c102370 0x7fce7c106290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 --2- 172.21.15.131:0/3873082543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c103570 0x7fce7c104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:37.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 >> 172.21.15.131:0/3873082543 conn(0x7fce7c0fd910 msgr2=0x7fce7c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 shutdown_connections 2024-04-26T20:11:37.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:37.034+0000 7fce83a00700 1 -- 172.21.15.131:0/3873082543 wait complete. 2024-04-26T20:11:37.863 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:36 smithi131 ceph-mon[27919]: pgmap v458: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:37.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T20:11:38.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:37 smithi157 ceph-mon[32703]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:38.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:37 smithi131 ceph-mon[27919]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:38.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:37 smithi148 ceph-mon[32281]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:11:39.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:38 smithi131 ceph-mon[27919]: pgmap v459: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:39.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:38 smithi157 ceph-mon[32703]: pgmap v459: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:39.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:38 smithi148 ceph-mon[32281]: pgmap v459: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.957+0000 7f0c49200700 1 -- 172.21.15.131:0/3352592665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44103490 msgr2=0x7f0c441038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:41.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.957+0000 7f0c49200700 1 --2- 172.21.15.131:0/3352592665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44103490 0x7f0c441038b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009920 tx=0x7f0c34009c30 comp rx=0 tx=0).stop 2024-04-26T20:11:41.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.958+0000 7f0c49200700 1 -- 172.21.15.131:0/3352592665 shutdown_connections 2024-04-26T20:11:41.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.958+0000 7f0c49200700 1 --2- 172.21.15.131:0/3352592665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44105030 0x7f0c4410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.958+0000 7f0c49200700 1 --2- 172.21.15.131:0/3352592665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44103490 0x7f0c441038b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.958+0000 7f0c49200700 1 --2- 172.21.15.131:0/3352592665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c44104690 0x7f0c44104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:41.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.958+0000 7f0c49200700 1 -- 172.21.15.131:0/3352592665 >> 172.21.15.131:0/3352592665 conn(0x7f0c440fe9f0 msgr2=0x7f0c44100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:41.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.958+0000 7f0c49200700 1 -- 172.21.15.131:0/3352592665 shutdown_connections 2024-04-26T20:11:41.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.958+0000 7f0c49200700 1 -- 172.21.15.131:0/3352592665 wait complete. 2024-04-26T20:11:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.959+0000 7f0c49200700 1 Processor -- start 2024-04-26T20:11:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.959+0000 7f0c49200700 1 -- start start 2024-04-26T20:11:41.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.959+0000 7f0c49200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44103490 0x7f0c4419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:41.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.959+0000 7f0c49200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c44104690 0x7f0c4419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.959+0000 7f0c49200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44105030 0x7f0c441a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c49200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44075b20 con 0x7f0c44103490 2024-04-26T20:11:41.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c49200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c440759a0 con 0x7f0c44105030 2024-04-26T20:11:41.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c49200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44075ca0 con 0x7f0c44104690 2024-04-26T20:11:41.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44103490 0x7f0c4419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:41.434 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:40 smithi148 ceph-mon[32281]: pgmap v460: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:41.465 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:40 smithi157 ceph-mon[32703]: pgmap v460: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:41.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44103490 0x7f0c4419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44534/0 (socket says 172.21.15.131:44534) 2024-04-26T20:11:41.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 -- 172.21.15.131:0/1666611190 learned_addr learned my addr 172.21.15.131:0/1666611190 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:41.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41400700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c44104690 0x7f0c4419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:41.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c42800700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44105030 0x7f0c441a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:41.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 -- 172.21.15.131:0/1666611190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c44104690 msgr2=0x7f0c4419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:41.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c44104690 0x7f0c4419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:41.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 -- 172.21.15.131:0/1666611190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44105030 msgr2=0x7f0c441a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44105030 0x7f0c441a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:41.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41e00700 1 -- 172.21.15.131:0/1666611190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c441a3540 con 0x7f0c44103490 2024-04-26T20:11:41.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c41400700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c44104690 0x7f0c4419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:41.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.960+0000 7f0c42800700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44105030 0x7f0c441a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:41.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.961+0000 7f0c41e00700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44103490 0x7f0c4419b8e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011040 tx=0x7f0c34005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:41.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.961+0000 7f0c3aa00700 1 -- 172.21.15.131:0/1666611190 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34027070 con 0x7f0c44103490 2024-04-26T20:11:41.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.961+0000 7f0c3aa00700 1 -- 172.21.15.131:0/1666611190 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c3400ba10 con 0x7f0c44103490 2024-04-26T20:11:41.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.961+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c441a37d0 con 0x7f0c44103490 2024-04-26T20:11:41.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.961+0000 7f0c3aa00700 1 -- 172.21.15.131:0/1666611190 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34004910 con 0x7f0c44103490 2024-04-26T20:11:41.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.961+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c441a3c20 con 0x7f0c44103490 2024-04-26T20:11:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.963+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4406a140 con 0x7f0c44103490 2024-04-26T20:11:41.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.965+0000 7f0c3aa00700 1 -- 172.21.15.131:0/1666611190 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0c3402a070 con 0x7f0c44103490 2024-04-26T20:11:41.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.966+0000 7f0c3aa00700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c280a3d50 0x7f0c280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.966+0000 7f0c3aa00700 1 -- 172.21.15.131:0/1666611190 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0c340d1430 con 0x7f0c44103490 2024-04-26T20:11:41.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.966+0000 7f0c41400700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c280a3d50 0x7f0c280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:41.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.967+0000 7f0c41400700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c280a3d50 0x7f0c280a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c30011490 tx=0x7f0c30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:41.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:40.971+0000 7f0c3aa00700 1 -- 172.21.15.131:0/1666611190 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0c3405f060 con 0x7f0c44103490 2024-04-26T20:11:41.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.204+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0c44100be0 con 0x7f0c280a3d50 2024-04-26T20:11:41.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:40 smithi131 ceph-mon[27919]: pgmap v460: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:41.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.208+0000 7f0c3aa00700 1 -- 172.21.15.131:0/1666611190 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0c44100be0 con 0x7f0c280a3d50 2024-04-26T20:11:41.996 INFO:teuthology.orchestra.run.smithi131.stdout:HOST ADDR LABELS STATUS 2024-04-26T20:11:42.007 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131 172.21.15.131 2024-04-26T20:11:42.020 INFO:teuthology.orchestra.run.smithi131.stdout:smithi148 172.21.15.148 2024-04-26T20:11:42.032 INFO:teuthology.orchestra.run.smithi131.stdout:smithi157 172.21.15.157 2024-04-26T20:11:42.044 INFO:teuthology.orchestra.run.smithi131.stdout:3 hosts in cluster 2024-04-26T20:11:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c280a3d50 msgr2=0x7f0c280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c280a3d50 0x7f0c280a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c30011490 tx=0x7f0c30009260 comp rx=0 tx=0).stop 2024-04-26T20:11:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44103490 msgr2=0x7f0c4419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:42.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44103490 0x7f0c4419b8e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011040 tx=0x7f0c34005270 comp rx=0 tx=0).stop 2024-04-26T20:11:42.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 shutdown_connections 2024-04-26T20:11:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c280a3d50 0x7f0c280a6210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:42.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c44105030 0x7f0c441a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:42.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c44103490 0x7f0c4419b8e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:42.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 --2- 172.21.15.131:0/1666611190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c44104690 0x7f0c4419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:42.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 >> 172.21.15.131:0/1666611190 conn(0x7f0c440fe9f0 msgr2=0x7f0c441004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:42.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 shutdown_connections 2024-04-26T20:11:42.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:41.211+0000 7f0c49200700 1 -- 172.21.15.131:0/1666611190 wait complete. 2024-04-26T20:11:42.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T20:11:42.296 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:41 smithi131 ceph-mon[27919]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:42.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:41 smithi157 ceph-mon[32703]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:42.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:41 smithi148 ceph-mon[32281]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:42.731 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:11:43.146 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:42 smithi131 ceph-mon[27919]: pgmap v461: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:43.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:42 smithi157 ceph-mon[32703]: pgmap v461: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:43.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:42 smithi148 ceph-mon[32281]: pgmap v461: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 -- 172.21.15.131:0/3448144922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 msgr2=0x7fe6b4106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:45.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/3448144922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b4106230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009990 tx=0x7fe6a4009ca0 comp rx=0 tx=0).stop 2024-04-26T20:11:45.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 -- 172.21.15.131:0/3448144922 shutdown_connections 2024-04-26T20:11:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/3448144922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b41090a0 0x7fe6b410b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/3448144922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b4106230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/3448144922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b4106770 0x7fe6b4108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:45.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 -- 172.21.15.131:0/3448144922 >> 172.21.15.131:0/3448144922 conn(0x7fe6b40fd850 msgr2=0x7fe6b40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:45.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:44 smithi131 ceph-mon[27919]: pgmap v462: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:45.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:44 smithi148 ceph-mon[32281]: pgmap v462: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:45.400 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:44 smithi157 ceph-mon[32703]: pgmap v462: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 -- 172.21.15.131:0/3448144922 shutdown_connections 2024-04-26T20:11:45.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.191+0000 7fe6b3e00700 1 -- 172.21.15.131:0/3448144922 wait complete. 2024-04-26T20:11:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.192+0000 7fe6b3e00700 1 Processor -- start 2024-04-26T20:11:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.192+0000 7fe6b3e00700 1 -- start start 2024-04-26T20:11:45.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:45.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b4106770 0x7fe6b419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:45.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b3e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b41090a0 0x7fe6b41a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:45.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b3e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b410ee80 con 0x7fe6b4103e40 2024-04-26T20:11:45.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b3e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b410ed00 con 0x7fe6b41090a0 2024-04-26T20:11:45.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b3e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b410f000 con 0x7fe6b4106770 2024-04-26T20:11:45.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:45.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b4106770 0x7fe6b419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:45.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36028/0 (socket says 172.21.15.131:36028) 2024-04-26T20:11:45.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b2a00700 1 -- 172.21.15.131:0/1786702962 learned_addr learned my addr 172.21.15.131:0/1786702962 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.193+0000 7fe6b3400700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b41090a0 0x7fe6b41a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:45.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b2a00700 1 -- 172.21.15.131:0/1786702962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b4106770 msgr2=0x7fe6b419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:45.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b2a00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b4106770 0x7fe6b419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:45.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b2a00700 1 -- 172.21.15.131:0/1786702962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b41090a0 msgr2=0x7fe6b41a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b2a00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b41090a0 0x7fe6b41a2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:45.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b2a00700 1 -- 172.21.15.131:0/1786702962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b41a3320 con 0x7fe6b4103e40 2024-04-26T20:11:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b2000700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b4106770 0x7fe6b419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:45.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b2a00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b419b6c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000c00 tx=0x7fe6a40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:45.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b8a00700 1 -- 172.21.15.131:0/1786702962 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4027070 con 0x7fe6b4103e40 2024-04-26T20:11:46.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b41a35b0 con 0x7fe6b4103e40 2024-04-26T20:11:46.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b8a00700 1 -- 172.21.15.131:0/1786702962 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6a4022470 con 0x7fe6b4103e40 2024-04-26T20:11:46.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.194+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b41a3a60 con 0x7fe6b4103e40 2024-04-26T20:11:46.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.195+0000 7fe6b8a00700 1 -- 172.21.15.131:0/1786702962 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004a30 con 0x7fe6b4103e40 2024-04-26T20:11:46.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.195+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b406a110 con 0x7fe6b4103e40 2024-04-26T20:11:46.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.198+0000 7fe6b8a00700 1 -- 172.21.15.131:0/1786702962 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe6a4009d10 con 0x7fe6b4103e40 2024-04-26T20:11:46.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.199+0000 7fe6b8a00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe6980a3cd0 0x7fe6980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:46.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.199+0000 7fe6b8a00700 1 -- 172.21.15.131:0/1786702962 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe6a40d1720 con 0x7fe6b4103e40 2024-04-26T20:11:46.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.200+0000 7fe6b2000700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe6980a3cd0 0x7fe6980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:46.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.200+0000 7fe6b2000700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe6980a3cd0 0x7fe6980a6190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0011490 tx=0x7fe6a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:46.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.202+0000 7fe6b8a00700 1 -- 172.21.15.131:0/1786702962 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe6a4031080 con 0x7fe6b4103e40 2024-04-26T20:11:46.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.438+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe6b4063860 con 0x7fe6980a3cd0 2024-04-26T20:11:46.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.442+0000 7fe6b8a00700 1 -- 172.21.15.131:0/1786702962 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe6b4063860 con 0x7fe6980a3cd0 2024-04-26T20:11:46.169 INFO:teuthology.orchestra.run.smithi131.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T20:11:46.178 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-04-26T20:11:46.279 INFO:teuthology.orchestra.run.smithi131.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-26T20:11:46.355 INFO:teuthology.orchestra.run.smithi131.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-04-26T20:11:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.445+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe6980a3cd0 msgr2=0x7fe6980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:46.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.445+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe6980a3cd0 0x7fe6980a6190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0011490 tx=0x7fe6a0009260 comp rx=0 tx=0).stop 2024-04-26T20:11:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.445+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 msgr2=0x7fe6b419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:46.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.445+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b419b6c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000c00 tx=0x7fe6a40047b0 comp rx=0 tx=0).stop 2024-04-26T20:11:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.445+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 shutdown_connections 2024-04-26T20:11:46.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.445+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe6980a3cd0 0x7fe6980a6190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:46.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.445+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b41090a0 0x7fe6b41a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:46.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.446+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6b4103e40 0x7fe6b419b6c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:46.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.446+0000 7fe6b3e00700 1 --2- 172.21.15.131:0/1786702962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b4106770 0x7fe6b419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:46.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.446+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 >> 172.21.15.131:0/1786702962 conn(0x7fe6b40fd850 msgr2=0x7fe6b41035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:46.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.446+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 shutdown_connections 2024-04-26T20:11:46.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:45.446+0000 7fe6b3e00700 1 -- 172.21.15.131:0/1786702962 wait complete. 2024-04-26T20:11:46.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:45 smithi131 ceph-mon[27919]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:46.533 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T20:11:46.548 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:11:46.596 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T20:11:46.647 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:45 smithi148 ceph-mon[32281]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:45 smithi157 ceph-mon[32703]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:11:47.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:46 smithi131 ceph-mon[27919]: pgmap v463: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:47.097 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:11:47.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:46 smithi157 ceph-mon[32703]: pgmap v463: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:47.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:46 smithi148 ceph-mon[32281]: pgmap v463: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:49.128 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:48 smithi131 ceph-mon[27919]: pgmap v464: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:49.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:48 smithi157 ceph-mon[32703]: pgmap v464: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:49.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:48 smithi148 ceph-mon[32281]: pgmap v464: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:11:49.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 -- 172.21.15.131:0/361144963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb468075e80 msgr2=0x7fb468076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 --2- 172.21.15.131:0/361144963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb468075e80 0x7fb468076300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4540098f0 tx=0x7fb454009c00 comp rx=0 tx=0).stop 2024-04-26T20:11:49.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 -- 172.21.15.131:0/361144963 shutdown_connections 2024-04-26T20:11:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 --2- 172.21.15.131:0/361144963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46810f980 0x7fb468111d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 --2- 172.21.15.131:0/361144963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468077450 0x7fb4680758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:49.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 --2- 172.21.15.131:0/361144963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb468075e80 0x7fb468076300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:49.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 -- 172.21.15.131:0/361144963 >> 172.21.15.131:0/361144963 conn(0x7fb4680fe960 msgr2=0x7fb468100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:49.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 -- 172.21.15.131:0/361144963 shutdown_connections 2024-04-26T20:11:49.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.425+0000 7fb467e00700 1 -- 172.21.15.131:0/361144963 wait complete. 2024-04-26T20:11:49.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.426+0000 7fb467e00700 1 Processor -- start 2024-04-26T20:11:49.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467e00700 1 -- start start 2024-04-26T20:11:49.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468075e80 0x7fb46819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb468077450 0x7fb4681a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:49.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb46810f980 0x7fb4681a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:49.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb4681154a0 con 0x7fb468075e80 2024-04-26T20:11:49.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb468115320 con 0x7fb46810f980 2024-04-26T20:11:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb468115620 con 0x7fb468077450 2024-04-26T20:11:49.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb466a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468075e80 0x7fb46819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:49.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb466000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb468077450 0x7fb4681a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:49.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb467400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb46810f980 0x7fb4681a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb466a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468075e80 0x7fb46819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36038/0 (socket says 172.21.15.131:36038) 2024-04-26T20:11:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb466000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb468077450 0x7fb4681a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41858/0 (socket says 172.21.15.131:41858) 2024-04-26T20:11:49.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.427+0000 7fb466a00700 1 -- 172.21.15.131:0/680443042 learned_addr learned my addr 172.21.15.131:0/680443042 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:50.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.428+0000 7fb466000700 1 -- 172.21.15.131:0/680443042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb46810f980 msgr2=0x7fb4681a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:50.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.428+0000 7fb466000700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb46810f980 0x7fb4681a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.428+0000 7fb466000700 1 -- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468075e80 msgr2=0x7fb46819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:50.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.428+0000 7fb466000700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468075e80 0x7fb46819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:50.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.428+0000 7fb466000700 1 -- 172.21.15.131:0/680443042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681a7810 con 0x7fb468077450 2024-04-26T20:11:50.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.428+0000 7fb466a00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468075e80 0x7fb46819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:50.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.428+0000 7fb466000700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb468077450 0x7fb4681a0140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb454000c00 tx=0x7fb454005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:50.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.429+0000 7fb45f400700 1 -- 172.21.15.131:0/680443042 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454025070 con 0x7fb468077450 2024-04-26T20:11:50.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.429+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681a7aa0 con 0x7fb468077450 2024-04-26T20:11:50.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.429+0000 7fb45f400700 1 -- 172.21.15.131:0/680443042 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45400ba10 con 0x7fb468077450 2024-04-26T20:11:50.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.429+0000 7fb45f400700 1 -- 172.21.15.131:0/680443042 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454003ba0 con 0x7fb468077450 2024-04-26T20:11:50.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.429+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4681a7f50 con 0x7fb468077450 2024-04-26T20:11:50.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.430+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46806a110 con 0x7fb468077450 2024-04-26T20:11:50.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.432+0000 7fb45f400700 1 -- 172.21.15.131:0/680443042 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb454028070 con 0x7fb468077450 2024-04-26T20:11:50.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.433+0000 7fb45f400700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb44c0a3d50 0x7fb44c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:50.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.434+0000 7fb45f400700 1 -- 172.21.15.131:0/680443042 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb4540d11f0 con 0x7fb468077450 2024-04-26T20:11:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.434+0000 7fb466a00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb44c0a3d50 0x7fb44c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.434+0000 7fb466a00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb44c0a3d50 0x7fb44c0a6210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb458011550 tx=0x7fb458009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.436+0000 7fb45f400700 1 -- 172.21.15.131:0/680443042 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb454013030 con 0x7fb468077450 2024-04-26T20:11:50.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:49 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/680443042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T20:11:50.419 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:49 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/680443042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T20:11:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.713+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb468064030 con 0x7fb468077450 2024-04-26T20:11:50.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.714+0000 7fb45f400700 1 -- 172.21.15.131:0/680443042 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3068 (secure 0 0 0) 0x7fb45409d6b0 con 0x7fb468077450 2024-04-26T20:11:50.460 INFO:teuthology.orchestra.run.smithi131.stdout:e26 2024-04-26T20:11:50.477 INFO:teuthology.orchestra.run.smithi131.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T20:11:50.507 INFO:teuthology.orchestra.run.smithi131.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T20:11:50.537 INFO:teuthology.orchestra.run.smithi131.stdout:legacy client fscid: 1 2024-04-26T20:11:50.544 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:50.551 INFO:teuthology.orchestra.run.smithi131.stdout:Filesystem 'cephfs' (1) 2024-04-26T20:11:50.557 INFO:teuthology.orchestra.run.smithi131.stdout:fs_name cephfs 2024-04-26T20:11:50.564 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 26 2024-04-26T20:11:50.571 INFO:teuthology.orchestra.run.smithi131.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T20:11:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-04-26T20:07:34.194239+0000 2024-04-26T20:11:50.584 INFO:teuthology.orchestra.run.smithi131.stdout:modified 2024-04-26T20:08:57.196498+0000 2024-04-26T20:11:50.590 INFO:teuthology.orchestra.run.smithi131.stdout:tableserver 0 2024-04-26T20:11:50.597 INFO:teuthology.orchestra.run.smithi131.stdout:root 0 2024-04-26T20:11:50.632 INFO:teuthology.orchestra.run.smithi131.stdout:session_timeout 300 2024-04-26T20:11:50.732 INFO:teuthology.orchestra.run.smithi131.stdout:session_autoclose 300 2024-04-26T20:11:50.740 INFO:teuthology.orchestra.run.smithi131.stdout:max_file_size 1099511627776 2024-04-26T20:11:50.748 INFO:teuthology.orchestra.run.smithi131.stdout:max_xattr_size 65536 2024-04-26T20:11:50.768 INFO:teuthology.orchestra.run.smithi131.stdout:required_client_features {} 2024-04-26T20:11:50.804 INFO:teuthology.orchestra.run.smithi131.stdout:last_failure 0 2024-04-26T20:11:50.832 INFO:teuthology.orchestra.run.smithi131.stdout:last_failure_osd_epoch 0 2024-04-26T20:11:50.871 INFO:teuthology.orchestra.run.smithi131.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T20:11:50.880 INFO:teuthology.orchestra.run.smithi131.stdout:max_mds 3 2024-04-26T20:11:50.888 INFO:teuthology.orchestra.run.smithi131.stdout:in 0,1,2 2024-04-26T20:11:50.897 INFO:teuthology.orchestra.run.smithi131.stdout:up {0=24575,1=24571,2=24557} 2024-04-26T20:11:50.904 INFO:teuthology.orchestra.run.smithi131.stdout:failed 2024-04-26T20:11:50.912 INFO:teuthology.orchestra.run.smithi131.stdout:damaged 2024-04-26T20:11:50.920 INFO:teuthology.orchestra.run.smithi131.stdout:stopped 2024-04-26T20:11:50.930 INFO:teuthology.orchestra.run.smithi131.stdout:data_pools [3,4] 2024-04-26T20:11:50.937 INFO:teuthology.orchestra.run.smithi131.stdout:metadata_pool 2 2024-04-26T20:11:51.046 INFO:teuthology.orchestra.run.smithi131.stdout:inline_data disabled 2024-04-26T20:11:51.055 INFO:teuthology.orchestra.run.smithi131.stdout:balancer 2024-04-26T20:11:51.092 INFO:teuthology.orchestra.run.smithi131.stdout:bal_rank_mask -1 2024-04-26T20:11:51.139 INFO:teuthology.orchestra.run.smithi131.stdout:standby_count_wanted 0 2024-04-26T20:11:51.148 INFO:teuthology.orchestra.run.smithi131.stdout:qdb_cluster leader: 24575 members: 24575,24557,24571 2024-04-26T20:11:51.176 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b{0:24575} state up:active seq 7 addr [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.185 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.i{0:24547} state up:standby-replay seq 1 addr [v2:172.21.15.157:6838/1960857281,v1:172.21.15.157:6839/1960857281] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.215 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a{1:24571} state up:active seq 26 addr [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.250 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.h{1:24536} state up:standby-replay seq 1 addr [v2:172.21.15.148:6836/1158905219,v1:172.21.15.148:6837/1158905219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.258 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.g{2:24557} state up:active seq 28 addr [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.267 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c{2:24523} state up:standby-replay seq 1 addr [v2:172.21.15.157:6836/3509306790,v1:172.21.15.157:6837/3509306790] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.273 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:51.281 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:51.288 INFO:teuthology.orchestra.run.smithi131.stdout:Standby daemons: 2024-04-26T20:11:51.295 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:51.303 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.j{-1:14379} state up:standby seq 1 addr [v2:172.21.15.131:6834/3820661643,v1:172.21.15.131:6835/3820661643] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.310 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d{-1:14397} state up:standby seq 1 addr [v2:172.21.15.131:6836/3688291148,v1:172.21.15.131:6837/3688291148] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.352 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.157:6832/1138993646,v1:172.21.15.157:6833/1138993646] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.358 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.k{-1:24452} state up:standby seq 1 addr [v2:172.21.15.148:6832/248659002,v1:172.21.15.148:6833/248659002] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.461 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.e{-1:24500} state up:standby seq 1 addr [v2:172.21.15.148:6834/2202047184,v1:172.21.15.148:6835/2202047184] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.467 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.f{-1:24511} state up:standby seq 1 addr [v2:172.21.15.157:6834/1007065155,v1:172.21.15.157:6835/1007065155] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T20:11:51.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.717+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb44c0a3d50 msgr2=0x7fb44c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:51.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.717+0000 7fb467e00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb44c0a3d50 0x7fb44c0a6210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb458011550 tx=0x7fb458009310 comp rx=0 tx=0).stop 2024-04-26T20:11:51.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.717+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb468077450 msgr2=0x7fb4681a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.717+0000 7fb467e00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb468077450 0x7fb4681a0140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb454000c00 tx=0x7fb454005270 comp rx=0 tx=0).stop 2024-04-26T20:11:51.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 shutdown_connections 2024-04-26T20:11:51.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb44c0a3d50 0x7fb44c0a6210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:51.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb46810f980 0x7fb4681a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:51.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb468075e80 0x7fb46819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:51.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 --2- 172.21.15.131:0/680443042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb468077450 0x7fb4681a0140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:51.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 >> 172.21.15.131:0/680443042 conn(0x7fb4680fe960 msgr2=0x7fb468102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:51.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 shutdown_connections 2024-04-26T20:11:51.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:49.718+0000 7fb467e00700 1 -- 172.21.15.131:0/680443042 wait complete. 2024-04-26T20:11:51.664 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-04-26T20:11:51.755 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:49 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/680443042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T20:11:51.880 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T20:11:51.886 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:50 smithi131 ceph-mon[27919]: pgmap v465: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:51.894 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:50 smithi148 ceph-mon[32281]: pgmap v465: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:51.901 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:50 smithi157 ceph-mon[32703]: pgmap v465: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:11:52.358 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:11:53.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:52 smithi157 ceph-mon[32703]: pgmap v466: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:53.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:52 smithi131 ceph-mon[27919]: pgmap v466: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:53.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:52 smithi148 ceph-mon[32281]: pgmap v466: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 -- 172.21.15.131:0/3453130885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc103e40 msgr2=0x7f50dc106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:54.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 --2- 172.21.15.131:0/3453130885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc103e40 0x7f50dc106230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00b0a0 tx=0x7f50cc00b3b0 comp rx=0 tx=0).stop 2024-04-26T20:11:54.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 -- 172.21.15.131:0/3453130885 shutdown_connections 2024-04-26T20:11:54.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 --2- 172.21.15.131:0/3453130885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50dc1090a0 0x7f50dc10b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:54.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 --2- 172.21.15.131:0/3453130885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc103e40 0x7f50dc106230 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:54.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 --2- 172.21.15.131:0/3453130885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc106770 0x7f50dc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 -- 172.21.15.131:0/3453130885 >> 172.21.15.131:0/3453130885 conn(0x7f50dc0fd830 msgr2=0x7f50dc0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:54.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 -- 172.21.15.131:0/3453130885 shutdown_connections 2024-04-26T20:11:54.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.728+0000 7f50db200700 1 -- 172.21.15.131:0/3453130885 wait complete. 2024-04-26T20:11:54.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.729+0000 7f50db200700 1 Processor -- start 2024-04-26T20:11:54.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50db200700 1 -- start start 2024-04-26T20:11:54.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50db200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc103e40 0x7f50dc1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:55.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50db200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc106770 0x7f50dc199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50db200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50dc1090a0 0x7f50dc1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:55.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50db200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc10ee80 con 0x7f50dc106770 2024-04-26T20:11:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50db200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc10ed00 con 0x7f50dc103e40 2024-04-26T20:11:55.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50db200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc10f000 con 0x7f50dc1090a0 2024-04-26T20:11:55.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50d9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc106770 0x7f50dc199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:55.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50da800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50dc1090a0 0x7f50dc1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:55.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50d9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc106770 0x7f50dc199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56716/0 (socket says 172.21.15.131:56716) 2024-04-26T20:11:55.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50d9400700 1 -- 172.21.15.131:0/1878578683 learned_addr learned my addr 172.21.15.131:0/1878578683 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.730+0000 7f50d9e00700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc103e40 0x7f50dc1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:55.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50da800700 1 -- 172.21.15.131:0/1878578683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc103e40 msgr2=0x7f50dc1994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:55.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50da800700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc103e40 0x7f50dc1994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:55.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50da800700 1 -- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc106770 msgr2=0x7f50dc199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:55.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50da800700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc106770 0x7f50dc199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:55.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50da800700 1 -- 172.21.15.131:0/1878578683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc1a1120 con 0x7f50dc1090a0 2024-04-26T20:11:55.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50d9400700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc106770 0x7f50dc199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50da800700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50dc1090a0 0x7f50dc1a0a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f50d400e970 tx=0x7f50d400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:55.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50d2a00700 1 -- 172.21.15.131:0/1878578683 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4004220 con 0x7f50dc1090a0 2024-04-26T20:11:55.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50d2a00700 1 -- 172.21.15.131:0/1878578683 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d40043c0 con 0x7f50dc1090a0 2024-04-26T20:11:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.731+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc1a1410 con 0x7f50dc1090a0 2024-04-26T20:11:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.732+0000 7f50d2a00700 1 -- 172.21.15.131:0/1878578683 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4010700 con 0x7f50dc1090a0 2024-04-26T20:11:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.732+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc1a1900 con 0x7f50dc1090a0 2024-04-26T20:11:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.733+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50dc06a110 con 0x7f50dc1090a0 2024-04-26T20:11:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.735+0000 7f50d2a00700 1 -- 172.21.15.131:0/1878578683 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f50d4004750 con 0x7f50dc1090a0 2024-04-26T20:11:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.736+0000 7f50d2a00700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f50c00a3cd0 0x7f50c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:55.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.736+0000 7f50d2a00700 1 -- 172.21.15.131:0/1878578683 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f50d40d07a0 con 0x7f50dc1090a0 2024-04-26T20:11:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.736+0000 7f50d9e00700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f50c00a3cd0 0x7f50c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.737+0000 7f50d9e00700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f50c00a3cd0 0x7f50c00a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009be0 tx=0x7f50cc00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:55.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.740+0000 7f50d2a00700 1 -- 172.21.15.131:0/1878578683 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f50d409cbd0 con 0x7f50dc1090a0 2024-04-26T20:11:55.650 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:54 smithi148 ceph-mon[32281]: pgmap v467: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:55.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:54 smithi157 ceph-mon[32703]: pgmap v467: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:55.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.973+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f50dc02e490 con 0x7f50dc1090a0 2024-04-26T20:11:55.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.975+0000 7f50d2a00700 1 -- 172.21.15.131:0/1878578683 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v97) ==== 56+0+5762 (secure 0 0 0) 0x7f50d409c4d0 con 0x7f50dc1090a0 2024-04-26T20:11:55.706 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 97 2024-04-26T20:11:55.713 INFO:teuthology.orchestra.run.smithi131.stdout:fsid 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:11:55.727 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-04-26T19:58:05.764598+0000 2024-04-26T20:11:55.734 INFO:teuthology.orchestra.run.smithi131.stdout:modified 2024-04-26T20:09:15.781789+0000 2024-04-26T20:11:55.781 INFO:teuthology.orchestra.run.smithi131.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T20:11:55.844 INFO:teuthology.orchestra.run.smithi131.stdout:crush_version 27 2024-04-26T20:11:55.849 INFO:teuthology.orchestra.run.smithi131.stdout:full_ratio 0.95 2024-04-26T20:11:55.855 INFO:teuthology.orchestra.run.smithi131.stdout:backfillfull_ratio 0.9 2024-04-26T20:11:55.860 INFO:teuthology.orchestra.run.smithi131.stdout:nearfull_ratio 0.85 2024-04-26T20:11:55.866 INFO:teuthology.orchestra.run.smithi131.stdout:require_min_compat_client luminous 2024-04-26T20:11:55.871 INFO:teuthology.orchestra.run.smithi131.stdout:min_compat_client luminous 2024-04-26T20:11:55.877 INFO:teuthology.orchestra.run.smithi131.stdout:require_osd_release squid 2024-04-26T20:11:55.883 INFO:teuthology.orchestra.run.smithi131.stdout:stretch_mode_enabled false 2024-04-26T20:11:55.890 INFO:teuthology.orchestra.run.smithi131.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T20:11:55.958 INFO:teuthology.orchestra.run.smithi131.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T20:11:56.016 INFO:teuthology.orchestra.run.smithi131.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-26T20:11:56.023 INFO:teuthology.orchestra.run.smithi131.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-26T20:11:56.030 INFO:teuthology.orchestra.run.smithi131.stdout:max_osd 12 2024-04-26T20:11:56.035 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 up in weight 1 up_from 8 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6802/1131821741,v1:172.21.15.131:6803/1131821741] [v2:172.21.15.131:6804/1131821741,v1:172.21.15.131:6805/1131821741] exists,up ec125b6b-d0b4-42f7-80a6-66edb6de6715 2024-04-26T20:11:56.043 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/2432608282,v1:172.21.15.148:6801/2432608282] [v2:172.21.15.148:6802/2432608282,v1:172.21.15.148:6803/2432608282] exists,up 35db8b5f-c13f-4d01-872d-aae172804538 2024-04-26T20:11:56.048 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/527328621,v1:172.21.15.157:6801/527328621] [v2:172.21.15.157:6802/527328621,v1:172.21.15.157:6803/527328621] exists,up cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44 2024-04-26T20:11:56.096 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 up in weight 1 up_from 25 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6810/1043436815,v1:172.21.15.131:6811/1043436815] [v2:172.21.15.131:6812/1043436815,v1:172.21.15.131:6813/1043436815] exists,up c248bb04-986a-4b38-af3b-2e2302a4313e 2024-04-26T20:11:56.168 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 up in weight 1 up_from 33 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/1242934013,v1:172.21.15.148:6809/1242934013] [v2:172.21.15.148:6810/1242934013,v1:172.21.15.148:6811/1242934013] exists,up de2985f5-6971-41c7-aee0-1d437e281514 2024-04-26T20:11:56.175 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 up in weight 1 up_from 38 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/2019329968,v1:172.21.15.157:6809/2019329968] [v2:172.21.15.157:6810/2019329968,v1:172.21.15.157:6811/2019329968] exists,up fa8194d9-9332-4a74-abe4-27a006a4f40a 2024-04-26T20:11:56.184 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 up in weight 1 up_from 46 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6818/1518784346,v1:172.21.15.131:6819/1518784346] [v2:172.21.15.131:6820/1518784346,v1:172.21.15.131:6821/1518784346] exists,up 4db9d2b9-02f6-4d5f-a40f-afbafa778151 2024-04-26T20:11:56.190 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 up in weight 1 up_from 51 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/367491900,v1:172.21.15.148:6817/367491900] [v2:172.21.15.148:6818/367491900,v1:172.21.15.148:6819/367491900] exists,up 5300878b-efa3-49be-afa6-09efbaae8c5b 2024-04-26T20:11:56.195 INFO:teuthology.orchestra.run.smithi131.stdout:osd.8 up in weight 1 up_from 59 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/1452104897,v1:172.21.15.157:6817/1452104897] [v2:172.21.15.157:6818/1452104897,v1:172.21.15.157:6819/1452104897] exists,up 3fb399a8-15a1-48dc-a538-63745daf7fa0 2024-04-26T20:11:56.200 INFO:teuthology.orchestra.run.smithi131.stdout:osd.9 up in weight 1 up_from 67 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6826/529934339,v1:172.21.15.131:6827/529934339] [v2:172.21.15.131:6828/529934339,v1:172.21.15.131:6829/529934339] exists,up 4b119ef2-f46f-4511-9440-283185ed5f09 2024-04-26T20:11:56.205 INFO:teuthology.orchestra.run.smithi131.stdout:osd.10 up in weight 1 up_from 72 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/3795716153,v1:172.21.15.148:6825/3795716153] [v2:172.21.15.148:6826/3795716153,v1:172.21.15.148:6827/3795716153] exists,up 0272c03a-fb92-456b-aee8-3380875cef29 2024-04-26T20:11:56.210 INFO:teuthology.orchestra.run.smithi131.stdout:osd.11 up in weight 1 up_from 80 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/1583012638,v1:172.21.15.157:6825/1583012638] [v2:172.21.15.157:6826/1583012638,v1:172.21.15.157:6827/1583012638] exists,up 0fc8dd61-ca71-4459-ba2e-2a6e07f933d2 2024-04-26T20:11:56.215 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.4 [2,11] 2024-04-26T20:11:56.220 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.1c [8,11] 2024-04-26T20:11:56.264 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.0 [4,6] 2024-04-26T20:11:56.270 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.6 [3,7] 2024-04-26T20:11:56.276 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.b [3,9] 2024-04-26T20:11:56.282 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.17 [4,7,5,8] 2024-04-26T20:11:56.293 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.1a [5,6] 2024-04-26T20:11:56.298 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.1f [1,2] 2024-04-26T20:11:56.355 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.20 [2,9] 2024-04-26T20:11:56.402 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.34 [4,9] 2024-04-26T20:11:56.407 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.3a [4,8] 2024-04-26T20:11:56.414 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.f [5,8] 2024-04-26T20:11:56.420 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.14 [9,10] 2024-04-26T20:11:56.427 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.16 [9,8] 2024-04-26T20:11:56.435 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.18 [9,8] 2024-04-26T20:11:56.443 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/3514900702 expires 2024-04-27T19:59:13.443197+0000 2024-04-26T20:11:56.500 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6801/3789992830 expires 2024-04-27T19:59:13.443197+0000 2024-04-26T20:11:56.507 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6800/3789992830 expires 2024-04-27T19:59:13.443197+0000 2024-04-26T20:11:56.515 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6801/2560715060 expires 2024-04-27T19:58:38.942812+0000 2024-04-26T20:11:56.610 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/2579641682 expires 2024-04-27T19:59:13.443197+0000 2024-04-26T20:11:56.689 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6800/2560715060 expires 2024-04-27T19:58:38.942812+0000 2024-04-26T20:11:56.725 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/3589087841 expires 2024-04-27T19:58:38.942812+0000 2024-04-26T20:11:56.730 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/3946098705 expires 2024-04-27T19:59:13.443197+0000 2024-04-26T20:11:56.744 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/1232073682 expires 2024-04-27T19:58:38.942812+0000 2024-04-26T20:11:56.750 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/537078740 expires 2024-04-27T19:58:38.942812+0000 2024-04-26T20:11:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f50c00a3cd0 msgr2=0x7f50c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:56.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f50c00a3cd0 0x7f50c00a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009be0 tx=0x7f50cc00b700 comp rx=0 tx=0).stop 2024-04-26T20:11:56.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50dc1090a0 msgr2=0x7f50dc1a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:56.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50dc1090a0 0x7f50dc1a0a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f50d400e970 tx=0x7f50d400ec80 comp rx=0 tx=0).stop 2024-04-26T20:11:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 shutdown_connections 2024-04-26T20:11:56.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50dc1090a0 0x7f50dc1a0a80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:56.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f50c00a3cd0 0x7f50c00a6190 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009be0 tx=0x7f50cc00b700 comp rx=0 tx=0).stop 2024-04-26T20:11:56.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc103e40 0x7f50dc1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:56.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 --2- 172.21.15.131:0/1878578683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50dc106770 0x7f50dc199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:56.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.979+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 >> 172.21.15.131:0/1878578683 conn(0x7f50dc0fd830 msgr2=0x7f50dc104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:56.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.980+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 shutdown_connections 2024-04-26T20:11:57.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:54.980+0000 7f50db200700 1 -- 172.21.15.131:0/1878578683 wait complete. 2024-04-26T20:11:57.029 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:54 smithi131 ceph-mon[27919]: pgmap v467: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:11:57.035 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T20:11:57.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:11:57.064 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:55 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1878578683' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T20:11:57.071 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:55 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1878578683' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T20:11:57.113 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:55 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1878578683' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T20:11:57.255 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:56 smithi131 ceph-mon[27919]: pgmap v468: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.286+0000 7f6242600640 1 -- 172.21.15.131:0/4139494506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 msgr2=0x7f623c10cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.286+0000 7f6242600640 1 --2- 172.21.15.131:0/4139494506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 0x7f623c10cdc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6234009be0 tx=0x7f623402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:11:57.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.286+0000 7f6242600640 1 -- 172.21.15.131:0/4139494506 shutdown_connections 2024-04-26T20:11:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.286+0000 7f6242600640 1 --2- 172.21.15.131:0/4139494506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 0x7f623c10cdc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.286+0000 7f6242600640 1 --2- 172.21.15.131:0/4139494506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 0x7f623c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:57.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.286+0000 7f6242600640 1 --2- 172.21.15.131:0/4139494506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c104980 0x7f623c104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.286+0000 7f6242600640 1 -- 172.21.15.131:0/4139494506 >> 172.21.15.131:0/4139494506 conn(0x7f623c0fff10 msgr2=0x7f623c102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.287+0000 7f6242600640 1 -- 172.21.15.131:0/4139494506 shutdown_connections 2024-04-26T20:11:57.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.287+0000 7f6242600640 1 -- 172.21.15.131:0/4139494506 wait complete. 2024-04-26T20:11:57.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 Processor -- start 2024-04-26T20:11:57.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 -- start start 2024-04-26T20:11:57.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c104980 0x7f623c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 0x7f623c19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:57.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 0x7f623c1a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:57.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f623c110490 con 0x7f623c105b70 2024-04-26T20:11:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f623c110310 con 0x7f623c106530 2024-04-26T20:11:57.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6242600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f623c110610 con 0x7f623c104980 2024-04-26T20:11:57.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.288+0000 7f6233e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 0x7f623c19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 0x7f623c19b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56740/0 (socket says 172.21.15.131:56740) 2024-04-26T20:11:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 -- 172.21.15.131:0/3535124089 learned_addr learned my addr 172.21.15.131:0/3535124089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:57.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6241200640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c104980 0x7f623c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6241c00640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 0x7f623c1a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 -- 172.21.15.131:0/3535124089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c104980 msgr2=0x7f623c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:57.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c104980 0x7f623c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:57.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 -- 172.21.15.131:0/3535124089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 msgr2=0x7f623c1a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:57.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 0x7f623c1a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:57.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 -- 172.21.15.131:0/3535124089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c1a2750 con 0x7f623c105b70 2024-04-26T20:11:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6241200640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c104980 0x7f623c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:57.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.289+0000 7f6233e00640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 0x7f623c19b070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f622400e9c0 tx=0x7f622400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:57.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.290+0000 7f623a000640 1 -- 172.21.15.131:0/3535124089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622400cde0 con 0x7f623c105b70 2024-04-26T20:11:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.290+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c1a2a40 con 0x7f623c105b70 2024-04-26T20:11:57.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.290+0000 7f623a000640 1 -- 172.21.15.131:0/3535124089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62240035f0 con 0x7f623c105b70 2024-04-26T20:11:57.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.290+0000 7f623a000640 1 -- 172.21.15.131:0/3535124089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224010690 con 0x7f623c105b70 2024-04-26T20:11:57.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.290+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c1a2fe0 con 0x7f623c105b70 2024-04-26T20:11:58.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.291+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c10bca0 con 0x7f623c105b70 2024-04-26T20:11:58.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.293+0000 7f623a000640 1 -- 172.21.15.131:0/3535124089 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6224004000 con 0x7f623c105b70 2024-04-26T20:11:58.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.294+0000 7f623a000640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f62200a3c40 0x7f62200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:58.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.294+0000 7f623a000640 1 -- 172.21.15.131:0/3535124089 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f62240d0cd0 con 0x7f623c105b70 2024-04-26T20:11:58.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.295+0000 7f6241200640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f62200a3c40 0x7f62200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:58.095 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:56 smithi157 ceph-mon[32703]: pgmap v468: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:58.100 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:56 smithi148 ceph-mon[32281]: pgmap v468: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:11:58.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.295+0000 7f6241200640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f62200a3c40 0x7f62200a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f623c1059d0 tx=0x7f622c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:58.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.299+0000 7f623a000640 1 -- 172.21.15.131:0/3535124089 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6224009510 con 0x7f623c105b70 2024-04-26T20:11:58.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.503+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f623c02dcb0 con 0x7f623c105b70 2024-04-26T20:11:58.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.505+0000 7f623a000640 1 -- 172.21.15.131:0/3535124089 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7f622409d100 con 0x7f623c105b70 2024-04-26T20:11:58.176 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:11:58.182 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:11:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f62200a3c40 msgr2=0x7f62200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:58.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f62200a3c40 0x7f62200a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f623c1059d0 tx=0x7f622c005f90 comp rx=0 tx=0).stop 2024-04-26T20:11:58.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 msgr2=0x7f623c19b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:58.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 0x7f623c19b070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f622400e9c0 tx=0x7f622400ee80 comp rx=0 tx=0).stop 2024-04-26T20:11:58.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 shutdown_connections 2024-04-26T20:11:58.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f623c106530 0x7f623c1a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:58.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f623c105b70 0x7f623c19b070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f62200a3c40 0x7f62200a6100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:58.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 --2- 172.21.15.131:0/3535124089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c104980 0x7f623c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.508+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 >> 172.21.15.131:0/3535124089 conn(0x7f623c0fff10 msgr2=0x7f623c108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:58.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.509+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 shutdown_connections 2024-04-26T20:11:58.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:57.509+0000 7f6242600640 1 -- 172.21.15.131:0/3535124089 wait complete. 2024-04-26T20:11:58.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:11:58.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:11:58.562 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:57 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3535124089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:58.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:57 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3535124089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:58.573 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:57 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3535124089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:11:58.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 -- 172.21.15.131:0/3002381759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 msgr2=0x7f740010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:58.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 --2- 172.21.15.131:0/3002381759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 0x7f740010a440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009900 tx=0x7f73ec02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:11:58.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 -- 172.21.15.131:0/3002381759 shutdown_connections 2024-04-26T20:11:58.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 --2- 172.21.15.131:0/3002381759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f740010a980 0x7f74001112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 --2- 172.21.15.131:0/3002381759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 0x7f740010a440 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 --2- 172.21.15.131:0/3002381759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 0x7f7400109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 -- 172.21.15.131:0/3002381759 >> 172.21.15.131:0/3002381759 conn(0x7f7400077e00 msgr2=0x7f740007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:59.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 -- 172.21.15.131:0/3002381759 shutdown_connections 2024-04-26T20:11:59.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.850+0000 7f7407200640 1 -- 172.21.15.131:0/3002381759 wait complete. 2024-04-26T20:11:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 Processor -- start 2024-04-26T20:11:59.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 -- start start 2024-04-26T20:11:59.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 0x7f740019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 0x7f740019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:59.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f740010a980 0x7f74001a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:59.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74001149d0 con 0x7f7400108e70 2024-04-26T20:11:59.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7400114850 con 0x7f740010a980 2024-04-26T20:11:59.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.851+0000 7f7407200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7400114b50 con 0x7f7400109fc0 2024-04-26T20:11:59.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7405e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 0x7f740019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7405e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 0x7f740019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56768/0 (socket says 172.21.15.131:56768) 2024-04-26T20:11:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7405e00640 1 -- 172.21.15.131:0/2303065300 learned_addr learned my addr 172.21.15.131:0/2303065300 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:11:59.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7406800640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f740010a980 0x7f74001a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7405400640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 0x7f740019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:59.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7406800640 1 -- 172.21.15.131:0/2303065300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 msgr2=0x7f740019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:59.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7406800640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 0x7f740019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7406800640 1 -- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 msgr2=0x7f740019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:59.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7406800640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 0x7f740019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7406800640 1 -- 172.21.15.131:0/2303065300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74001a6c90 con 0x7f740010a980 2024-04-26T20:11:59.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7405e00640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 0x7f740019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:11:59.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.852+0000 7f7406800640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f740010a980 0x7f74001a6620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009fd0 tx=0x7f73fc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.853+0000 7f73faa00640 1 -- 172.21.15.131:0/2303065300 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00ce20 con 0x7f740010a980 2024-04-26T20:11:59.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.853+0000 7f73faa00640 1 -- 172.21.15.131:0/2303065300 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73fc005020 con 0x7f740010a980 2024-04-26T20:11:59.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.853+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74001a6f80 con 0x7f740010a980 2024-04-26T20:11:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.853+0000 7f73faa00640 1 -- 172.21.15.131:0/2303065300 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc010690 con 0x7f740010a980 2024-04-26T20:11:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.853+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001a7440 con 0x7f740010a980 2024-04-26T20:11:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.854+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7400110200 con 0x7f740010a980 2024-04-26T20:11:59.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.856+0000 7f73faa00640 1 -- 172.21.15.131:0/2303065300 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f73fc0054d0 con 0x7f740010a980 2024-04-26T20:11:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.857+0000 7f73faa00640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f73e80a3c40 0x7f73e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:11:59.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.858+0000 7f73faa00640 1 -- 172.21.15.131:0/2303065300 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f73fc0d0fe0 con 0x7f740010a980 2024-04-26T20:11:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.858+0000 7f7405e00640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f73e80a3c40 0x7f73e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:11:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.858+0000 7f7405e00640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f73e80a3c40 0x7f73e80a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f73f4011550 tx=0x7f73f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:11:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:58.860+0000 7f73faa00640 1 -- 172.21.15.131:0/2303065300 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f73fc09d450 con 0x7f740010a980 2024-04-26T20:11:59.656 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:11:59 smithi157 ceph-mon[32703]: pgmap v469: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T20:11:59.661 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:11:59 smithi148 ceph-mon[32281]: pgmap v469: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T20:11:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.061+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f740002dc60 con 0x7f740010a980 2024-04-26T20:11:59.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.062+0000 7f73faa00640 1 -- 172.21.15.131:0/2303065300 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f73fc09cd50 con 0x7f740010a980 2024-04-26T20:11:59.678 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:11:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f73e80a3c40 msgr2=0x7f73e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:59.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f73e80a3c40 0x7f73e80a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f73f4011550 tx=0x7f73f4005f90 comp rx=0 tx=0).stop 2024-04-26T20:11:59.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f740010a980 msgr2=0x7f74001a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:11:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f740010a980 0x7f74001a6620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009fd0 tx=0x7f73fc00ef10 comp rx=0 tx=0).stop 2024-04-26T20:11:59.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 shutdown_connections 2024-04-26T20:11:59.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f740010a980 0x7f74001a6620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f73e80a3c40 0x7f73e80a6100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7400109fc0 0x7f740019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 --2- 172.21.15.131:0/2303065300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7400108e70 0x7f740019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:11:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 >> 172.21.15.131:0/2303065300 conn(0x7f7400077e00 msgr2=0x7f740010d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:11:59.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 shutdown_connections 2024-04-26T20:11:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:11:59.065+0000 7f7407200640 1 -- 172.21.15.131:0/2303065300 wait complete. 2024-04-26T20:12:00.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:12:00.013 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:11:59 smithi131 ceph-mon[27919]: pgmap v469: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T20:12:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 -- 172.21.15.131:0/2353394760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4104a70 msgr2=0x7f60f4104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:00.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 --2- 172.21.15.131:0/2353394760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4104a70 0x7f60f4104e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009900 tx=0x7f60e802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:12:00.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 -- 172.21.15.131:0/2353394760 shutdown_connections 2024-04-26T20:12:00.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 --2- 172.21.15.131:0/2353394760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f4106620 0x7f60f410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:00.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 --2- 172.21.15.131:0/2353394760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f4105c60 0x7f60f41060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:00.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 --2- 172.21.15.131:0/2353394760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4104a70 0x7f60f4104e90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:00.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 -- 172.21.15.131:0/2353394760 >> 172.21.15.131:0/2353394760 conn(0x7f60f40fffe0 msgr2=0x7f60f4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:00.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 -- 172.21.15.131:0/2353394760 shutdown_connections 2024-04-26T20:12:00.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.226+0000 7f60fb000640 1 -- 172.21.15.131:0/2353394760 wait complete. 2024-04-26T20:12:00.517 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:00 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2303065300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:00.524 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:00 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2303065300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.227+0000 7f60fb000640 1 Processor -- start 2024-04-26T20:12:00.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.227+0000 7f60fb000640 1 -- start start 2024-04-26T20:12:00.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fb000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f4104a70 0x7f60f419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:00.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fb000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4105c60 0x7f60f419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:00.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fb000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f4106620 0x7f60f41a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:00.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fb000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41105e0 con 0x7f60f4105c60 2024-04-26T20:12:00.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fb000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4110460 con 0x7f60f4104a70 2024-04-26T20:12:00.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fb000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4110760 con 0x7f60f4106620 2024-04-26T20:12:00.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60f9c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f4104a70 0x7f60f419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:00.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60f9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4105c60 0x7f60f419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fa600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f4106620 0x7f60f41a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60f9c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f4104a70 0x7f60f419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:54564/0 (socket says 172.21.15.131:54564) 2024-04-26T20:12:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60f9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4105c60 0x7f60f419b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56794/0 (socket says 172.21.15.131:56794) 2024-04-26T20:12:00.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60f9c00640 1 -- 172.21.15.131:0/796055560 learned_addr learned my addr 172.21.15.131:0/796055560 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:00.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fa600640 1 -- 172.21.15.131:0/796055560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f4104a70 msgr2=0x7f60f419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:00.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fa600640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f4104a70 0x7f60f419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:00.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fa600640 1 -- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4105c60 msgr2=0x7f60f419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:01.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fa600640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4105c60 0x7f60f419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:01.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.228+0000 7f60fa600640 1 -- 172.21.15.131:0/796055560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f41a2910 con 0x7f60f4106620 2024-04-26T20:12:01.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.229+0000 7f60fa600640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f4106620 0x7f60f41a2270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00ec50 tx=0x7f60ec00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:01.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.229+0000 7f60f2000640 1 -- 172.21.15.131:0/796055560 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec003b70 con 0x7f60f4106620 2024-04-26T20:12:01.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.229+0000 7f60f2000640 1 -- 172.21.15.131:0/796055560 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec003d10 con 0x7f60f4106620 2024-04-26T20:12:01.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.229+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f41a2c00 con 0x7f60f4106620 2024-04-26T20:12:01.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.229+0000 7f60f2000640 1 -- 172.21.15.131:0/796055560 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec010830 con 0x7f60f4106620 2024-04-26T20:12:01.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.229+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f41a3100 con 0x7f60f4106620 2024-04-26T20:12:01.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.230+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f410be30 con 0x7f60f4106620 2024-04-26T20:12:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.232+0000 7f60f2000640 1 -- 172.21.15.131:0/796055560 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f60ec003ec0 con 0x7f60f4106620 2024-04-26T20:12:01.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.233+0000 7f60f2000640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60dc0a3c40 0x7f60dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.234+0000 7f60f2000640 1 -- 172.21.15.131:0/796055560 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f60ec0d0fc0 con 0x7f60f4106620 2024-04-26T20:12:01.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.234+0000 7f60f9c00640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60dc0a3c40 0x7f60dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:01.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.234+0000 7f60f9c00640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60dc0a3c40 0x7f60dc0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.238+0000 7f60f2000640 1 -- 172.21.15.131:0/796055560 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f60ec09d3f0 con 0x7f60f4106620 2024-04-26T20:12:01.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.437+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f60f402dd20 con 0x7f60f4106620 2024-04-26T20:12:01.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.438+0000 7f60f2000640 1 -- 172.21.15.131:0/796055560 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f60ec09ccf0 con 0x7f60f4106620 2024-04-26T20:12:01.442 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:12:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.440+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60dc0a3c40 msgr2=0x7f60dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.440+0000 7f60fb000640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60dc0a3c40 0x7f60dc0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e8005a20 comp rx=0 tx=0).stop 2024-04-26T20:12:01.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.440+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f4106620 msgr2=0x7f60f41a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:01.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.440+0000 7f60fb000640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f4106620 0x7f60f41a2270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00ec50 tx=0x7f60ec00c6a0 comp rx=0 tx=0).stop 2024-04-26T20:12:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.440+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 shutdown_connections 2024-04-26T20:12:01.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.441+0000 7f60fb000640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f60dc0a3c40 0x7f60dc0a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:01.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.441+0000 7f60fb000640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60f4106620 0x7f60f41a2270 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:01.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.441+0000 7f60fb000640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60f4105c60 0x7f60f419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:01.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.441+0000 7f60fb000640 1 --2- 172.21.15.131:0/796055560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f4104a70 0x7f60f419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:01.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.441+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 >> 172.21.15.131:0/796055560 conn(0x7f60f40fffe0 msgr2=0x7f60f4109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:01.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.441+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 shutdown_connections 2024-04-26T20:12:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:00.441+0000 7f60fb000640 1 -- 172.21.15.131:0/796055560 wait complete. 2024-04-26T20:12:01.743 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:00 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2303065300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:01.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:12:01.781 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:01 smithi131 ceph-mon[27919]: pgmap v470: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T20:12:01.820 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:01 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/796055560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:01.873 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:01 smithi157 ceph-mon[32703]: pgmap v470: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T20:12:01.932 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:01 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/796055560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:01.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:01 smithi148 ceph-mon[32281]: pgmap v470: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T20:12:01.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:01 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/796055560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 -- 172.21.15.131:0/1850392821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414108430 msgr2=0x7f041410a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:02.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 --2- 172.21.15.131:0/1850392821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414108430 0x7f041410a880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f040c009900 tx=0x7f040c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:12:02.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 -- 172.21.15.131:0/1850392821 shutdown_connections 2024-04-26T20:12:02.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 --2- 172.21.15.131:0/1850392821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414108430 0x7f041410a880 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:02.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 --2- 172.21.15.131:0/1850392821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0414105b00 0x7f0414107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:02.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 --2- 172.21.15.131:0/1850392821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04141031d0 0x7f04141055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:02.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 -- 172.21.15.131:0/1850392821 >> 172.21.15.131:0/1850392821 conn(0x7f04140fcc00 msgr2=0x7f04140ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:02.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 -- 172.21.15.131:0/1850392821 shutdown_connections 2024-04-26T20:12:02.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.084+0000 7f0413200640 1 -- 172.21.15.131:0/1850392821 wait complete. 2024-04-26T20:12:02.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 Processor -- start 2024-04-26T20:12:02.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 -- start start 2024-04-26T20:12:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04141031d0 0x7f0414198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414105b00 0x7f0414198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:02.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0414108430 0x7f041419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:02.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f041410e2f0 con 0x7f0414105b00 2024-04-26T20:12:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f041410e170 con 0x7f04141031d0 2024-04-26T20:12:02.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.085+0000 7f0413200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f041410e470 con 0x7f0414108430 2024-04-26T20:12:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04141031d0 0x7f0414198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:02.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04141031d0 0x7f0414198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:54568/0 (socket says 172.21.15.131:54568) 2024-04-26T20:12:02.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 -- 172.21.15.131:0/815983634 learned_addr learned my addr 172.21.15.131:0/815983634 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411400640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414105b00 0x7f0414198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0412800640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0414108430 0x7f041419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:02.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 -- 172.21.15.131:0/815983634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0414108430 msgr2=0x7f041419ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0414108430 0x7f041419ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 -- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414105b00 msgr2=0x7f0414198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414105b00 0x7f0414198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:02.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 -- 172.21.15.131:0/815983634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04141a0570 con 0x7f04141031d0 2024-04-26T20:12:02.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.086+0000 7f0411e00640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04141031d0 0x7f0414198950 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f040400e9f0 tx=0x7f040400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.087+0000 7f040aa00640 1 -- 172.21.15.131:0/815983634 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040400cd80 con 0x7f04141031d0 2024-04-26T20:12:02.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.087+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04141a0860 con 0x7f04141031d0 2024-04-26T20:12:02.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.087+0000 7f040aa00640 1 -- 172.21.15.131:0/815983634 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04040054d0 con 0x7f04141031d0 2024-04-26T20:12:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.087+0000 7f040aa00640 1 -- 172.21.15.131:0/815983634 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404010690 con 0x7f04141031d0 2024-04-26T20:12:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.087+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04141a0db0 con 0x7f04141031d0 2024-04-26T20:12:02.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.088+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04140fe6f0 con 0x7f04141031d0 2024-04-26T20:12:02.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.090+0000 7f040aa00640 1 -- 172.21.15.131:0/815983634 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0404004e90 con 0x7f04141031d0 2024-04-26T20:12:02.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.091+0000 7f040aa00640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03f40a3cc0 0x7f03f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:02.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.091+0000 7f040aa00640 1 -- 172.21.15.131:0/815983634 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f04040d0de0 con 0x7f04141031d0 2024-04-26T20:12:02.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.091+0000 7f0411400640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03f40a3cc0 0x7f03f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:02.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.092+0000 7f0411400640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03f40a3cc0 0x7f03f40a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0414199e60 tx=0x7f03fc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.096+0000 7f040aa00640 1 -- 172.21.15.131:0/815983634 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f040409d250 con 0x7f04141031d0 2024-04-26T20:12:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.299+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f041402dca0 con 0x7f04141031d0 2024-04-26T20:12:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.300+0000 7f040aa00640 1 -- 172.21.15.131:0/815983634 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f040409cb50 con 0x7f04141031d0 2024-04-26T20:12:03.059 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:12:03.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03f40a3cc0 msgr2=0x7f03f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:03.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03f40a3cc0 0x7f03f40a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0414199e60 tx=0x7f03fc009500 comp rx=0 tx=0).stop 2024-04-26T20:12:03.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04141031d0 msgr2=0x7f0414198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:03.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04141031d0 0x7f0414198950 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f040400e9f0 tx=0x7f040400edf0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 shutdown_connections 2024-04-26T20:12:03.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03f40a3cc0 0x7f03f40a6180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0414108430 0x7f041419ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0414105b00 0x7f0414198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 --2- 172.21.15.131:0/815983634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04141031d0 0x7f0414198950 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.302+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 >> 172.21.15.131:0/815983634 conn(0x7f04140fcc00 msgr2=0x7f0414103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.303+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 shutdown_connections 2024-04-26T20:12:03.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:02.303+0000 7f0413200640 1 -- 172.21.15.131:0/815983634 wait complete. 2024-04-26T20:12:03.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:12:03.436 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:03 smithi148 ceph-mon[32281]: pgmap v471: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:12:03.448 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:03 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/815983634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:03.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:03 smithi157 ceph-mon[32703]: pgmap v471: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:12:03.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:03 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/815983634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:03.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:03 smithi131 ceph-mon[27919]: pgmap v471: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:12:03.533 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:03 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/815983634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:03.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 -- 172.21.15.131:0/1478698194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630076970 msgr2=0x7f7630074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 --2- 172.21.15.131:0/1478698194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630076970 0x7f7630074dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7620009ca0 tx=0x7f762002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 -- 172.21.15.131:0/1478698194 shutdown_connections 2024-04-26T20:12:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 --2- 172.21.15.131:0/1478698194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763010a700 0x7f763010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 --2- 172.21.15.131:0/1478698194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7630075390 0x7f7630075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 --2- 172.21.15.131:0/1478698194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630076970 0x7f7630074dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 -- 172.21.15.131:0/1478698194 >> 172.21.15.131:0/1478698194 conn(0x7f76300fddd0 msgr2=0x7f76301001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:03.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 -- 172.21.15.131:0/1478698194 shutdown_connections 2024-04-26T20:12:03.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.651+0000 7f762fe00640 1 -- 172.21.15.131:0/1478698194 wait complete. 2024-04-26T20:12:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 Processor -- start 2024-04-26T20:12:03.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 -- start start 2024-04-26T20:12:03.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630075390 0x7f763019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7630076970 0x7f763019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:03.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763010a700 0x7f76301a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:03.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7630110530 con 0x7f763010a700 2024-04-26T20:12:03.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f76301103b0 con 0x7f7630075390 2024-04-26T20:12:04.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.652+0000 7f762fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76301106b0 con 0x7f7630076970 2024-04-26T20:12:04.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630075390 0x7f763019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:04.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630075390 0x7f763019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:54594/0 (socket says 172.21.15.131:54594) 2024-04-26T20:12:04.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 -- 172.21.15.131:0/3759235691 learned_addr learned my addr 172.21.15.131:0/3759235691 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:04.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763010a700 0x7f76301a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:04.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762e000640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7630076970 0x7f763019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:04.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 -- 172.21.15.131:0/3759235691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7630076970 msgr2=0x7f763019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:04.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7630076970 0x7f763019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 -- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763010a700 msgr2=0x7f76301a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:04.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763010a700 0x7f76301a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 -- 172.21.15.131:0/3759235691 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76301a2860 con 0x7f7630075390 2024-04-26T20:12:04.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762f400640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763010a700 0x7f76301a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:04.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.653+0000 7f762ea00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630075390 0x7f763019ac50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7620006010 tx=0x7f76200317a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.654+0000 7f7627400640 1 -- 172.21.15.131:0/3759235691 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620003c30 con 0x7f7630075390 2024-04-26T20:12:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.654+0000 7f7627400640 1 -- 172.21.15.131:0/3759235691 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7620003dd0 con 0x7f7630075390 2024-04-26T20:12:04.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.654+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76301a2af0 con 0x7f7630075390 2024-04-26T20:12:04.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.654+0000 7f7627400640 1 -- 172.21.15.131:0/3759235691 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762002fc20 con 0x7f7630075390 2024-04-26T20:12:04.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.654+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76301a3040 con 0x7f7630075390 2024-04-26T20:12:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.655+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7630109430 con 0x7f7630075390 2024-04-26T20:12:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.657+0000 7f7627400640 1 -- 172.21.15.131:0/3759235691 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f76200046a0 con 0x7f7630075390 2024-04-26T20:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.658+0000 7f7627400640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f76140a3c40 0x7f76140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:04.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.659+0000 7f7627400640 1 -- 172.21.15.131:0/3759235691 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f76200f1e30 con 0x7f7630075390 2024-04-26T20:12:04.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.659+0000 7f762e000640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f76140a3c40 0x7f76140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:04.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.659+0000 7f762e000640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f76140a3c40 0x7f76140a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7618009bd0 tx=0x7f7618005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.663+0000 7f7627400640 1 -- 172.21.15.131:0/3759235691 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f76200be370 con 0x7f7630075390 2024-04-26T20:12:04.668 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:04 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3759235691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:04.680 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:04 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3759235691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:04.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.869+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f763002dd20 con 0x7f7630075390 2024-04-26T20:12:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.870+0000 7f7627400640 1 -- 172.21.15.131:0/3759235691 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f76200bdc70 con 0x7f7630075390 2024-04-26T20:12:04.793 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:12:04.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f76140a3c40 msgr2=0x7f76140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:04.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f76140a3c40 0x7f76140a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7618009bd0 tx=0x7f7618005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:04.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630075390 msgr2=0x7f763019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:04.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630075390 0x7f763019ac50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7620006010 tx=0x7f76200317a0 comp rx=0 tx=0).stop 2024-04-26T20:12:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 shutdown_connections 2024-04-26T20:12:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763010a700 0x7f76301a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:04.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f76140a3c40 0x7f76140a6100 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:04.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7630076970 0x7f763019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 --2- 172.21.15.131:0/3759235691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7630075390 0x7f763019ac50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:05.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 >> 172.21.15.131:0/3759235691 conn(0x7f76300fddd0 msgr2=0x7f7630106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:05.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.873+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 shutdown_connections 2024-04-26T20:12:05.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:03.874+0000 7f762fe00640 1 -- 172.21.15.131:0/3759235691 wait complete. 2024-04-26T20:12:05.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:12:05.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:04 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3759235691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:05.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 -- 172.21.15.131:0/937524197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54108630 msgr2=0x7f1f5410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 --2- 172.21.15.131:0/937524197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54108630 0x7f1f5410aa80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009be0 tx=0x7f1f4c02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:12:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 -- 172.21.15.131:0/937524197 shutdown_connections 2024-04-26T20:12:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 --2- 172.21.15.131:0/937524197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54108630 0x7f1f5410aa80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:05.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 --2- 172.21.15.131:0/937524197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54105d00 0x7f1f541080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:05.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 --2- 172.21.15.131:0/937524197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f541033d0 0x7f1f541057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:05.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 -- 172.21.15.131:0/937524197 >> 172.21.15.131:0/937524197 conn(0x7f1f540fcdc0 msgr2=0x7f1f540ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 -- 172.21.15.131:0/937524197 shutdown_connections 2024-04-26T20:12:05.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.299+0000 7f1f5a800640 1 -- 172.21.15.131:0/937524197 wait complete. 2024-04-26T20:12:05.581 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:05 smithi157 ceph-mon[32703]: pgmap v472: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:05.638 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:05 smithi148 ceph-mon[32281]: pgmap v472: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:05.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.300+0000 7f1f5a800640 1 Processor -- start 2024-04-26T20:12:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.300+0000 7f1f5a800640 1 -- start start 2024-04-26T20:12:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.300+0000 7f1f5a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541033d0 0x7f1f54198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:05.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f5a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f54105d00 0x7f1f54199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f5a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54108630 0x7f1f541a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:05.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f5a800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410e480 con 0x7f1f541033d0 2024-04-26T20:12:05.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f5a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410e300 con 0x7f1f54108630 2024-04-26T20:12:05.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f5a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410e600 con 0x7f1f54105d00 2024-04-26T20:12:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541033d0 0x7f1f54198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:05.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541033d0 0x7f1f54198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48652/0 (socket says 172.21.15.131:48652) 2024-04-26T20:12:05.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 -- 172.21.15.131:0/2113147496 learned_addr learned my addr 172.21.15.131:0/2113147496 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:05.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53e00640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54108630 0x7f1f541a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:05.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f4be00640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f54105d00 0x7f1f54199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 -- 172.21.15.131:0/2113147496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f54105d00 msgr2=0x7f1f54199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:05.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f54105d00 0x7f1f54199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:06.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 -- 172.21.15.131:0/2113147496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54108630 msgr2=0x7f1f541a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:06.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54108630 0x7f1f541a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53400640 1 -- 172.21.15.131:0/2113147496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f541a0720 con 0x7f1f541033d0 2024-04-26T20:12:06.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.301+0000 7f1f53e00640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54108630 0x7f1f541a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:06.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.302+0000 7f1f53400640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541033d0 0x7f1f54198b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400e9c0 tx=0x7f1f4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.302+0000 7f1f50c00640 1 -- 172.21.15.131:0/2113147496 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4400cde0 con 0x7f1f541033d0 2024-04-26T20:12:06.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.302+0000 7f1f50c00640 1 -- 172.21.15.131:0/2113147496 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f440035f0 con 0x7f1f541033d0 2024-04-26T20:12:06.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.302+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f541a0a10 con 0x7f1f541033d0 2024-04-26T20:12:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.302+0000 7f1f50c00640 1 -- 172.21.15.131:0/2113147496 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44010690 con 0x7f1f541033d0 2024-04-26T20:12:06.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.302+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a0f60 con 0x7f1f541033d0 2024-04-26T20:12:06.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.303+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f540697c0 con 0x7f1f541033d0 2024-04-26T20:12:06.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.305+0000 7f1f50c00640 1 -- 172.21.15.131:0/2113147496 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1f44004000 con 0x7f1f541033d0 2024-04-26T20:12:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.307+0000 7f1f50c00640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1f380a3c40 0x7f1f380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.307+0000 7f1f50c00640 1 -- 172.21.15.131:0/2113147496 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f1f440d0cd0 con 0x7f1f541033d0 2024-04-26T20:12:06.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.307+0000 7f1f4be00640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1f380a3c40 0x7f1f380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:06.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.308+0000 7f1f4be00640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1f380a3c40 0x7f1f380a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f5419a010 tx=0x7f1f3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:06.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.312+0000 7f1f50c00640 1 -- 172.21.15.131:0/2113147496 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1f44009510 con 0x7f1f541033d0 2024-04-26T20:12:06.377 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:05 smithi131 ceph-mon[27919]: pgmap v472: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.554+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f540fe8b0 con 0x7f1f541033d0 2024-04-26T20:12:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.555+0000 7f1f50c00640 1 -- 172.21.15.131:0/2113147496 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11895 (secure 0 0 0) 0x7f1f4409d100 con 0x7f1f541033d0 2024-04-26T20:12:06.428 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:12:06.435 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:08:57.196498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24557,24571]},"id":1}]} 2024-04-26T20:12:06.445 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-04-26T20:12:06.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1f380a3c40 msgr2=0x7f1f380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:06.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1f380a3c40 0x7f1f380a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f5419a010 tx=0x7f1f3c005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541033d0 msgr2=0x7f1f54198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:06.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541033d0 0x7f1f54198b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400e9c0 tx=0x7f1f4400ee80 comp rx=0 tx=0).stop 2024-04-26T20:12:06.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 shutdown_connections 2024-04-26T20:12:06.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54108630 0x7f1f541a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1f380a3c40 0x7f1f380a6100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f54105d00 0x7f1f54199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 --2- 172.21.15.131:0/2113147496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541033d0 0x7f1f54198b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:06.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 >> 172.21.15.131:0/2113147496 conn(0x7f1f540fcdc0 msgr2=0x7f1f54103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 shutdown_connections 2024-04-26T20:12:06.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:05.558+0000 7f1f5a800640 1 -- 172.21.15.131:0/2113147496 wait complete. 2024-04-26T20:12:06.747 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:12:06.754 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:06 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2113147496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:12:06.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:06 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2113147496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:12:06.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:06 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2113147496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:12:06.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 -- 172.21.15.131:0/3281870744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 msgr2=0x7f8768104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:06.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 --2- 172.21.15.131:0/3281870744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f8768104fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f875c009900 tx=0x7f875c02a410 comp rx=0 tx=0).stop 2024-04-26T20:12:06.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 -- 172.21.15.131:0/3281870744 shutdown_connections 2024-04-26T20:12:07.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 --2- 172.21.15.131:0/3281870744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8768106740 0x7f876810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 --2- 172.21.15.131:0/3281870744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8768105d80 0x7f8768106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 --2- 172.21.15.131:0/3281870744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f8768104fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 -- 172.21.15.131:0/3281870744 >> 172.21.15.131:0/3281870744 conn(0x7f8768100120 msgr2=0x7f8768102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:07.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 -- 172.21.15.131:0/3281870744 shutdown_connections 2024-04-26T20:12:07.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.978+0000 7f8770600640 1 -- 172.21.15.131:0/3281870744 wait complete. 2024-04-26T20:12:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 Processor -- start 2024-04-26T20:12:07.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 -- start start 2024-04-26T20:12:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f876819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8768105d80 0x7f876819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:07.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8768106740 0x7f87681a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:07.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87681106c0 con 0x7f8768106740 2024-04-26T20:12:07.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8768110540 con 0x7f8768104b90 2024-04-26T20:12:07.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.980+0000 7f8770600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8768110840 con 0x7f8768105d80 2024-04-26T20:12:07.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f876819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:07.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8768106740 0x7f87681a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:07.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f876819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:40244/0 (socket says 172.21.15.131:40244) 2024-04-26T20:12:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 -- 172.21.15.131:0/1263780337 learned_addr learned my addr 172.21.15.131:0/1263780337 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:07.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876d200640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8768105d80 0x7f876819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:07.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 -- 172.21.15.131:0/1263780337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8768105d80 msgr2=0x7f876819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8768105d80 0x7f876819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 -- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8768106740 msgr2=0x7f87681a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:07.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8768106740 0x7f87681a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 -- 172.21.15.131:0/1263780337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87681a29d0 con 0x7f8768104b90 2024-04-26T20:12:07.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.981+0000 7f876dc00640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f876819adc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f875c006010 tx=0x7f875c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:07.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.982+0000 7f8766000640 1 -- 172.21.15.131:0/1263780337 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c03f030 con 0x7f8768104b90 2024-04-26T20:12:07.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.982+0000 7f8766000640 1 -- 172.21.15.131:0/1263780337 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875c031910 con 0x7f8768104b90 2024-04-26T20:12:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.982+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87681a2c60 con 0x7f8768104b90 2024-04-26T20:12:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.982+0000 7f8766000640 1 -- 172.21.15.131:0/1263780337 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c035070 con 0x7f8768104b90 2024-04-26T20:12:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.982+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87681a3150 con 0x7f8768104b90 2024-04-26T20:12:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.983+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876810bed0 con 0x7f8768104b90 2024-04-26T20:12:07.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.985+0000 7f8766000640 1 -- 172.21.15.131:0/1263780337 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f875c042030 con 0x7f8768104b90 2024-04-26T20:12:07.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.986+0000 7f8766000640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87500a3c40 0x7f87500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:07.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.987+0000 7f8766000640 1 -- 172.21.15.131:0/1263780337 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f875c0f1350 con 0x7f8768104b90 2024-04-26T20:12:07.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.987+0000 7f876d200640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87500a3c40 0x7f87500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:07.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.987+0000 7f876d200640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87500a3c40 0x7f87500a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8754009bd0 tx=0x7f8754005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:07.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:06.989+0000 7f8766000640 1 -- 172.21.15.131:0/1263780337 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f875c0bd130 con 0x7f8768104b90 2024-04-26T20:12:07.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.192+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f876802dd20 con 0x7f8768104b90 2024-04-26T20:12:07.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.195+0000 7f8766000640 1 -- 172.21.15.131:0/1263780337 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7f875c0c1430 con 0x7f8768104b90 2024-04-26T20:12:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:12:07.831 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:12:07.848 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:07 smithi157 ceph-mon[32703]: pgmap v473: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:12:07.857 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:07 smithi148 ceph-mon[32281]: pgmap v473: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:12:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87500a3c40 msgr2=0x7f87500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:07.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87500a3c40 0x7f87500a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8754009bd0 tx=0x7f8754005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 msgr2=0x7f876819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f876819adc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f875c006010 tx=0x7f875c0042f0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 shutdown_connections 2024-04-26T20:12:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8768106740 0x7f87681a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f87500a3c40 0x7f87500a6100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8768105d80 0x7f876819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:07.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 --2- 172.21.15.131:0/1263780337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8768104b90 0x7f876819adc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:08.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.197+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 >> 172.21.15.131:0/1263780337 conn(0x7f8768100120 msgr2=0x7f87681090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.198+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 shutdown_connections 2024-04-26T20:12:08.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:07.198+0000 7f8770600640 1 -- 172.21.15.131:0/1263780337 wait complete. 2024-04-26T20:12:08.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:07 smithi131 ceph-mon[27919]: pgmap v473: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:12:08.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:12:08.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:12:08.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:08 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1263780337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 -- 172.21.15.131:0/4066496432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078530 msgr2=0x7f7f80078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:08.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 --2- 172.21.15.131:0/4066496432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078530 0x7f7f80078990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009900 tx=0x7f7f6c02a400 comp rx=0 tx=0).stop 2024-04-26T20:12:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 -- 172.21.15.131:0/4066496432 shutdown_connections 2024-04-26T20:12:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 --2- 172.21.15.131:0/4066496432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f80078ed0 0x7f7f80079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 --2- 172.21.15.131:0/4066496432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078530 0x7f7f80078990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 --2- 172.21.15.131:0/4066496432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f800772f0 0x7f7f80077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:08.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 -- 172.21.15.131:0/4066496432 >> 172.21.15.131:0/4066496432 conn(0x7f7f800fffe0 msgr2=0x7f7f80102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:08.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 -- 172.21.15.131:0/4066496432 shutdown_connections 2024-04-26T20:12:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.432+0000 7f7f7f200640 1 -- 172.21.15.131:0/4066496432 wait complete. 2024-04-26T20:12:08.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:08 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1263780337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:08.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 Processor -- start 2024-04-26T20:12:08.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 -- start start 2024-04-26T20:12:08.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f800772f0 0x7f7f8019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:08.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f80078530 0x7f7f8019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:08.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078ed0 0x7f7f801a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:08.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801149f0 con 0x7f7f800772f0 2024-04-26T20:12:08.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80114870 con 0x7f7f80078ed0 2024-04-26T20:12:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80114b70 con 0x7f7f80078530 2024-04-26T20:12:08.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.434+0000 7f7f7de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f800772f0 0x7f7f8019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:08.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078ed0 0x7f7f801a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f800772f0 0x7f7f8019f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48692/0 (socket says 172.21.15.131:48692) 2024-04-26T20:12:08.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f80078530 0x7f7f8019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:08.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7de00640 1 -- 172.21.15.131:0/2994997706 learned_addr learned my addr 172.21.15.131:0/2994997706 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:08.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7e800640 1 -- 172.21.15.131:0/2994997706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f80078530 msgr2=0x7f7f8019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:08.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7e800640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f80078530 0x7f7f8019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:08.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7e800640 1 -- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f800772f0 msgr2=0x7f7f8019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:08.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7e800640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f800772f0 0x7f7f8019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:08.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7e800640 1 -- 172.21.15.131:0/2994997706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f801a6c70 con 0x7f7f80078ed0 2024-04-26T20:12:08.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.435+0000 7f7f7e800640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078ed0 0x7f7f801a6600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800da10 tx=0x7f7f7800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:09.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.436+0000 7f7f76a00640 1 -- 172.21.15.131:0/2994997706 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7800bd70 con 0x7f7f80078ed0 2024-04-26T20:12:09.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.436+0000 7f7f76a00640 1 -- 172.21.15.131:0/2994997706 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f78003e20 con 0x7f7f80078ed0 2024-04-26T20:12:09.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.436+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f801a6f60 con 0x7f7f80078ed0 2024-04-26T20:12:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.436+0000 7f7f76a00640 1 -- 172.21.15.131:0/2994997706 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780106c0 con 0x7f7f80078ed0 2024-04-26T20:12:09.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.436+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a7420 con 0x7f7f80078ed0 2024-04-26T20:12:09.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.437+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f801113e0 con 0x7f7f80078ed0 2024-04-26T20:12:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.439+0000 7f7f76a00640 1 -- 172.21.15.131:0/2994997706 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7f78004270 con 0x7f7f80078ed0 2024-04-26T20:12:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.440+0000 7f7f76a00640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7f640a3cc0 0x7f7f640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.440+0000 7f7f76a00640 1 -- 172.21.15.131:0/2994997706 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7f780d0ff0 con 0x7f7f80078ed0 2024-04-26T20:12:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.441+0000 7f7f7de00640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7f640a3cc0 0x7f7f640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:09.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.441+0000 7f7f7de00640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7f640a3cc0 0x7f7f640a6180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009c90 tx=0x7f7f70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.443+0000 7f7f76a00640 1 -- 172.21.15.131:0/2994997706 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7f7809d4e0 con 0x7f7f80078ed0 2024-04-26T20:12:09.237 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:08 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1263780337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:09.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.645+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f8002dad0 con 0x7f7f80078ed0 2024-04-26T20:12:09.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.646+0000 7f7f76a00640 1 -- 172.21.15.131:0/2994997706 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f7f7809cde0 con 0x7f7f80078ed0 2024-04-26T20:12:09.261 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:12:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7f640a3cc0 msgr2=0x7f7f640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:09.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7f640a3cc0 0x7f7f640a6180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009c90 tx=0x7f7f70005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:09.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078ed0 msgr2=0x7f7f801a6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:09.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078ed0 0x7f7f801a6600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800da10 tx=0x7f7f7800ded0 comp rx=0 tx=0).stop 2024-04-26T20:12:09.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 shutdown_connections 2024-04-26T20:12:09.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f80078ed0 0x7f7f801a6600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:09.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7f640a3cc0 0x7f7f640a6180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:09.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f80078530 0x7f7f8019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 --2- 172.21.15.131:0/2994997706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f800772f0 0x7f7f8019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:09.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.648+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 >> 172.21.15.131:0/2994997706 conn(0x7f7f800fffe0 msgr2=0x7f7f80103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:09.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.649+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 shutdown_connections 2024-04-26T20:12:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:08.649+0000 7f7f7f200640 1 -- 172.21.15.131:0/2994997706 wait complete. 2024-04-26T20:12:09.577 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:12:09.587 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:09 smithi157 ceph-mon[32703]: pgmap v474: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T20:12:09.594 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:09 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2994997706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:09.602 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:09 smithi148 ceph-mon[32281]: pgmap v474: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T20:12:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:09 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2994997706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:09.706 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:09 smithi131 ceph-mon[27919]: pgmap v474: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T20:12:09.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:09 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2994997706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 -- 172.21.15.131:0/975444901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 msgr2=0x7f245c1112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 --2- 172.21.15.131:0/975444901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 0x7f245c1112a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2454009be0 tx=0x7f245402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:12:09.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 -- 172.21.15.131:0/975444901 shutdown_connections 2024-04-26T20:12:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 --2- 172.21.15.131:0/975444901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 0x7f245c1112a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 --2- 172.21.15.131:0/975444901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f245c109fc0 0x7f245c10a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 --2- 172.21.15.131:0/975444901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 0x7f245c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 -- 172.21.15.131:0/975444901 >> 172.21.15.131:0/975444901 conn(0x7f245c077e00 msgr2=0x7f245c07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 -- 172.21.15.131:0/975444901 shutdown_connections 2024-04-26T20:12:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.963+0000 7f2461c00640 1 -- 172.21.15.131:0/975444901 wait complete. 2024-04-26T20:12:10.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.964+0000 7f2461c00640 1 Processor -- start 2024-04-26T20:12:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.964+0000 7f2461c00640 1 -- start start 2024-04-26T20:12:10.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 0x7f245c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:10.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f245c109fc0 0x7f245c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:10.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 0x7f245c1a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:10.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f245c1149a0 con 0x7f245c10a980 2024-04-26T20:12:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f245c114820 con 0x7f245c108e70 2024-04-26T20:12:10.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f245c114b20 con 0x7f245c109fc0 2024-04-26T20:12:10.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 0x7f245c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:10.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 0x7f245c19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:40292/0 (socket says 172.21.15.131:40292) 2024-04-26T20:12:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 -- 172.21.15.131:0/1939520022 learned_addr learned my addr 172.21.15.131:0/1939520022 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:10.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461200640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 0x7f245c1a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:10.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245b400640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f245c109fc0 0x7f245c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 -- 172.21.15.131:0/1939520022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f245c109fc0 msgr2=0x7f245c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f245c109fc0 0x7f245c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 -- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 msgr2=0x7f245c1a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:10.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 0x7f245c1a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f245be00640 1 -- 172.21.15.131:0/1939520022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c1a6c60 con 0x7f245c108e70 2024-04-26T20:12:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.965+0000 7f2461200640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 0x7f245c1a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:12:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.966+0000 7f245be00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 0x7f245c19f040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e9c0 tx=0x7f244c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:10.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.966+0000 7f2458c00640 1 -- 172.21.15.131:0/1939520022 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c00cde0 con 0x7f245c108e70 2024-04-26T20:12:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.966+0000 7f2458c00640 1 -- 172.21.15.131:0/1939520022 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f244c0035f0 con 0x7f245c108e70 2024-04-26T20:12:10.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.966+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c1a6f50 con 0x7f245c108e70 2024-04-26T20:12:10.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.966+0000 7f2458c00640 1 -- 172.21.15.131:0/1939520022 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c010690 con 0x7f245c108e70 2024-04-26T20:12:10.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.966+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c1a7410 con 0x7f245c108e70 2024-04-26T20:12:10.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.967+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c110120 con 0x7f245c108e70 2024-04-26T20:12:10.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.969+0000 7f2458c00640 1 -- 172.21.15.131:0/1939520022 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f244c004000 con 0x7f245c108e70 2024-04-26T20:12:10.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.972+0000 7f2458c00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24400a3cc0 0x7f24400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:10.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.972+0000 7f2458c00640 1 -- 172.21.15.131:0/1939520022 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f244c0d0d20 con 0x7f245c108e70 2024-04-26T20:12:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.972+0000 7f245b400640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24400a3cc0 0x7f24400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.973+0000 7f245b400640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24400a3cc0 0x7f24400a6180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f245c1a0550 tx=0x7f2448005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:10.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:09.975+0000 7f2458c00640 1 -- 172.21.15.131:0/1939520022 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f244c009510 con 0x7f245c108e70 2024-04-26T20:12:10.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.181+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f245c02dd60 con 0x7f245c108e70 2024-04-26T20:12:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.182+0000 7f2458c00640 1 -- 172.21.15.131:0/1939520022 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f244c09d150 con 0x7f245c108e70 2024-04-26T20:12:10.798 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:12:10.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.184+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24400a3cc0 msgr2=0x7f24400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:10.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.184+0000 7f2461c00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24400a3cc0 0x7f24400a6180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f245c1a0550 tx=0x7f2448005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:10.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.184+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 msgr2=0x7f245c19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:10.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.184+0000 7f2461c00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 0x7f245c19f040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e9c0 tx=0x7f244c00ee80 comp rx=0 tx=0).stop 2024-04-26T20:12:10.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 shutdown_connections 2024-04-26T20:12:10.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c10a980 0x7f245c1a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24400a3cc0 0x7f24400a6180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f245c109fc0 0x7f245c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 --2- 172.21.15.131:0/1939520022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245c108e70 0x7f245c19f040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:10.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 >> 172.21.15.131:0/1939520022 conn(0x7f245c077e00 msgr2=0x7f245c10d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 shutdown_connections 2024-04-26T20:12:11.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:10.185+0000 7f2461c00640 1 -- 172.21.15.131:0/1939520022 wait complete. 2024-04-26T20:12:11.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:12:11.183 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:11 smithi131 ceph-mon[27919]: pgmap v475: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:11.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:11 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1939520022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:11.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.323+0000 7fc461600640 1 -- 172.21.15.131:0/2897052704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c10a940 msgr2=0x7fc45c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:11.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.323+0000 7fc461600640 1 --2- 172.21.15.131:0/2897052704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c10a940 0x7fc45c1111d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc454009860 tx=0x7fc45402a340 comp rx=0 tx=0).stop 2024-04-26T20:12:11.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.324+0000 7fc461600640 1 -- 172.21.15.131:0/2897052704 shutdown_connections 2024-04-26T20:12:11.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.324+0000 7fc461600640 1 --2- 172.21.15.131:0/2897052704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c10a940 0x7fc45c1111d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:11.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.324+0000 7fc461600640 1 --2- 172.21.15.131:0/2897052704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c109f80 0x7fc45c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:11.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.324+0000 7fc461600640 1 --2- 172.21.15.131:0/2897052704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 0x7fc45c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.324+0000 7fc461600640 1 -- 172.21.15.131:0/2897052704 >> 172.21.15.131:0/2897052704 conn(0x7fc45c077d30 msgr2=0x7fc45c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:11.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.324+0000 7fc461600640 1 -- 172.21.15.131:0/2897052704 shutdown_connections 2024-04-26T20:12:11.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.324+0000 7fc461600640 1 -- 172.21.15.131:0/2897052704 wait complete. 2024-04-26T20:12:11.532 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:11 smithi157 ceph-mon[32703]: pgmap v475: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:11 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1939520022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:11.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 Processor -- start 2024-04-26T20:12:11.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 -- start start 2024-04-26T20:12:11.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 0x7fc45c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:11.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c109f80 0x7fc45c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:11.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c10a940 0x7fc45c1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:11.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1148a0 con 0x7fc45c10a940 2024-04-26T20:12:11.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c114720 con 0x7fc45c108d90 2024-04-26T20:12:11.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.325+0000 7fc461600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c114a20 con 0x7fc45c109f80 2024-04-26T20:12:11.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc460c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c10a940 0x7fc45c1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:11.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 0x7fc45c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:11.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 0x7fc45c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:40314/0 (socket says 172.21.15.131:40314) 2024-04-26T20:12:11.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 -- 172.21.15.131:0/4052506076 learned_addr learned my addr 172.21.15.131:0/4052506076 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:11.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc453400640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c109f80 0x7fc45c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:11.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 -- 172.21.15.131:0/4052506076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c109f80 msgr2=0x7fc45c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:11.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c109f80 0x7fc45c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:11.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 -- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c10a940 msgr2=0x7fc45c1a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:11.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c10a940 0x7fc45c1a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:11.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 -- 172.21.15.131:0/4052506076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c1a6b60 con 0x7fc45c108d90 2024-04-26T20:12:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc460c00640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c10a940 0x7fc45c1a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:12.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc453400640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c109f80 0x7fc45c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.326+0000 7fc45be00640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 0x7fc45c19ef40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00ebc0 tx=0x7fc44c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.327+0000 7fc459600640 1 -- 172.21.15.131:0/4052506076 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c003970 con 0x7fc45c108d90 2024-04-26T20:12:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.327+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c1a6e50 con 0x7fc45c108d90 2024-04-26T20:12:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.327+0000 7fc459600640 1 -- 172.21.15.131:0/4052506076 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc44c004030 con 0x7fc45c108d90 2024-04-26T20:12:12.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.327+0000 7fc459600640 1 -- 172.21.15.131:0/4052506076 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0196e0 con 0x7fc45c108d90 2024-04-26T20:12:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.327+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c1a72b0 con 0x7fc45c108d90 2024-04-26T20:12:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.328+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45c110100 con 0x7fc45c108d90 2024-04-26T20:12:12.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.330+0000 7fc459600640 1 -- 172.21.15.131:0/4052506076 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc44c003b10 con 0x7fc45c108d90 2024-04-26T20:12:12.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.331+0000 7fc459600640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4400a3c40 0x7fc4400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.331+0000 7fc459600640 1 -- 172.21.15.131:0/4052506076 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc44c0d12a0 con 0x7fc45c108d90 2024-04-26T20:12:12.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.332+0000 7fc453400640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4400a3c40 0x7fc4400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:12.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.332+0000 7fc453400640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4400a3c40 0x7fc4400a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc45c1a0450 tx=0x7fc444009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:12.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.336+0000 7fc459600640 1 -- 172.21.15.131:0/4052506076 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc44c09d6d0 con 0x7fc45c108d90 2024-04-26T20:12:12.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.539+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc45c02dd80 con 0x7fc45c108d90 2024-04-26T20:12:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.540+0000 7fc459600640 1 -- 172.21.15.131:0/4052506076 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fc44c09cfd0 con 0x7fc45c108d90 2024-04-26T20:12:12.381 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:12:12.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4400a3c40 msgr2=0x7fc4400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:12.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4400a3c40 0x7fc4400a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc45c1a0450 tx=0x7fc444009260 comp rx=0 tx=0).stop 2024-04-26T20:12:12.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 msgr2=0x7fc45c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:12.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 0x7fc45c19ef40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00ebc0 tx=0x7fc44c00c2d0 comp rx=0 tx=0).stop 2024-04-26T20:12:12.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 shutdown_connections 2024-04-26T20:12:12.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c10a940 0x7fc45c1a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:12.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4400a3c40 0x7fc4400a6100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c109f80 0x7fc45c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 --2- 172.21.15.131:0/4052506076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc45c108d90 0x7fc45c19ef40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:12.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 >> 172.21.15.131:0/4052506076 conn(0x7fc45c077d30 msgr2=0x7fc45c10d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:12.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 shutdown_connections 2024-04-26T20:12:12.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:11.543+0000 7fc461600640 1 -- 172.21.15.131:0/4052506076 wait complete. 2024-04-26T20:12:12.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:12:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:12 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4052506076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:12.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:12 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4052506076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:12.730 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:11 smithi148 ceph-mon[32281]: pgmap v475: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:12.758 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:11 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1939520022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:12.770 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:12 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4052506076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:12.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 -- 172.21.15.131:0/156819501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 msgr2=0x7fdab8104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:12.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 --2- 172.21.15.131:0/156819501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab8104fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009900 tx=0x7fdaac02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:12:13.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 -- 172.21.15.131:0/156819501 shutdown_connections 2024-04-26T20:12:13.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 --2- 172.21.15.131:0/156819501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8106740 0x7fdab810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 --2- 172.21.15.131:0/156819501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8105d80 0x7fdab8106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 --2- 172.21.15.131:0/156819501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab8104fb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 -- 172.21.15.131:0/156819501 >> 172.21.15.131:0/156819501 conn(0x7fdab8100120 msgr2=0x7fdab8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 -- 172.21.15.131:0/156819501 shutdown_connections 2024-04-26T20:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.932+0000 7fdac0600640 1 -- 172.21.15.131:0/156819501 wait complete. 2024-04-26T20:12:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.933+0000 7fdac0600640 1 Processor -- start 2024-04-26T20:12:13.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.933+0000 7fdac0600640 1 -- start start 2024-04-26T20:12:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdac0600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:13.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdac0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8105d80 0x7fdab819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:13.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdac0600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8106740 0x7fdab81a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:13.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdac0600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81106e0 con 0x7fdab8106740 2024-04-26T20:12:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdac0600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8110560 con 0x7fdab8104b90 2024-04-26T20:12:13.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdac0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8110860 con 0x7fdab8105d80 2024-04-26T20:12:13.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:13.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:40322/0 (socket says 172.21.15.131:40322) 2024-04-26T20:12:13.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 -- 172.21.15.131:0/50500478 learned_addr learned my addr 172.21.15.131:0/50500478 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:13.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabe600640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8106740 0x7fdab81a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:13.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabd200640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8105d80 0x7fdab819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:13.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 -- 172.21.15.131:0/50500478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8105d80 msgr2=0x7fdab819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8105d80 0x7fdab819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:13.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 -- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8106740 msgr2=0x7fdab81a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:13.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8106740 0x7fdab81a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:13.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.934+0000 7fdabdc00640 1 -- 172.21.15.131:0/50500478 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab81a2a20 con 0x7fdab8104b90 2024-04-26T20:12:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.935+0000 7fdabdc00640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab819ae00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdaac031500 tx=0x7fdaac003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:13.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.935+0000 7fdab6000640 1 -- 172.21.15.131:0/50500478 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac03f070 con 0x7fdab8104b90 2024-04-26T20:12:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.935+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab81a2cb0 con 0x7fdab8104b90 2024-04-26T20:12:13.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.935+0000 7fdab6000640 1 -- 172.21.15.131:0/50500478 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaac031c50 con 0x7fdab8104b90 2024-04-26T20:12:13.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.935+0000 7fdab6000640 1 -- 172.21.15.131:0/50500478 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac035030 con 0x7fdab8104b90 2024-04-26T20:12:13.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.935+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab81a31a0 con 0x7fdab8104b90 2024-04-26T20:12:13.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.936+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab810bf80 con 0x7fdab8104b90 2024-04-26T20:12:13.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.938+0000 7fdab6000640 1 -- 172.21.15.131:0/50500478 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdaac042070 con 0x7fdab8104b90 2024-04-26T20:12:13.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.940+0000 7fdab6000640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdaa00a3c40 0x7fdaa00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:13.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.940+0000 7fdab6000640 1 -- 172.21.15.131:0/50500478 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fdaac0f17c0 con 0x7fdab8104b90 2024-04-26T20:12:13.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.940+0000 7fdabd200640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdaa00a3c40 0x7fdaa00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:13.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.941+0000 7fdabd200640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdaa00a3c40 0x7fdaa00a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdab819c310 tx=0x7fdaa4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:14.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:12.942+0000 7fdab6000640 1 -- 172.21.15.131:0/50500478 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdaac0bd5a0 con 0x7fdab8104b90 2024-04-26T20:12:14.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.146+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdab802dc80 con 0x7fdab8104b90 2024-04-26T20:12:14.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.148+0000 7fdab6000640 1 -- 172.21.15.131:0/50500478 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7fdaac0c18a0 con 0x7fdab8104b90 2024-04-26T20:12:14.034 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:12:14.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:13 smithi148 ceph-mon[32281]: pgmap v476: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:14.138 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:13 smithi157 ceph-mon[32703]: pgmap v476: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:14.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdaa00a3c40 msgr2=0x7fdaa00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:14.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdaa00a3c40 0x7fdaa00a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdab819c310 tx=0x7fdaa4005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:14.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 msgr2=0x7fdab819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab819ae00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdaac031500 tx=0x7fdaac003c10 comp rx=0 tx=0).stop 2024-04-26T20:12:14.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 shutdown_connections 2024-04-26T20:12:14.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8106740 0x7fdab81a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdaa00a3c40 0x7fdaa00a6100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8105d80 0x7fdab819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 --2- 172.21.15.131:0/50500478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8104b90 0x7fdab819ae00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 >> 172.21.15.131:0/50500478 conn(0x7fdab8100120 msgr2=0x7fdab8109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:14.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 shutdown_connections 2024-04-26T20:12:14.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:13.150+0000 7fdac0600640 1 -- 172.21.15.131:0/50500478 wait complete. 2024-04-26T20:12:14.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:12:14.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:13 smithi131 ceph-mon[27919]: pgmap v476: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:14.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:14 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/50500478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:14.502 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:14 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/50500478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:14.575 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:14 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/50500478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:14.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 -- 172.21.15.131:0/2632079735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c075330 msgr2=0x7efe5c075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:14.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 --2- 172.21.15.131:0/2632079735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c075330 0x7efe5c075790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe44009860 tx=0x7efe4402a3d0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 -- 172.21.15.131:0/2632079735 shutdown_connections 2024-04-26T20:12:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 --2- 172.21.15.131:0/2632079735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c10a5e0 0x7efe5c10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 --2- 172.21.15.131:0/2632079735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c075330 0x7efe5c075790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 --2- 172.21.15.131:0/2632079735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c076910 0x7efe5c074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:14.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 -- 172.21.15.131:0/2632079735 >> 172.21.15.131:0/2632079735 conn(0x7efe5c0fdcc0 msgr2=0x7efe5c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:14.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 -- 172.21.15.131:0/2632079735 shutdown_connections 2024-04-26T20:12:14.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.638+0000 7efe5b200640 1 -- 172.21.15.131:0/2632079735 wait complete. 2024-04-26T20:12:14.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.639+0000 7efe5b200640 1 Processor -- start 2024-04-26T20:12:14.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.639+0000 7efe5b200640 1 -- start start 2024-04-26T20:12:14.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe5b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c075330 0x7efe5c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:14.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe5b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c076910 0x7efe5c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:14.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe5b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c10a5e0 0x7efe5c1a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:14.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe5b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1104b0 con 0x7efe5c075330 2024-04-26T20:12:14.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe5b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c110330 con 0x7efe5c076910 2024-04-26T20:12:15.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe5b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c110630 con 0x7efe5c10a5e0 2024-04-26T20:12:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c075330 0x7efe5c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:15.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe5a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c10a5e0 0x7efe5c1a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:15.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c075330 0x7efe5c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59484/0 (socket says 172.21.15.131:59484) 2024-04-26T20:12:15.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 -- 172.21.15.131:0/4047817950 learned_addr learned my addr 172.21.15.131:0/4047817950 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:15.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe51400640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c076910 0x7efe5c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:15.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 -- 172.21.15.131:0/4047817950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c10a5e0 msgr2=0x7efe5c1a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:15.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c10a5e0 0x7efe5c1a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:15.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 -- 172.21.15.131:0/4047817950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c076910 msgr2=0x7efe5c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:15.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c076910 0x7efe5c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:15.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.640+0000 7efe59e00640 1 -- 172.21.15.131:0/4047817950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c1a2790 con 0x7efe5c075330 2024-04-26T20:12:15.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.641+0000 7efe59e00640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c075330 0x7efe5c19ab70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e9f0 tx=0x7efe4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.641+0000 7efe53400640 1 -- 172.21.15.131:0/4047817950 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00cd80 con 0x7efe5c075330 2024-04-26T20:12:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.641+0000 7efe53400640 1 -- 172.21.15.131:0/4047817950 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe4c0108e0 con 0x7efe5c075330 2024-04-26T20:12:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.641+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1a2a80 con 0x7efe5c075330 2024-04-26T20:12:15.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.641+0000 7efe53400640 1 -- 172.21.15.131:0/4047817950 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c018980 con 0x7efe5c075330 2024-04-26T20:12:15.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.641+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a2fa0 con 0x7efe5c075330 2024-04-26T20:12:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.642+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c1093a0 con 0x7efe5c075330 2024-04-26T20:12:15.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.645+0000 7efe53400640 1 -- 172.21.15.131:0/4047817950 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efe4c010430 con 0x7efe5c075330 2024-04-26T20:12:15.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.647+0000 7efe53400640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe400a3c40 0x7efe400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:15.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.647+0000 7efe53400640 1 -- 172.21.15.131:0/4047817950 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7efe4c0d0c90 con 0x7efe5c075330 2024-04-26T20:12:15.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.648+0000 7efe51400640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe400a3c40 0x7efe400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.648+0000 7efe51400640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe400a3c40 0x7efe400a6100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efe44000c00 tx=0x7efe44030980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.652+0000 7efe53400640 1 -- 172.21.15.131:0/4047817950 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efe4c09ca70 con 0x7efe5c075330 2024-04-26T20:12:15.661 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:15 smithi148 ceph-mon[32281]: pgmap v477: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:15.702 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:15 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4047817950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:12:15.742 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:15 smithi157 ceph-mon[32703]: pgmap v477: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:15.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:15 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4047817950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:12:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.903+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe5c02dad0 con 0x7efe5c075330 2024-04-26T20:12:15.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.905+0000 7efe53400640 1 -- 172.21.15.131:0/4047817950 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11895 (secure 0 0 0) 0x7efe4c0a0d70 con 0x7efe5c075330 2024-04-26T20:12:15.787 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:12:15.796 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:08:57.196498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24557,24571]},"id":1}]} 2024-04-26T20:12:15.804 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-04-26T20:12:15.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe400a3c40 msgr2=0x7efe400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:15.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe400a3c40 0x7efe400a6100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efe44000c00 tx=0x7efe44030980 comp rx=0 tx=0).stop 2024-04-26T20:12:15.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c075330 msgr2=0x7efe5c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c075330 0x7efe5c19ab70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e9f0 tx=0x7efe4c00eeb0 comp rx=0 tx=0).stop 2024-04-26T20:12:15.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 shutdown_connections 2024-04-26T20:12:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efe400a3c40 0x7efe400a6100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:16.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5c10a5e0 0x7efe5c1a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:16.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c076910 0x7efe5c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:16.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 --2- 172.21.15.131:0/4047817950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe5c075330 0x7efe5c19ab70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 >> 172.21.15.131:0/4047817950 conn(0x7efe5c0fdcc0 msgr2=0x7efe5c106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:16.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 shutdown_connections 2024-04-26T20:12:16.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:14.908+0000 7efe5b200640 1 -- 172.21.15.131:0/4047817950 wait complete. 2024-04-26T20:12:16.142 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:12:16.149 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:15 smithi131 ceph-mon[27919]: pgmap v477: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:16.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:15 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4047817950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:12:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 -- 172.21.15.131:0/3571192522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 msgr2=0x7f7c9c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 --2- 172.21.15.131:0/3571192522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c1091b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009860 tx=0x7f7c9002a340 comp rx=0 tx=0).stop 2024-04-26T20:12:16.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 -- 172.21.15.131:0/3571192522 shutdown_connections 2024-04-26T20:12:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 --2- 172.21.15.131:0/3571192522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c9c10a940 0x7f7c9c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:16.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 --2- 172.21.15.131:0/3571192522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c109f80 0x7f7c9c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 --2- 172.21.15.131:0/3571192522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c1091b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:16.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 -- 172.21.15.131:0/3571192522 >> 172.21.15.131:0/3571192522 conn(0x7f7c9c077d30 msgr2=0x7f7c9c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:16.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 -- 172.21.15.131:0/3571192522 shutdown_connections 2024-04-26T20:12:16.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.421+0000 7f7ca2200640 1 -- 172.21.15.131:0/3571192522 wait complete. 2024-04-26T20:12:16.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.422+0000 7f7ca2200640 1 Processor -- start 2024-04-26T20:12:16.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca2200640 1 -- start start 2024-04-26T20:12:16.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca2200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:16.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c109f80 0x7f7c9c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca2200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c9c10a940 0x7f7c9c1a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca2200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1148b0 con 0x7f7c9c10a940 2024-04-26T20:12:16.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca2200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c114730 con 0x7f7c9c108d90 2024-04-26T20:12:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c114a30 con 0x7f7c9c109f80 2024-04-26T20:12:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca0e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:16.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c9c10a940 0x7f7c9c1a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:16.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca0e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42438/0 (socket says 172.21.15.131:42438) 2024-04-26T20:12:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7ca0e00640 1 -- 172.21.15.131:0/1921234968 learned_addr learned my addr 172.21.15.131:0/1921234968 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.423+0000 7f7c9be00640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c109f80 0x7f7c9c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:17.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7ca0e00640 1 -- 172.21.15.131:0/1921234968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c109f80 msgr2=0x7f7c9c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7ca0e00640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c109f80 0x7f7c9c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:17.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7ca0e00640 1 -- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c9c10a940 msgr2=0x7f7c9c1a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:17.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7ca0e00640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c9c10a940 0x7f7c9c1a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:17.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7ca0e00640 1 -- 172.21.15.131:0/1921234968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c1a6c20 con 0x7f7c9c108d90 2024-04-26T20:12:17.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7ca0e00640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c19f000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c900057c0 tx=0x7f7c90032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:17.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7c99600640 1 -- 172.21.15.131:0/1921234968 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90037040 con 0x7f7c9c108d90 2024-04-26T20:12:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7c99600640 1 -- 172.21.15.131:0/1921234968 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c90030bd0 con 0x7f7c9c108d90 2024-04-26T20:12:17.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c1a6eb0 con 0x7f7c9c108d90 2024-04-26T20:12:17.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.424+0000 7f7c99600640 1 -- 172.21.15.131:0/1921234968 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90005280 con 0x7f7c9c108d90 2024-04-26T20:12:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.425+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c1a7260 con 0x7f7c9c108d90 2024-04-26T20:12:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.426+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c110180 con 0x7f7c9c108d90 2024-04-26T20:12:17.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.428+0000 7f7c99600640 1 -- 172.21.15.131:0/1921234968 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7c90043020 con 0x7f7c9c108d90 2024-04-26T20:12:17.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.429+0000 7f7c99600640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c840a3c40 0x7f7c840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:17.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.430+0000 7f7c99600640 1 -- 172.21.15.131:0/1921234968 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7c900f0730 con 0x7f7c9c108d90 2024-04-26T20:12:17.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.430+0000 7f7c9be00640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c840a3c40 0x7f7c840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:17.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.430+0000 7f7c9be00640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c840a3c40 0x7f7c840a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c1a0510 tx=0x7f7c8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:17.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.432+0000 7f7c99600640 1 -- 172.21.15.131:0/1921234968 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c900bccd0 con 0x7f7c9c108d90 2024-04-26T20:12:17.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.635+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c9c02dca0 con 0x7f7c9c108d90 2024-04-26T20:12:17.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.637+0000 7f7c99600640 1 -- 172.21.15.131:0/1921234968 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7f7c900bc5d0 con 0x7f7c9c108d90 2024-04-26T20:12:17.410 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:12:17.418 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:12:17.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c840a3c40 msgr2=0x7f7c840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:17.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c840a3c40 0x7f7c840a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c1a0510 tx=0x7f7c8c005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:17.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 msgr2=0x7f7c9c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:17.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c19f000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c900057c0 tx=0x7f7c90032da0 comp rx=0 tx=0).stop 2024-04-26T20:12:17.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 shutdown_connections 2024-04-26T20:12:17.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c9c10a940 0x7f7c9c1a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:17.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c840a3c40 0x7f7c840a6100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:17.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c9c109f80 0x7f7c9c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:17.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 --2- 172.21.15.131:0/1921234968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c9c108d90 0x7f7c9c19f000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:17.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.639+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 >> 172.21.15.131:0/1921234968 conn(0x7f7c9c077d30 msgr2=0x7f7c9c10d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:17.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.640+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 shutdown_connections 2024-04-26T20:12:17.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:16.640+0000 7f7ca2200640 1 -- 172.21.15.131:0/1921234968 wait complete. 2024-04-26T20:12:17.888 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T20:07:34.194239+0000', 'modified': '2024-04-26T20:08:57.196498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24575, 'mds_1': 24571, 'mds_2': 24557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/3509306790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 3509306790}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 3509306790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6837/1158905219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1158905219}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1158905219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6839/1960857281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1960857281}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1960857281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'g', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.131:6839/2686418281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 2686418281}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 2686418281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24571': {'gid': 24571, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.131:6841/1288851219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6840', 'nonce': 1288851219}, {'type': 'v1', 'addr': '172.21.15.131:6841', 'nonce': 1288851219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6839/4047119723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 4047119723}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 4047119723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24575, 'qdb_cluster': [24575, 24557, 24571]} 2024-04-26T20:12:17.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T20:12:17.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-26T20:12:17.964 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:17 smithi148 ceph-mon[32281]: pgmap v478: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:18.000 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:17 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1921234968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:18.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:17 smithi157 ceph-mon[32703]: pgmap v478: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:18.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:17 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1921234968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:18.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:17 smithi131 ceph-mon[27919]: pgmap v478: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:18.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:17 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1921234968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 -- 172.21.15.131:0/196457788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980772f0 msgr2=0x7f2898077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:18.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 --2- 172.21.15.131:0/196457788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980772f0 0x7f2898077710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f288c009900 tx=0x7f288c02a410 comp rx=0 tx=0).stop 2024-04-26T20:12:18.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 -- 172.21.15.131:0/196457788 shutdown_connections 2024-04-26T20:12:18.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 --2- 172.21.15.131:0/196457788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2898078ed0 0x7f2898079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 --2- 172.21.15.131:0/196457788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2898078530 0x7f2898078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:18.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 --2- 172.21.15.131:0/196457788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980772f0 0x7f2898077710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:18.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 -- 172.21.15.131:0/196457788 >> 172.21.15.131:0/196457788 conn(0x7f28980ffec0 msgr2=0x7f2898102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:18.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 -- 172.21.15.131:0/196457788 shutdown_connections 2024-04-26T20:12:18.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.299+0000 7f289e400640 1 -- 172.21.15.131:0/196457788 wait complete. 2024-04-26T20:12:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 Processor -- start 2024-04-26T20:12:18.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 -- start start 2024-04-26T20:12:18.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28980772f0 0x7f289819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:18.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2898078530 0x7f289819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:18.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898078ed0 0x7f28981a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:18.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f28981148f0 con 0x7f2898078ed0 2024-04-26T20:12:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2898114770 con 0x7f28980772f0 2024-04-26T20:12:18.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2898114a70 con 0x7f2898078530 2024-04-26T20:12:18.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898078ed0 0x7f28981a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:18.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.300+0000 7f289d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28980772f0 0x7f289819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:18.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28980772f0 0x7f289819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42460/0 (socket says 172.21.15.131:42460) 2024-04-26T20:12:18.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289d000640 1 -- 172.21.15.131:0/1381713133 learned_addr learned my addr 172.21.15.131:0/1381713133 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f2897e00640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2898078530 0x7f289819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:18.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289da00640 1 -- 172.21.15.131:0/1381713133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2898078530 msgr2=0x7f289819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:19.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289da00640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2898078530 0x7f289819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:19.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289da00640 1 -- 172.21.15.131:0/1381713133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28980772f0 msgr2=0x7f289819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:19.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289da00640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28980772f0 0x7f289819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:19.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289da00640 1 -- 172.21.15.131:0/1381713133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28981a6bd0 con 0x7f2898078ed0 2024-04-26T20:12:19.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289d000640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28980772f0 0x7f289819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f289da00640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898078ed0 0x7f28981a6560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2890009fd0 tx=0x7f289000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f2895600640 1 -- 172.21.15.131:0/1381713133 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289000ce20 con 0x7f2898078ed0 2024-04-26T20:12:19.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.301+0000 7f2895600640 1 -- 172.21.15.131:0/1381713133 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2890005020 con 0x7f2898078ed0 2024-04-26T20:12:19.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.302+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28981a6ec0 con 0x7f2898078ed0 2024-04-26T20:12:19.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.302+0000 7f2895600640 1 -- 172.21.15.131:0/1381713133 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890010690 con 0x7f2898078ed0 2024-04-26T20:12:19.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.302+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981a7380 con 0x7f2898078ed0 2024-04-26T20:12:19.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 Processor -- start 2024-04-26T20:12:19.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f2895600640 1 -- 172.21.15.131:0/1381713133 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f28900054d0 con 0x7f2898078ed0 2024-04-26T20:12:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 -- start start 2024-04-26T20:12:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c72c0 0x5639701c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c7cb0 0x5639701cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:19.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56397012b600 con 0x5639701cce20 2024-04-26T20:12:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5639700c8100 con 0x5639701c7cb0 2024-04-26T20:12:19.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.305+0000 7f28a29da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5639701275c0 con 0x5639701c72c0 2024-04-26T20:12:19.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c7cb0 0x5639701cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:19.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f2897e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:19.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c7cb0 0x5639701cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42472/0 (socket says 172.21.15.131:42472) 2024-04-26T20:12:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 -- 172.21.15.131:0/2563540204 learned_addr learned my addr 172.21.15.131:0/2563540204 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f2897e00640 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c72c0 0x5639701c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:19.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 -- 172.21.15.131:0/2563540204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c72c0 msgr2=0x5639701c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:19.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c72c0 0x5639701c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 -- 172.21.15.131:0/2563540204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 msgr2=0x5639701cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:19.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:19.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.306+0000 7f289d000640 1 -- 172.21.15.131:0/2563540204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56396fe3d2d0 con 0x5639701c7cb0 2024-04-26T20:12:19.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.307+0000 7f289d000640 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c7cb0 0x5639701cc7b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f288c0044a0 tx=0x7f288c0044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e49f85d9b7c5315 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.307+0000 7f2895600640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28800a3cc0 0x7f28800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:19.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.307+0000 7f288a000640 1 -- 172.21.15.131:0/2563540204 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c035030 con 0x5639701c7cb0 2024-04-26T20:12:19.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.307+0000 7f288a000640 1 -- 172.21.15.131:0/2563540204 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288c040070 con 0x5639701c7cb0 2024-04-26T20:12:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.307+0000 7f288a000640 1 -- 172.21.15.131:0/2563540204 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c031e10 con 0x5639701c7cb0 2024-04-26T20:12:19.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.307+0000 7f2895600640 1 -- 172.21.15.131:0/1381713133 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f28900d0f90 con 0x7f2898078ed0 2024-04-26T20:12:19.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.307+0000 7f2897e00640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28800a3cc0 0x7f28800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:19.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f2897e00640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28800a3cc0 0x7f28800a6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f288400bfd0 tx=0x7f288400b650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 -- 172.21.15.131:0/2563540204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c7cb0 msgr2=0x5639701cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:19.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c7cb0 0x5639701cc7b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f288c0044a0 tx=0x7f288c0044d0 comp rx=0 tx=0).stop 2024-04-26T20:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 -- 172.21.15.131:0/2563540204 shutdown_connections 2024-04-26T20:12:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c7cb0 0x5639701cc7b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:20.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 --2- 172.21.15.131:0/2563540204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c72c0 0x5639701c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:20.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 -- 172.21.15.131:0/2563540204 >> 172.21.15.131:0/2563540204 conn(0x56397015bd10 msgr2=0x5639701c2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 -- 172.21.15.131:0/2563540204 shutdown_connections 2024-04-26T20:12:20.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.308+0000 7f28a29da740 1 -- 172.21.15.131:0/2563540204 wait complete. 2024-04-26T20:12:20.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 -- start start 2024-04-26T20:12:20.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 Processor -- start 2024-04-26T20:12:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:20.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701c7cb0 0x5639701d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:20.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701cce20 0x5639701d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5639701cfd70 con 0x5639701c7cb0 2024-04-26T20:12:20.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5639700b2f70 con 0x5639701c72c0 2024-04-26T20:12:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5639701cfef0 con 0x5639701cce20 2024-04-26T20:12:20.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.309+0000 7f28a29da740 10 client.0 _ll_register_callbacks cb 0x563970121480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:12:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f2897e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701c7cb0 0x5639701d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:20.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f2897e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701c7cb0 0x5639701d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59534/0 (socket says 172.21.15.131:59534) 2024-04-26T20:12:20.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:20.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f2897e00640 1 -- 172.21.15.131:0/1239397229 learned_addr learned my addr 172.21.15.131:0/1239397229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289da00640 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701cce20 0x5639701d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289d000640 1 -- 172.21.15.131:0/1239397229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701cce20 msgr2=0x5639701d8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:20.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289d000640 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701cce20 0x5639701d8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:20.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289d000640 1 -- 172.21.15.131:0/1239397229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701c7cb0 msgr2=0x5639701d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:20.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289d000640 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701c7cb0 0x5639701d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:20.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289d000640 1 -- 172.21.15.131:0/1239397229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56396ff076b0 con 0x5639701c72c0 2024-04-26T20:12:20.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f289d000640 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d1710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5639701c3bf0 tx=0x7f288c031e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ebb85e8180ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.310+0000 7f287ea00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:12:20.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.311+0000 7f287ea00640 1 -- 172.21.15.131:0/1239397229 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c053060 con 0x5639701c72c0 2024-04-26T20:12:20.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.311+0000 7f287ea00640 1 -- 172.21.15.131:0/1239397229 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f288c003a80 con 0x5639701c72c0 2024-04-26T20:12:20.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.311+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288c040070 con 0x5639701c72c0 2024-04-26T20:12:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.311+0000 7f287ea00640 1 -- 172.21.15.131:0/1239397229 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f288c05a050 con 0x5639701c72c0 2024-04-26T20:12:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.311+0000 7f28a29da740 10 client.25322 fetch_fsmap learned FSMap version 26 2024-04-26T20:12:20.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.311+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639701dbc70 con 0x5639701c72c0 2024-04-26T20:12:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.311+0000 7f287ea00640 1 -- 172.21.15.131:0/1239397229 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288c040070 con 0x5639701c72c0 2024-04-26T20:12:20.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f287ea00640 1 -- 172.21.15.131:0/1239397229 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f288c04d030 con 0x5639701c72c0 2024-04-26T20:12:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f28a29da740 10 client.25322 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:12:20.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f28a29da740 10 client.25322 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T20:12:20.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f28a29da740 20 client.25322 populate_metadata read hostname 'smithi131' 2024-04-26T20:12:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f28a29da740 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5639701dde40 0x5639701df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:20.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f28a29da740 4 client.25322 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:12:20.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563970139210 con 0x5639701dde40 2024-04-26T20:12:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.312+0000 7f2897e00640 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5639701dde40 0x5639701df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.313+0000 7f2897e00640 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5639701dde40 0x5639701df2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51c7e0c4ebcd226e server_cookie=dd8543c8ba38e6d1 in_seq=0 out_seq=0 2024-04-26T20:12:21.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.313+0000 7f287ea00640 10 client.25322 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:12:21.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.313+0000 7f287ea00640 1 -- 172.21.15.131:0/1239397229 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563970139210 con 0x5639701dde40 2024-04-26T20:12:21.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.313+0000 7f287ea00640 10 client.25322 handle_command_reply: tid=0 2024-04-26T20:12:21.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 client.25322 shutdown 2024-04-26T20:12:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 msgr2=0x5639701d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:21.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d1710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5639701c3bf0 tx=0x7f288c031e00 comp rx=0 tx=0).stop 2024-04-26T20:12:21.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 shutdown_connections 2024-04-26T20:12:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5639701dde40 0x5639701df2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:21.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701cce20 0x5639701d8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:21.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701c7cb0 0x5639701d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 --2- 172.21.15.131:0/1239397229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d1710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:21.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f287ea00640 0 client.25322 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:12:21.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 >> 172.21.15.131:0/1239397229 conn(0x56397015bd10 msgr2=0x5639701c1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:21.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 shutdown_connections 2024-04-26T20:12:21.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 1 -- 172.21.15.131:0/1239397229 wait complete. 2024-04-26T20:12:21.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.314+0000 7f28a29da740 20 client.25322 trim_cache size 0 max 16384 2024-04-26T20:12:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 Processor -- start 2024-04-26T20:12:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 -- start start 2024-04-26T20:12:21.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:21.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:21.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:21.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5639701dd5f0 con 0x5639701cce20 2024-04-26T20:12:21.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5639701ddcc0 con 0x5639701c72c0 2024-04-26T20:12:21.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.334+0000 7f28a29da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5639701de390 con 0x5639701c7cb0 2024-04-26T20:12:21.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:21.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42488/0 (socket says 172.21.15.131:42488) 2024-04-26T20:12:21.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f2897e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:21.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 -- 172.21.15.131:0/3845222079 learned_addr learned my addr 172.21.15.131:0/3845222079 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:21.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 -- 172.21.15.131:0/3845222079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 msgr2=0x5639701d8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:21.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 -- 172.21.15.131:0/3845222079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 msgr2=0x5639701c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:21.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:21.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 -- 172.21.15.131:0/3845222079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639701c8090 con 0x5639701c72c0 2024-04-26T20:12:22.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f2897e00640 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:22.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f289d000640 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701c5c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f288c045970 tx=0x7f288c001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a93818b19edcb293 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f2889600640 1 -- 172.21.15.131:0/3845222079 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c053030 con 0x5639701c72c0 2024-04-26T20:12:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f2889600640 1 -- 172.21.15.131:0/3845222079 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288c040040 con 0x5639701c72c0 2024-04-26T20:12:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.335+0000 7f2889600640 1 -- 172.21.15.131:0/3845222079 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c035070 con 0x5639701c72c0 2024-04-26T20:12:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 -- 172.21.15.131:0/3845222079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 msgr2=0x5639701c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701c5c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f288c045970 tx=0x7f288c001e70 comp rx=0 tx=0).stop 2024-04-26T20:12:22.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 -- 172.21.15.131:0/3845222079 shutdown_connections 2024-04-26T20:12:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 --2- 172.21.15.131:0/3845222079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701c5c80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:22.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 -- 172.21.15.131:0/3845222079 >> 172.21.15.131:0/3845222079 conn(0x56397015bd10 msgr2=0x5639701c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:22.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 -- 172.21.15.131:0/3845222079 shutdown_connections 2024-04-26T20:12:22.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.336+0000 7f28a29da740 1 -- 172.21.15.131:0/3845222079 wait complete. 2024-04-26T20:12:22.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 -- start start 2024-04-26T20:12:22.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 Processor -- start 2024-04-26T20:12:22.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:22.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:22.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5639701df130 con 0x5639701cce20 2024-04-26T20:12:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5639701dea60 con 0x5639701c72c0 2024-04-26T20:12:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5639701df800 con 0x5639701c7cb0 2024-04-26T20:12:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f28a29da740 10 client.0 _ll_register_callbacks cb 0x7f288400bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:12:22.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f289da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f289da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701d8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59544/0 (socket says 172.21.15.131:59544) 2024-04-26T20:12:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f289da00640 1 -- 172.21.15.131:0/3393390690 learned_addr learned my addr 172.21.15.131:0/3393390690 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:22.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f2897e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.337+0000 7f289d000640 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:22.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f289da00640 1 -- 172.21.15.131:0/3393390690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 msgr2=0x5639701d53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f289da00640 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:22.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f289da00640 1 -- 172.21.15.131:0/3393390690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 msgr2=0x5639701d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f289da00640 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:22.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f289da00640 1 -- 172.21.15.131:0/3393390690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639701d8960 con 0x5639701cce20 2024-04-26T20:12:22.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f289da00640 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701d8370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f28900ce520 tx=0x7f289009cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b675bfc6ff37c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:22.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f287ea00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:12:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f287ea00640 1 -- 172.21.15.131:0/3393390690 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900a5930 con 0x5639701cce20 2024-04-26T20:12:23.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f287ea00640 1 -- 172.21.15.131:0/3393390690 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f28900051d0 con 0x5639701cce20 2024-04-26T20:12:23.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.338+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288c040040 con 0x5639701cce20 2024-04-26T20:12:23.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f287ea00640 1 -- 172.21.15.131:0/3393390690 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2890082050 con 0x5639701cce20 2024-04-26T20:12:23.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 10 client.15204 fetch_fsmap learned FSMap version 26 2024-04-26T20:12:23.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639702449c0 con 0x5639701cce20 2024-04-26T20:12:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f287ea00640 1 -- 172.21.15.131:0/3393390690 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288c040040 con 0x5639701cce20 2024-04-26T20:12:23.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f287ea00640 1 -- 172.21.15.131:0/3393390690 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f28900ceb00 con 0x5639701cce20 2024-04-26T20:12:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 10 client.15204 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:12:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 10 client.15204 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T20:12:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 20 client.15204 populate_metadata read hostname 'smithi131' 2024-04-26T20:12:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x563970249540 0x563970248250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:23.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 4 client.15204 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:12:23.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "status"}) -- 0x563970133080 con 0x563970249540 2024-04-26T20:12:23.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.339+0000 7f289d000640 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x563970249540 0x563970248250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:23.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.340+0000 7f289d000640 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x563970249540 0x563970248250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c2c320ea265386c server_cookie=ef28b41bf5f22b8c in_seq=0 out_seq=0 2024-04-26T20:12:23.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.340+0000 7f287ea00640 10 client.15204 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:12:23.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.340+0000 7f287ea00640 1 -- 172.21.15.131:0/3393390690 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563970133080 con 0x563970249540 2024-04-26T20:12:23.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.340+0000 7f287ea00640 10 client.15204 handle_command_reply: tid=0 2024-04-26T20:12:23.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.340+0000 7f28a29da740 1 client.15204 shutdown 2024-04-26T20:12:23.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.340+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 msgr2=0x5639701d8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:23.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.340+0000 7f28a29da740 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701d8370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f28900ce520 tx=0x7f289009cd80 comp rx=0 tx=0).stop 2024-04-26T20:12:23.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 shutdown_connections 2024-04-26T20:12:23.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x563970249540 0x563970248250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:23.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639701cce20 0x5639701d8370 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:23.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5639701c7cb0 0x5639701d53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:23.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 --2- 172.21.15.131:0/3393390690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639701c72c0 0x5639701d4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:23.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f287ea00640 0 client.15204 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:12:23.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 >> 172.21.15.131:0/3393390690 conn(0x56397015bd10 msgr2=0x5639701c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:23.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 shutdown_connections 2024-04-26T20:12:23.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 1 -- 172.21.15.131:0/3393390690 wait complete. 2024-04-26T20:12:23.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f28a29da740 20 client.15204 trim_cache size 0 max 16384 2024-04-26T20:12:23.778 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:12:23.783 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T20:12:23.873 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-04-26T20:12:23.916 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24557, 2024-04-26T20:12:23.923 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-26T20:12:23.929 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-26T20:12:23.935 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-26T20:12:23.953 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 208.398987436, 2024-04-26T20:12:23.959 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-04-26T20:12:23.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 97, 2024-04-26T20:12:24.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T20:12:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 313.34188536900001 2024-04-26T20:12:24.177 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:12:24.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28800a3cc0 msgr2=0x7f28800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:24.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f289e400640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28800a3cc0 0x7f28800a6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f288400bfd0 tx=0x7f288400b650 comp rx=0 tx=0).stop 2024-04-26T20:12:24.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.341+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898078ed0 msgr2=0x7f28981a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:24.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898078ed0 0x7f28981a6560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2890009fd0 tx=0x7f289000ef10 comp rx=0 tx=0).stop 2024-04-26T20:12:24.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 shutdown_connections 2024-04-26T20:12:24.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898078ed0 0x7f28981a6560 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f28800a3cc0 0x7f28800a6180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2898078530 0x7f289819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 --2- 172.21.15.131:0/1381713133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28980772f0 0x7f289819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:24.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 >> 172.21.15.131:0/1381713133 conn(0x7f28980ffec0 msgr2=0x7f28981019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:24.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 shutdown_connections 2024-04-26T20:12:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:18.342+0000 7f289e400640 1 -- 172.21.15.131:0/1381713133 wait complete. 2024-04-26T20:12:24.504 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:12:24.533 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:19 smithi131 ceph-mon[27919]: pgmap v479: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:24.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:21 smithi131 ceph-mon[27919]: pgmap v480: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:24.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:23 smithi131 ceph-mon[27919]: pgmap v481: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:24.635 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:19 smithi148 ceph-mon[32281]: pgmap v479: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:24.645 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:21 smithi148 ceph-mon[32281]: pgmap v480: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:24.652 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:23 smithi148 ceph-mon[32281]: pgmap v481: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:24.659 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:19 smithi157 ceph-mon[32703]: pgmap v479: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:24.667 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:21 smithi157 ceph-mon[32703]: pgmap v480: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:24.673 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:23 smithi157 ceph-mon[32703]: pgmap v481: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:24.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 -- 172.21.15.131:0/4067891975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8103b60 msgr2=0x7f50b8103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:24.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 --2- 172.21.15.131:0/4067891975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8103b60 0x7f50b8103fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a002a400 comp rx=0 tx=0).stop 2024-04-26T20:12:24.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 -- 172.21.15.131:0/4067891975 shutdown_connections 2024-04-26T20:12:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 --2- 172.21.15.131:0/4067891975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b8104520 0x7f50b810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 --2- 172.21.15.131:0/4067891975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8103b60 0x7f50b8103fe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:24.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 --2- 172.21.15.131:0/4067891975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b8102970 0x7f50b8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 -- 172.21.15.131:0/4067891975 >> 172.21.15.131:0/4067891975 conn(0x7f50b80fdee0 msgr2=0x7f50b8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 -- 172.21.15.131:0/4067891975 shutdown_connections 2024-04-26T20:12:24.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.846+0000 7f50bd800640 1 -- 172.21.15.131:0/4067891975 wait complete. 2024-04-26T20:12:25.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.847+0000 7f50bd800640 1 Processor -- start 2024-04-26T20:12:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.847+0000 7f50bd800640 1 -- start start 2024-04-26T20:12:25.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50bd800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8102970 0x7f50b80766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50bd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b8103b60 0x7f50b8074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:25.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50bd800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b8104520 0x7f50b8075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50bd800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810e480 con 0x7f50b8103b60 2024-04-26T20:12:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50bd800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810e300 con 0x7f50b8104520 2024-04-26T20:12:25.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50bd800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810e600 con 0x7f50b8102970 2024-04-26T20:12:25.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b8103b60 0x7f50b8074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8102970 0x7f50b80766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8102970 0x7f50b80766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44864/0 (socket says 172.21.15.131:44864) 2024-04-26T20:12:25.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 -- 172.21.15.131:0/863382416 learned_addr learned my addr 172.21.15.131:0/863382416 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b7400640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b8104520 0x7f50b8075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 -- 172.21.15.131:0/863382416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b8104520 msgr2=0x7f50b8075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:25.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b8104520 0x7f50b8075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:25.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 -- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b8103b60 msgr2=0x7f50b8074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:25.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b8103b60 0x7f50b8074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:25.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50b6a00640 1 -- 172.21.15.131:0/863382416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b8075af0 con 0x7f50b8102970 2024-04-26T20:12:25.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.848+0000 7f50afe00640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b8103b60 0x7f50b8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:25.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.849+0000 7f50b6a00640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8102970 0x7f50b80766e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e9c0 tx=0x7f50a800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:25.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.849+0000 7f50af400640 1 -- 172.21.15.131:0/863382416 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a800cd80 con 0x7f50b8102970 2024-04-26T20:12:25.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.849+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b81a2ca0 con 0x7f50b8102970 2024-04-26T20:12:25.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.849+0000 7f50af400640 1 -- 172.21.15.131:0/863382416 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a80035f0 con 0x7f50b8102970 2024-04-26T20:12:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.849+0000 7f50af400640 1 -- 172.21.15.131:0/863382416 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8010690 con 0x7f50b8102970 2024-04-26T20:12:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.849+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b81a31c0 con 0x7f50b8102970 2024-04-26T20:12:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.852+0000 7f50c0567740 1 Processor -- start 2024-04-26T20:12:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.852+0000 7f50c0567740 1 -- start start 2024-04-26T20:12:25.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50c0567740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25caa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50c0567740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25caf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:25.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50c0567740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:25.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50c0567740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556e25c0e600 con 0x556e25caa0f0 2024-04-26T20:12:25.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50c0567740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e25bab100 con 0x556e25cafc50 2024-04-26T20:12:25.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50c0567740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e25c0a5c0 con 0x556e25caaae0 2024-04-26T20:12:25.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50af400640 1 -- 172.21.15.131:0/863382416 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f50a80037a0 con 0x7f50b8102970 2024-04-26T20:12:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50b7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25caa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:26.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50b7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25caa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55962/0 (socket says 172.21.15.131:55962) 2024-04-26T20:12:26.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50b7400640 1 -- 172.21.15.131:0/898243658 learned_addr learned my addr 172.21.15.131:0/898243658 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:26.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50afe00640 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50b6a00640 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25caf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50afe00640 1 -- 172.21.15.131:0/898243658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 msgr2=0x556e25caf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:26.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50afe00640 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25caf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50afe00640 1 -- 172.21.15.131:0/898243658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 msgr2=0x556e25caa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:26.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50afe00640 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25caa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50afe00640 1 -- 172.21.15.131:0/898243658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e259202d0 con 0x556e25cafc50 2024-04-26T20:12:26.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.853+0000 7f50b6a00640 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25caf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:12:26.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.854+0000 7f50afe00640 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb20e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005ab0 tx=0x7f50a0005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d2e1a36c6d9681 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:26.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.854+0000 7f50a7e00640 1 -- 172.21.15.131:0/898243658 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0040070 con 0x556e25cafc50 2024-04-26T20:12:26.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.854+0000 7f50a7e00640 1 -- 172.21.15.131:0/898243658 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a00318a0 con 0x556e25cafc50 2024-04-26T20:12:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.854+0000 7f50a7e00640 1 -- 172.21.15.131:0/898243658 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0035040 con 0x556e25cafc50 2024-04-26T20:12:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.854+0000 7f50af400640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f509c0a3cc0 0x7f509c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:26.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50af400640 1 -- 172.21.15.131:0/863382416 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f50a80d0ca0 con 0x7f50b8102970 2024-04-26T20:12:26.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50b7400640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f509c0a3cc0 0x7f509c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:26.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 -- 172.21.15.131:0/898243658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 msgr2=0x556e25cb20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:26.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb20e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005ab0 tx=0x7f50a0005df0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 -- 172.21.15.131:0/898243658 shutdown_connections 2024-04-26T20:12:26.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb20e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25caf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 --2- 172.21.15.131:0/898243658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25caa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 -- 172.21.15.131:0/898243658 >> 172.21.15.131:0/898243658 conn(0x556e25c3ed10 msgr2=0x556e25ca5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:26.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 -- 172.21.15.131:0/898243658 shutdown_connections 2024-04-26T20:12:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50c0567740 1 -- 172.21.15.131:0/898243658 wait complete. 2024-04-26T20:12:26.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.855+0000 7f50b7400640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f509c0a3cc0 0x7f509c0a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x556e25cab340 tx=0x7f50b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:26.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 -- start start 2024-04-26T20:12:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 Processor -- start 2024-04-26T20:12:26.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:26.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cb4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:26.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cbbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:26.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556e25cb2ba0 con 0x556e25caa0f0 2024-04-26T20:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e25b95f70 con 0x556e25cafc50 2024-04-26T20:12:26.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e25cb2d20 con 0x556e25caaae0 2024-04-26T20:12:26.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50c0567740 10 client.0 _ll_register_callbacks cb 0x556e25c04480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:12:26.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.856+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:26.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55970/0 (socket says 172.21.15.131:55970) 2024-04-26T20:12:26.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 -- 172.21.15.131:0/2745891574 learned_addr learned my addr 172.21.15.131:0/2745891574 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:26.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b7400640 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cbbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50afe00640 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cb4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:26.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 -- 172.21.15.131:0/2745891574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 msgr2=0x556e25cb4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:26.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cb4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 -- 172.21.15.131:0/2745891574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 msgr2=0x556e25cbbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:26.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cbbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:26.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 -- 172.21.15.131:0/2745891574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e259ea6b0 con 0x556e25caa0f0 2024-04-26T20:12:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50b6a00640 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb4520 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000c00 tx=0x7f50a809c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff80da82cb2953f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:12:27.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50a4c00640 1 -- 172.21.15.131:0/2745891574 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a80a0de0 con 0x556e25caa0f0 2024-04-26T20:12:27.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.857+0000 7f50a4c00640 1 -- 172.21.15.131:0/2745891574 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f50a80a5210 con 0x556e25caa0f0 2024-04-26T20:12:27.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a00318a0 con 0x556e25caa0f0 2024-04-26T20:12:27.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50a4c00640 1 -- 172.21.15.131:0/2745891574 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50a80a56a0 con 0x556e25caa0f0 2024-04-26T20:12:27.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 10 client.15216 fetch_fsmap learned FSMap version 26 2024-04-26T20:12:27.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e25cbeac0 con 0x556e25caa0f0 2024-04-26T20:12:27.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50a4c00640 1 -- 172.21.15.131:0/2745891574 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a00318a0 con 0x556e25caa0f0 2024-04-26T20:12:27.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50a4c00640 1 -- 172.21.15.131:0/2745891574 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f50a809b8f0 con 0x556e25caa0f0 2024-04-26T20:12:27.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:12:27.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 10 client.15216 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:12:27.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 20 client.15216 populate_metadata read hostname 'smithi131' 2024-04-26T20:12:27.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25cc1180 0x556e25cc25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:27.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 4 client.15216 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:12:27.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.858+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e25c1c210 con 0x556e25cc1180 2024-04-26T20:12:27.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.859+0000 7f50afe00640 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25cc1180 0x556e25cc25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:27.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.859+0000 7f50afe00640 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25cc1180 0x556e25cc25f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91242fadf8de32b8 server_cookie=e15716e9b3e9d546 in_seq=0 out_seq=0 2024-04-26T20:12:27.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.859+0000 7f50a4c00640 10 client.15216 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:12:27.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50a4c00640 1 -- 172.21.15.131:0/2745891574 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e25c1c210 con 0x556e25cc1180 2024-04-26T20:12:27.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50a4c00640 10 client.15216 handle_command_reply: tid=0 2024-04-26T20:12:27.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 client.15216 shutdown 2024-04-26T20:12:27.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 msgr2=0x556e25cb4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:27.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb4520 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000c00 tx=0x7f50a809c390 comp rx=0 tx=0).stop 2024-04-26T20:12:27.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 shutdown_connections 2024-04-26T20:12:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25cc1180 0x556e25cc25f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:27.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cbbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:27.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cb4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 --2- 172.21.15.131:0/2745891574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb4520 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50a4c00640 0 client.15216 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:12:27.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.860+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 >> 172.21.15.131:0/2745891574 conn(0x556e25c3ed10 msgr2=0x556e25ca4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:27.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.861+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 shutdown_connections 2024-04-26T20:12:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.861+0000 7f50c0567740 1 -- 172.21.15.131:0/2745891574 wait complete. 2024-04-26T20:12:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.861+0000 7f50c0567740 20 client.15216 trim_cache size 0 max 16384 2024-04-26T20:12:27.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.880+0000 7f50c0567740 1 Processor -- start 2024-04-26T20:12:28.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.880+0000 7f50c0567740 1 -- start start 2024-04-26T20:12:28.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50c0567740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50c0567740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cbba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50c0567740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:28.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50c0567740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556e25cc0930 con 0x556e25caa0f0 2024-04-26T20:12:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50c0567740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e25cc1000 con 0x556e25cafc50 2024-04-26T20:12:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50c0567740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e25cc16d0 con 0x556e25caaae0 2024-04-26T20:12:28.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:28.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50afe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cbba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:28.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55982/0 (socket says 172.21.15.131:55982) 2024-04-26T20:12:28.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50b6a00640 1 -- 172.21.15.131:0/4157762253 learned_addr learned my addr 172.21.15.131:0/4157762253 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:28.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50b7400640 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:28.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50afe00640 1 -- 172.21.15.131:0/4157762253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 msgr2=0x556e25cb4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:28.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50afe00640 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50afe00640 1 -- 172.21.15.131:0/4157762253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 msgr2=0x556e25cb2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50afe00640 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:28.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50afe00640 1 -- 172.21.15.131:0/4157762253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e25ca6820 con 0x556e25caaae0 2024-04-26T20:12:28.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50b7400640 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:12:28.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.881+0000 7f50b6a00640 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.882+0000 7f50afe00640 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cbba30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000c00 tx=0x7f50a0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd319dadb43fd4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:28.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.882+0000 7f50a7400640 1 -- 172.21.15.131:0/4157762253 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0035040 con 0x556e25caaae0 2024-04-26T20:12:28.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.882+0000 7f50a7400640 1 -- 172.21.15.131:0/4157762253 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a00054d0 con 0x556e25caaae0 2024-04-26T20:12:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.882+0000 7f50a7400640 1 -- 172.21.15.131:0/4157762253 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0040040 con 0x556e25caaae0 2024-04-26T20:12:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 -- 172.21.15.131:0/4157762253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 msgr2=0x556e25cbba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:28.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cbba30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000c00 tx=0x7f50a0005fd0 comp rx=0 tx=0).stop 2024-04-26T20:12:28.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 -- 172.21.15.131:0/4157762253 shutdown_connections 2024-04-26T20:12:28.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25cafc50 0x556e25cb4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:28.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caaae0 0x556e25cbba30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:28.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 --2- 172.21.15.131:0/4157762253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25caa0f0 0x556e25cb2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 -- 172.21.15.131:0/4157762253 >> 172.21.15.131:0/4157762253 conn(0x556e25c3ed10 msgr2=0x556e25cabf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 -- 172.21.15.131:0/4157762253 shutdown_connections 2024-04-26T20:12:28.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 -- 172.21.15.131:0/4157762253 wait complete. 2024-04-26T20:12:28.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 -- start start 2024-04-26T20:12:28.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.883+0000 7f50c0567740 1 Processor -- start 2024-04-26T20:12:28.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50c0567740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caa0f0 0x556e25cb7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50c0567740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25caaae0 0x556e25cb7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:28.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50c0567740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25cafc50 0x556e25cbaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:28.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50c0567740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556e25cc2470 con 0x556e25cafc50 2024-04-26T20:12:28.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50c0567740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e25cc1da0 con 0x556e25caaae0 2024-04-26T20:12:28.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50c0567740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556e25cc2b40 con 0x556e25caa0f0 2024-04-26T20:12:28.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50c0567740 10 client.0 _ll_register_callbacks cb 0x7f50a8003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:12:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caa0f0 0x556e25cb7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:28.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caa0f0 0x556e25cb7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44890/0 (socket says 172.21.15.131:44890) 2024-04-26T20:12:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 -- 172.21.15.131:0/1500505030 learned_addr learned my addr 172.21.15.131:0/1500505030 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50afe00640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25caaae0 0x556e25cb7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:28.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b7400640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25cafc50 0x556e25cbaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:28.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 -- 172.21.15.131:0/1500505030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25caaae0 msgr2=0x556e25cb7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:29.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25caaae0 0x556e25cb7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 -- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25cafc50 msgr2=0x556e25cbaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25cafc50 0x556e25cbaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:29.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50b6a00640 1 -- 172.21.15.131:0/1500505030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e25cbb550 con 0x556e25caa0f0 2024-04-26T20:12:29.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.884+0000 7f50afe00640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25caaae0 0x556e25cb7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:29.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50b6a00640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caa0f0 0x556e25cb7a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50a809ace0 tx=0x7f50a800c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9765ea08ea73d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:29.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:12:29.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50a4c00640 1 -- 172.21.15.131:0/1500505030 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a80d3070 con 0x556e25caa0f0 2024-04-26T20:12:29.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a00054d0 con 0x556e25caa0f0 2024-04-26T20:12:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50a4c00640 1 -- 172.21.15.131:0/1500505030 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f50a809be80 con 0x556e25caa0f0 2024-04-26T20:12:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50a4c00640 1 -- 172.21.15.131:0/1500505030 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50a80bd0b0 con 0x556e25caa0f0 2024-04-26T20:12:29.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50c0567740 10 client.25342 fetch_fsmap learned FSMap version 26 2024-04-26T20:12:29.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.885+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e25d279c0 con 0x556e25caa0f0 2024-04-26T20:12:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50a4c00640 1 -- 172.21.15.131:0/1500505030 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a00054d0 con 0x556e25caa0f0 2024-04-26T20:12:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50a4c00640 1 -- 172.21.15.131:0/1500505030 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f50a80870b0 con 0x556e25caa0f0 2024-04-26T20:12:29.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50c0567740 10 client.25342 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:12:29.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50c0567740 10 client.25342 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:12:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50c0567740 20 client.25342 populate_metadata read hostname 'smithi131' 2024-04-26T20:12:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50c0567740 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25d2c3a0 0x556e25d2b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50c0567740 4 client.25342 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:12:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "status"}) -- 0x556e25c16080 con 0x556e25d2c3a0 2024-04-26T20:12:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.886+0000 7f50afe00640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25d2c3a0 0x556e25d2b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50afe00640 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25d2c3a0 0x556e25d2b0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9cd804cde124fa1 server_cookie=7558b095ce018df9 in_seq=0 out_seq=0 2024-04-26T20:12:29.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50a4c00640 10 client.25342 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:12:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50a4c00640 1 -- 172.21.15.131:0/1500505030 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556e25c16080 con 0x556e25d2c3a0 2024-04-26T20:12:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50a4c00640 10 client.25342 handle_command_reply: tid=0 2024-04-26T20:12:29.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 client.25342 shutdown 2024-04-26T20:12:29.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caa0f0 msgr2=0x556e25cb7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caa0f0 0x556e25cb7a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50a809ace0 tx=0x7f50a800c840 comp rx=0 tx=0).stop 2024-04-26T20:12:29.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 shutdown_connections 2024-04-26T20:12:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x556e25d2c3a0 0x556e25d2b0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e25cafc50 0x556e25cbaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e25caaae0 0x556e25cb7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 --2- 172.21.15.131:0/1500505030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556e25caa0f0 0x556e25cb7a00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:29.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50a4c00640 0 client.25342 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:12:29.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 >> 172.21.15.131:0/1500505030 conn(0x556e25c3ed10 msgr2=0x556e25cb4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 shutdown_connections 2024-04-26T20:12:29.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 1 -- 172.21.15.131:0/1500505030 wait complete. 2024-04-26T20:12:29.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.887+0000 7f50c0567740 20 client.25342 trim_cache size 0 max 16384 2024-04-26T20:12:30.072 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:12:30.165 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T20:12:30.172 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-04-26T20:12:30.179 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24571, 2024-04-26T20:12:30.185 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-26T20:12:30.191 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-26T20:12:30.230 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-26T20:12:30.235 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 215.95936764300001, 2024-04-26T20:12:30.240 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-04-26T20:12:30.247 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 97, 2024-04-26T20:12:30.357 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T20:12:30.364 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 315.68222058399999 2024-04-26T20:12:30.371 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:12:30.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f509c0a3cc0 msgr2=0x7f509c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:30.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f509c0a3cc0 0x7f509c0a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x556e25cab340 tx=0x7f50b0009500 comp rx=0 tx=0).stop 2024-04-26T20:12:30.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8102970 msgr2=0x7f50b80766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:30.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8102970 0x7f50b80766e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e9c0 tx=0x7f50a800ee80 comp rx=0 tx=0).stop 2024-04-26T20:12:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 shutdown_connections 2024-04-26T20:12:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b8104520 0x7f50b8075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:30.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b8103b60 0x7f50b8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:30.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f509c0a3cc0 0x7f509c0a6180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:30.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 --2- 172.21.15.131:0/863382416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b8102970 0x7f50b80766e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:30.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 >> 172.21.15.131:0/863382416 conn(0x7f50b80fdee0 msgr2=0x7f50b8106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:30.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 shutdown_connections 2024-04-26T20:12:30.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:24.888+0000 7f50bd800640 1 -- 172.21.15.131:0/863382416 wait complete. 2024-04-26T20:12:30.635 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T20:12:30.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:25 smithi131 ceph-mon[27919]: pgmap v482: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:30.740 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:27 smithi131 ceph-mon[27919]: pgmap v483: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:30.746 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:29 smithi131 ceph-mon[27919]: pgmap v484: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:30.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:25 smithi148 ceph-mon[32281]: pgmap v482: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:30.867 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:27 smithi148 ceph-mon[32281]: pgmap v483: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:30.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:29 smithi148 ceph-mon[32281]: pgmap v484: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:30.898 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:25 smithi157 ceph-mon[32703]: pgmap v482: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:30.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:27 smithi157 ceph-mon[32703]: pgmap v483: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:30.948 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:29 smithi157 ceph-mon[32703]: pgmap v484: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:31.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.066+0000 7f37f4c00640 1 -- 172.21.15.131:0/1512095577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 msgr2=0x7f37f01113f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:31.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.066+0000 7f37f4c00640 1 --2- 172.21.15.131:0/1512095577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 0x7f37f01113f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009ca0 tx=0x7f37e802a8c0 comp rx=0 tx=0).stop 2024-04-26T20:12:31.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.067+0000 7f37f4c00640 1 -- 172.21.15.131:0/1512095577 shutdown_connections 2024-04-26T20:12:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.067+0000 7f37f4c00640 1 --2- 172.21.15.131:0/1512095577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 0x7f37f01113f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:31.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.067+0000 7f37f4c00640 1 --2- 172.21.15.131:0/1512095577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f010a1a0 0x7f37f010a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:31.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.067+0000 7f37f4c00640 1 --2- 172.21.15.131:0/1512095577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 0x7f37f01093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:31.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.067+0000 7f37f4c00640 1 -- 172.21.15.131:0/1512095577 >> 172.21.15.131:0/1512095577 conn(0x7f37f0077f40 msgr2=0x7f37f007a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:31.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.067+0000 7f37f4c00640 1 -- 172.21.15.131:0/1512095577 shutdown_connections 2024-04-26T20:12:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.067+0000 7f37f4c00640 1 -- 172.21.15.131:0/1512095577 wait complete. 2024-04-26T20:12:31.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 Processor -- start 2024-04-26T20:12:31.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 -- start start 2024-04-26T20:12:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 0x7f37f019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:31.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f010a1a0 0x7f37f019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 0x7f37f01a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:31.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0114af0 con 0x7f37f010a1a0 2024-04-26T20:12:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0114970 con 0x7f37f0108fb0 2024-04-26T20:12:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37f4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0114c70 con 0x7f37f010ab60 2024-04-26T20:12:31.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37ed400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f010a1a0 0x7f37f019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.068+0000 7f37ede00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 0x7f37f019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 0x7f37f019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:58178/0 (socket says 172.21.15.131:58178) 2024-04-26T20:12:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 -- 172.21.15.131:0/820239355 learned_addr learned my addr 172.21.15.131:0/820239355 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:31.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ee800640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 0x7f37f01a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:31.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 -- 172.21.15.131:0/820239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 msgr2=0x7f37f01a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:31.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 0x7f37f01a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:31.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 -- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f010a1a0 msgr2=0x7f37f019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:31.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f010a1a0 0x7f37f019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:31.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 -- 172.21.15.131:0/820239355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f01a6d90 con 0x7f37f0108fb0 2024-04-26T20:12:31.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ed400640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f010a1a0 0x7f37f019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:31.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.069+0000 7f37ede00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 0x7f37f019f170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37e000e9c0 tx=0x7f37e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.070+0000 7f37e6a00640 1 -- 172.21.15.131:0/820239355 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e000cd80 con 0x7f37f0108fb0 2024-04-26T20:12:31.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.070+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f01a7080 con 0x7f37f0108fb0 2024-04-26T20:12:31.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.070+0000 7f37e6a00640 1 -- 172.21.15.131:0/820239355 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e00035f0 con 0x7f37f0108fb0 2024-04-26T20:12:31.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.070+0000 7f37e6a00640 1 -- 172.21.15.131:0/820239355 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0010690 con 0x7f37f0108fb0 2024-04-26T20:12:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.070+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f01a7580 con 0x7f37f0108fb0 2024-04-26T20:12:31.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.073+0000 7f37f79d5740 1 Processor -- start 2024-04-26T20:12:31.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.073+0000 7f37f79d5740 1 -- start start 2024-04-26T20:12:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.073+0000 7f37e6a00640 1 -- 172.21.15.131:0/820239355 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f37e0004000 con 0x7f37f0108fb0 2024-04-26T20:12:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c882260f0 0x560c88226510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:31.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c88226ae0 0x560c8822b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:31.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c8822e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37f79d5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c8818a600 con 0x560c88226ae0 2024-04-26T20:12:31.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37f79d5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560c88127100 con 0x560c8822bc50 2024-04-26T20:12:31.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37f79d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c881865c0 con 0x560c882260f0 2024-04-26T20:12:31.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c88226ae0 0x560c8822b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c882260f0 0x560c88226510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c882260f0 0x560c88226510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44916/0 (socket says 172.21.15.131:44916) 2024-04-26T20:12:31.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ed400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c8822e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:31.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 -- 172.21.15.131:0/4052052898 learned_addr learned my addr 172.21.15.131:0/4052052898 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:31.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 -- 172.21.15.131:0/4052052898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 msgr2=0x560c8822e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:32.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c8822e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 -- 172.21.15.131:0/4052052898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c88226ae0 msgr2=0x560c8822b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:32.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c88226ae0 0x560c8822b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ee800640 1 -- 172.21.15.131:0/4052052898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c87e9c2d0 con 0x560c882260f0 2024-04-26T20:12:32.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.074+0000 7f37ed400640 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c8822e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:32.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.075+0000 7f37ee800640 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c882260f0 0x560c88226510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009ca0 tx=0x7f37e8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21e516e706f179d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:32.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.075+0000 7f37db000640 1 -- 172.21.15.131:0/4052052898 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8031650 con 0x560c882260f0 2024-04-26T20:12:32.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.075+0000 7f37db000640 1 -- 172.21.15.131:0/4052052898 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e8036030 con 0x560c882260f0 2024-04-26T20:12:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.075+0000 7f37db000640 1 -- 172.21.15.131:0/4052052898 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8004790 con 0x560c882260f0 2024-04-26T20:12:32.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.075+0000 7f37e6a00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.075+0000 7f37e6a00640 1 -- 172.21.15.131:0/820239355 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f37e00d0ed0 con 0x7f37f0108fb0 2024-04-26T20:12:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37ede00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:32.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 -- 172.21.15.131:0/4052052898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c882260f0 msgr2=0x560c88226510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:32.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c882260f0 0x560c88226510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009ca0 tx=0x7f37e8005ef0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 -- 172.21.15.131:0/4052052898 shutdown_connections 2024-04-26T20:12:32.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c8822e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c88226ae0 0x560c8822b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 --2- 172.21.15.131:0/4052052898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c882260f0 0x560c88226510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 -- 172.21.15.131:0/4052052898 >> 172.21.15.131:0/4052052898 conn(0x560c881bad10 msgr2=0x560c88221b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:32.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 -- 172.21.15.131:0/4052052898 shutdown_connections 2024-04-26T20:12:32.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37f79d5740 1 -- 172.21.15.131:0/4052052898 wait complete. 2024-04-26T20:12:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.076+0000 7f37ede00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f37d40a3c40 0x7f37d40a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x560c88226ee0 tx=0x7f37e00d4980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:32.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 -- start start 2024-04-26T20:12:32.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 Processor -- start 2024-04-26T20:12:32.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c882303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:32.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c88230930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:32.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88237970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:32.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c8822ea20 con 0x560c882260f0 2024-04-26T20:12:32.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560c88111f70 con 0x560c88226ae0 2024-04-26T20:12:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c8822eba0 con 0x560c8822bc50 2024-04-26T20:12:32.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37f79d5740 10 client.0 _ll_register_callbacks cb 0x560c88180480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:12:32.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.077+0000 7f37ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c882303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:32.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c882303f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56036/0 (socket says 172.21.15.131:56036) 2024-04-26T20:12:32.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ee800640 1 -- 172.21.15.131:0/395921604 learned_addr learned my addr 172.21.15.131:0/395921604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ed400640 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c88230930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ee800640 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88237970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:32.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ed400640 1 -- 172.21.15.131:0/395921604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 msgr2=0x560c88237970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ed400640 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88237970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ed400640 1 -- 172.21.15.131:0/395921604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 msgr2=0x560c882303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:32.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ed400640 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c882303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:32.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ed400640 1 -- 172.21.15.131:0/395921604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c87f666b0 con 0x560c88226ae0 2024-04-26T20:12:32.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37ed400640 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c88230930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37f01a0680 tx=0x7f37dc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe31830cd06631d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:32.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.078+0000 7f37cf400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:12:32.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.079+0000 7f37cf400640 1 -- 172.21.15.131:0/395921604 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0120d0 con 0x560c88226ae0 2024-04-26T20:12:32.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.079+0000 7f37cf400640 1 -- 172.21.15.131:0/395921604 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f37dc004030 con 0x560c88226ae0 2024-04-26T20:12:32.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.079+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e8036030 con 0x560c88226ae0 2024-04-26T20:12:33.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.079+0000 7f37cf400640 1 -- 172.21.15.131:0/395921604 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37dc025050 con 0x560c88226ae0 2024-04-26T20:12:33.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.079+0000 7f37f79d5740 10 client.25358 fetch_fsmap learned FSMap version 26 2024-04-26T20:12:33.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.079+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c8823a9c0 con 0x560c88226ae0 2024-04-26T20:12:33.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.079+0000 7f37cf400640 1 -- 172.21.15.131:0/395921604 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e8036030 con 0x560c88226ae0 2024-04-26T20:12:33.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37cf400640 1 -- 172.21.15.131:0/395921604 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f37dc01c460 con 0x560c88226ae0 2024-04-26T20:12:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37f79d5740 10 client.25358 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:12:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37f79d5740 10 client.25358 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:12:33.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37f79d5740 20 client.25358 populate_metadata read hostname 'smithi131' 2024-04-26T20:12:33.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37f79d5740 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c8823cfc0 0x560c8823e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:33.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37f79d5740 4 client.25358 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:12:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c88198210 con 0x560c8823cfc0 2024-04-26T20:12:33.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.080+0000 7f37ee800640 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c8823cfc0 0x560c8823e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:33.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.081+0000 7f37ee800640 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c8823cfc0 0x560c8823e430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28f08468e97e059 server_cookie=f27b7b1f47b68078 in_seq=0 out_seq=0 2024-04-26T20:12:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.081+0000 7f37cf400640 10 client.25358 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:12:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.081+0000 7f37cf400640 1 -- 172.21.15.131:0/395921604 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c88198210 con 0x560c8823cfc0 2024-04-26T20:12:33.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.081+0000 7f37cf400640 10 client.25358 handle_command_reply: tid=0 2024-04-26T20:12:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.081+0000 7f37f79d5740 1 client.25358 shutdown 2024-04-26T20:12:33.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 msgr2=0x560c88230930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:33.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c88230930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37f01a0680 tx=0x7f37dc00f790 comp rx=0 tx=0).stop 2024-04-26T20:12:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 shutdown_connections 2024-04-26T20:12:33.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c8823cfc0 0x560c8823e430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88237970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:33.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c88230930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:33.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37cf400640 0 client.25358 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:12:33.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 --2- 172.21.15.131:0/395921604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c882303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:33.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 >> 172.21.15.131:0/395921604 conn(0x560c881bad10 msgr2=0x560c88220eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:33.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 shutdown_connections 2024-04-26T20:12:33.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 1 -- 172.21.15.131:0/395921604 wait complete. 2024-04-26T20:12:33.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.082+0000 7f37f79d5740 20 client.25358 trim_cache size 0 max 16384 2024-04-26T20:12:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 Processor -- start 2024-04-26T20:12:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 -- start start 2024-04-26T20:12:33.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88227fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:33.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c88226ae0 0x560c88237930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:33.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c88228520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:33.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c8823c770 con 0x560c882260f0 2024-04-26T20:12:33.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560c8823ce40 con 0x560c8822bc50 2024-04-26T20:12:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.102+0000 7f37f79d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c8823d510 con 0x560c88226ae0 2024-04-26T20:12:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88227fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88227fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56044/0 (socket says 172.21.15.131:56044) 2024-04-26T20:12:33.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 -- 172.21.15.131:0/1333077159 learned_addr learned my addr 172.21.15.131:0/1333077159 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:33.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ed400640 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c88226ae0 0x560c88237930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 -- 172.21.15.131:0/1333077159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c88226ae0 msgr2=0x560c88237930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:33.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c88226ae0 0x560c88237930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:33.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 -- 172.21.15.131:0/1333077159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 msgr2=0x560c88228520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c88228520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:33.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 -- 172.21.15.131:0/1333077159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8822b280 con 0x560c882260f0 2024-04-26T20:12:33.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.103+0000 7f37ee800640 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88227fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f37e802ade0 tx=0x7f37e802f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eae5664b49b19bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:33.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37da600640 1 -- 172.21.15.131:0/1333077159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8058070 con 0x560c882260f0 2024-04-26T20:12:33.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37da600640 1 -- 172.21.15.131:0/1333077159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e8036070 con 0x560c882260f0 2024-04-26T20:12:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37da600640 1 -- 172.21.15.131:0/1333077159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e802f660 con 0x560c882260f0 2024-04-26T20:12:34.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37f79d5740 1 -- 172.21.15.131:0/1333077159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 msgr2=0x560c88227fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:34.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37f79d5740 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88227fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f37e802ade0 tx=0x7f37e802f310 comp rx=0 tx=0).stop 2024-04-26T20:12:34.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37f79d5740 1 -- 172.21.15.131:0/1333077159 shutdown_connections 2024-04-26T20:12:34.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37f79d5740 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c8822bc50 0x560c88228520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:34.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37f79d5740 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c88226ae0 0x560c88237930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:34.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37f79d5740 1 --2- 172.21.15.131:0/1333077159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88227fe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:34.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.104+0000 7f37f79d5740 1 -- 172.21.15.131:0/1333077159 >> 172.21.15.131:0/1333077159 conn(0x560c881bad10 msgr2=0x560c882215f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:34.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.105+0000 7f37f79d5740 1 -- 172.21.15.131:0/1333077159 shutdown_connections 2024-04-26T20:12:34.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.105+0000 7f37f79d5740 1 -- 172.21.15.131:0/1333077159 wait complete. 2024-04-26T20:12:34.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.105+0000 7f37f79d5740 1 -- start start 2024-04-26T20:12:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.105+0000 7f37f79d5740 1 Processor -- start 2024-04-26T20:12:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88233280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c882337c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37f79d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88236770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37f79d5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c8823e2b0 con 0x560c882260f0 2024-04-26T20:12:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37f79d5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560c8823dbe0 con 0x560c88226ae0 2024-04-26T20:12:34.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37f79d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c8823e980 con 0x560c8822bc50 2024-04-26T20:12:34.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37f79d5740 10 client.0 _ll_register_callbacks cb 0x7f37dc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:12:34.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88233280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:34.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c882337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:34.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c882337c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:58204/0 (socket says 172.21.15.131:58204) 2024-04-26T20:12:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88233280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56056/0 (socket says 172.21.15.131:56056) 2024-04-26T20:12:34.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 -- 172.21.15.131:0/2035379168 learned_addr learned my addr 172.21.15.131:0/2035379168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:34.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ee800640 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88236770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:34.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 -- 172.21.15.131:0/2035379168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 msgr2=0x560c88236770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:34.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88236770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 -- 172.21.15.131:0/2035379168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 msgr2=0x560c88233280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:12:34.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88233280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:34.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ed400640 1 -- 172.21.15.131:0/2035379168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c88236d60 con 0x560c88226ae0 2024-04-26T20:12:34.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.106+0000 7f37ee800640 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88233280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37ed400640 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c882337c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f37dc01c880 tx=0x7f37dc012300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990c7e9aa2685257 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:34.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37cf400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:12:34.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37cf400640 1 -- 172.21.15.131:0/2035379168 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc02e060 con 0x560c88226ae0 2024-04-26T20:12:34.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37cf400640 1 -- 172.21.15.131:0/2035379168 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f37dc018040 con 0x560c88226ae0 2024-04-26T20:12:34.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e8036070 con 0x560c88226ae0 2024-04-26T20:12:34.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37cf400640 1 -- 172.21.15.131:0/2035379168 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37dc035050 con 0x560c88226ae0 2024-04-26T20:12:34.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37f79d5740 10 client.25361 fetch_fsmap learned FSMap version 26 2024-04-26T20:12:34.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.107+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c882a3190 con 0x560c88226ae0 2024-04-26T20:12:34.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37cf400640 1 -- 172.21.15.131:0/2035379168 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e8036070 con 0x560c88226ae0 2024-04-26T20:12:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37cf400640 1 -- 172.21.15.131:0/2035379168 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f37dc005460 con 0x560c88226ae0 2024-04-26T20:12:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37f79d5740 10 client.25361 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:12:34.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37f79d5740 10 client.25361 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:12:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37f79d5740 20 client.25361 populate_metadata read hostname 'smithi131' 2024-04-26T20:12:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37f79d5740 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c882a7d80 0x560c882a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37f79d5740 4 client.25361 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:12:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "status"}) -- 0x560c88192080 con 0x560c882a7d80 2024-04-26T20:12:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.108+0000 7f37ee800640 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c882a7d80 0x560c882a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:35.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37ee800640 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c882a7d80 0x560c882a6a90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62fd967261754d7 server_cookie=a013ceb068808043 in_seq=0 out_seq=0 2024-04-26T20:12:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37cf400640 10 client.25361 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:12:35.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37cf400640 1 -- 172.21.15.131:0/2035379168 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560c88192080 con 0x560c882a7d80 2024-04-26T20:12:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37cf400640 10 client.25361 handle_command_reply: tid=0 2024-04-26T20:12:35.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 client.25361 shutdown 2024-04-26T20:12:35.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 msgr2=0x560c882337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:35.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c882337c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f37dc01c880 tx=0x7f37dc012300 comp rx=0 tx=0).stop 2024-04-26T20:12:35.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 shutdown_connections 2024-04-26T20:12:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560c882a7d80 0x560c882a6a90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:35.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8822bc50 0x560c88236770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c88226ae0 0x560c882337c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 --2- 172.21.15.131:0/2035379168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c882260f0 0x560c88233280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:35.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37cf400640 0 client.25361 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:12:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 >> 172.21.15.131:0/2035379168 conn(0x560c881bad10 msgr2=0x560c882215f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:35.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 shutdown_connections 2024-04-26T20:12:35.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.109+0000 7f37f79d5740 1 -- 172.21.15.131:0/2035379168 wait complete. 2024-04-26T20:12:35.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f79d5740 20 client.25361 trim_cache size 0 max 16384 2024-04-26T20:12:35.471 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:12:35.478 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "2027e7ca-0407-11ef-bc93-c7b262605968", 2024-04-26T20:12:35.486 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-04-26T20:12:35.498 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24575, 2024-04-26T20:12:35.533 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-26T20:12:35.539 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-26T20:12:35.547 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-26T20:12:35.603 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 296.38522285800002, 2024-04-26T20:12:35.609 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-04-26T20:12:35.671 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 97, 2024-04-26T20:12:35.737 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T20:12:35.743 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 318.96488671600002 2024-04-26T20:12:35.808 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:12:35.815 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:31 smithi148 ceph-mon[32281]: pgmap v485: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:35.821 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:33 smithi148 ceph-mon[32281]: pgmap v486: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:35.828 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:34 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:12:35.834 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:35 smithi148 ceph-mon[32281]: pgmap v487: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:35.840 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:31 smithi157 ceph-mon[32703]: pgmap v485: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:35.896 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:33 smithi157 ceph-mon[32703]: pgmap v486: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:35.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:34 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:12:35.952 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:35 smithi157 ceph-mon[32703]: pgmap v487: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:35.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f37d40a3c40 msgr2=0x7f37d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:36.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f37d40a3c40 0x7f37d40a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x560c88226ee0 tx=0x7f37e00d4980 comp rx=0 tx=0).stop 2024-04-26T20:12:36.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 msgr2=0x7f37f019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:36.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 0x7f37f019f170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37e000e9c0 tx=0x7f37e000ee80 comp rx=0 tx=0).stop 2024-04-26T20:12:36.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 shutdown_connections 2024-04-26T20:12:36.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f010ab60 0x7f37f01a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:36.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f010a1a0 0x7f37f019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 --2- 172.21.15.131:0/820239355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f0108fb0 0x7f37f019f170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 >> 172.21.15.131:0/820239355 conn(0x7f37f0077f40 msgr2=0x7f37f010d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 shutdown_connections 2024-04-26T20:12:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:31.110+0000 7f37f4c00640 1 -- 172.21.15.131:0/820239355 wait complete. 2024-04-26T20:12:36.235 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T20:12:36.252 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T20:12:36.258 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T20:12:36.305 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T20:12:36.353 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T20:12:36.371 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T20:12:36.403 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T20:12:36.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:36.462 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-26T20:12:36.470 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:31 smithi131 ceph-mon[27919]: pgmap v485: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:36.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:33 smithi131 ceph-mon[27919]: pgmap v486: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:36.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:34 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:12:36.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:35 smithi131 ceph-mon[27919]: pgmap v487: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:36.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:12:36.609 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:12:36.653 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:12:36.658 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:12:36.665 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:12:36.674 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:12:36.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:12:36.691 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:12:36.728 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:12:36.737 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T20:12:36.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:36.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:12:36.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:36.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-04-26T20:12:37.081 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-26T20:12:37.088 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:12:37.094 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:12:37.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T20:12:37.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:37 smithi131 ceph-mon[27919]: pgmap v488: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:37.619 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:12:37.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:37 smithi157 ceph-mon[32703]: pgmap v488: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:37.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:37 smithi148 ceph-mon[32281]: pgmap v488: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:39.543 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:39 smithi131 ceph-mon[27919]: pgmap v489: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:39.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:39 smithi157 ceph-mon[32703]: pgmap v489: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:39.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:39 smithi148 ceph-mon[32281]: pgmap v489: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:12:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 -- 172.21.15.131:0/1531720998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0406bad0 msgr2=0x7fbc041102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:40.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 --2- 172.21.15.131:0/1531720998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0406bad0 0x7fbc041102e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009990 tx=0x7fbbfc009ca0 comp rx=0 tx=0).stop 2024-04-26T20:12:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 -- 172.21.15.131:0/1531720998 shutdown_connections 2024-04-26T20:12:40.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 --2- 172.21.15.131:0/1531720998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0406bad0 0x7fbc041102e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:40.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 --2- 172.21.15.131:0/1531720998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc041094a0 0x7fbc04109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:40.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 --2- 172.21.15.131:0/1531720998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc0406b130 0x7fbc0406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 -- 172.21.15.131:0/1531720998 >> 172.21.15.131:0/1531720998 conn(0x7fbc040792d0 msgr2=0x7fbc040796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 -- 172.21.15.131:0/1531720998 shutdown_connections 2024-04-26T20:12:40.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.045+0000 7fbc0a000700 1 -- 172.21.15.131:0/1531720998 wait complete. 2024-04-26T20:12:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.046+0000 7fbc0a000700 1 Processor -- start 2024-04-26T20:12:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc0a000700 1 -- start start 2024-04-26T20:12:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc0a000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc0406b130 0x7fbc041a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc0a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0406bad0 0x7fbc041a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc0a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041094a0 0x7fbc0419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:40.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc0a000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbc041135a0 con 0x7fbc041094a0 2024-04-26T20:12:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc0a000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc04113420 con 0x7fbc0406b130 2024-04-26T20:12:40.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc0a000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc04113720 con 0x7fbc0406bad0 2024-04-26T20:12:40.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc03e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041094a0 0x7fbc0419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:40.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc03400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc0406b130 0x7fbc041a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:40.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc03e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041094a0 0x7fbc0419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37610/0 (socket says 172.21.15.131:37610) 2024-04-26T20:12:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbc03e00700 1 -- 172.21.15.131:0/3715643135 learned_addr learned my addr 172.21.15.131:0/3715643135 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:40.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.047+0000 7fbbfaa00700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0406bad0 0x7fbc041a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:40.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc03400700 1 -- 172.21.15.131:0/3715643135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0406bad0 msgr2=0x7fbc041a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:40.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc03400700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0406bad0 0x7fbc041a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc03400700 1 -- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041094a0 msgr2=0x7fbc0419bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc03400700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041094a0 0x7fbc0419bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc03400700 1 -- 172.21.15.131:0/3715643135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0419c2b0 con 0x7fbc0406b130 2024-04-26T20:12:40.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc03e00700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041094a0 0x7fbc0419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:40.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc03400700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc0406b130 0x7fbc041a1940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400ef10 tx=0x7fbbf400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc00c00700 1 -- 172.21.15.131:0/3715643135 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400c9c0 con 0x7fbc0406b130 2024-04-26T20:12:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0419c540 con 0x7fbc0406b130 2024-04-26T20:12:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc00c00700 1 -- 172.21.15.131:0/3715643135 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf4019070 con 0x7fbc0406b130 2024-04-26T20:12:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.048+0000 7fbc00c00700 1 -- 172.21.15.131:0/3715643135 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4014480 con 0x7fbc0406b130 2024-04-26T20:12:40.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.049+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc041a8810 con 0x7fbc0406b130 2024-04-26T20:12:40.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.050+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0406a110 con 0x7fbc0406b130 2024-04-26T20:12:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.052+0000 7fbc00c00700 1 -- 172.21.15.131:0/3715643135 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbbf4012050 con 0x7fbc0406b130 2024-04-26T20:12:40.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.053+0000 7fbc00c00700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbe80a3d50 0x7fbbe80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.053+0000 7fbc00c00700 1 -- 172.21.15.131:0/3715643135 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbbf40d0f30 con 0x7fbc0406b130 2024-04-26T20:12:41.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.053+0000 7fbbfaa00700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbe80a3d50 0x7fbbe80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:41.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.054+0000 7fbbfaa00700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbe80a3d50 0x7fbbe80a6210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbc04079940 tx=0x7fbbec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:41.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.055+0000 7fbc00c00700 1 -- 172.21.15.131:0/3715643135 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbbf4100060 con 0x7fbc0406b130 2024-04-26T20:12:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.332+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbc0402e490 con 0x7fbc0406b130 2024-04-26T20:12:41.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.333+0000 7fbc00c00700 1 -- 172.21.15.131:0/3715643135 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fbbf40bbe20 con 0x7fbc0406b130 2024-04-26T20:12:41.112 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:12:41.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbe80a3d50 msgr2=0x7fbbe80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:41.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbe80a3d50 0x7fbbe80a6210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbc04079940 tx=0x7fbbec009260 comp rx=0 tx=0).stop 2024-04-26T20:12:41.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc0406b130 msgr2=0x7fbc041a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:41.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc0406b130 0x7fbc041a1940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400ef10 tx=0x7fbbf400c3a0 comp rx=0 tx=0).stop 2024-04-26T20:12:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 shutdown_connections 2024-04-26T20:12:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041094a0 0x7fbc0419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbe80a3d50 0x7fbbe80a6210 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:41.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc0406b130 0x7fbc041a1940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:41.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 --2- 172.21.15.131:0/3715643135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0406bad0 0x7fbc041a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.335+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 >> 172.21.15.131:0/3715643135 conn(0x7fbc040792d0 msgr2=0x7fbc0410e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:41.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.336+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 shutdown_connections 2024-04-26T20:12:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:40.336+0000 7fbc0a000700 1 -- 172.21.15.131:0/3715643135 wait complete. 2024-04-26T20:12:41.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:40 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3715643135' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T20:12:41.408 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:40 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3715643135' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T20:12:41.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:40 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3715643135' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T20:12:41.427 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:12:41.435 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:12:41.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-04-26T20:12:41.452 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:12:41.461 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:12:41.467 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:12:41.473 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:12:41.480 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:12:41.543 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:12:41.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:41.661 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-04-26T20:12:41.731 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:12:41.786 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:12:41.792 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:12:41.799 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:12:41.809 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-26T20:12:41.815 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:12:41.821 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:12:41.828 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:12:41.834 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-04-26T20:12:41.841 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:12:41.846 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:12:41.851 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-04-26T20:12:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:12:41.864 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:12:41.870 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:12:41.923 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:12:41.959 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:12:41.980 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41803sec preferred_lft 41803sec 2024-04-26T20:12:42.031 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-04-26T20:12:42.038 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:12:42.045 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:12:42.068 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:12:42.073 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:12:42.082 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:12:42.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-04-26T20:12:42.095 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-04-26T20:12:42.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:12:42.111 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:12:42.121 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:41 smithi148 ceph-mon[32281]: pgmap v490: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:42.130 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:41 smithi157 ceph-mon[32703]: pgmap v490: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:42.139 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:41 smithi131 ceph-mon[27919]: pgmap v490: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:12:42.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:12:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:42.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:42.408 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:12:42.540 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-04-26T20:12:42.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:42.558 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-04-26T20:12:42.652 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-04-26T20:12:42.659 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:12:42.666 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:12:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:12:42.717 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:12:42.725 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:12:42.734 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:12:42.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:12:42.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:12:42.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:12:42.765 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:12:42.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:12:42.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:42.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:43.044 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-26T20:12:43.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:43.113 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-04-26T20:12:43.192 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:12:43.236 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:12:43.243 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:12:43.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T20:12:43.255 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:12:43.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:12:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:43.456 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T20:12:43.463 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:12:43.469 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:12:43.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T20:12:43.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:12:43.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T20:12:43.652 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T20:12:43.716 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T20:12:43.724 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:12:43.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:43 smithi157 ceph-mon[32703]: pgmap v491: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:43.762 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:43 smithi131 ceph-mon[27919]: pgmap v491: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:43.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:43 smithi148 ceph-mon[32281]: pgmap v491: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:43.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:12:44.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:44.036 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:12:44.044 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-26T20:12:44.054 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-04-26T20:12:44.063 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:12:44.071 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-26T20:12:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:12:44.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:44.290 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:12:44.319 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:12:44.384 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:12:44.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:12:44.413 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:12:44.535 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T20:12:44.542 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T20:12:44.548 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/a29373f3-2ca2-48d2-9585-6ab9ff653e9e 2024-04-26T20:12:44.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:44.561 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/a29373f3-2ca2-48d2-9585-6ab9ff653e9e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-26T20:12:44.694 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-26T20:12:44.718 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-26T20:12:45.468 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-26T20:12:45.474 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-26T20:12:45.483 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.148:3300,172.21.15.157:3300" 2024-04-26T20:12:45.490 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@2027e7ca-0407-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/a29373f3-2ca2-48d2-9585-6ab9ff653e9e 2024-04-26T20:12:45.499 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.131:3300/172.21.15.148:3300/172.21.15.157:3300" will pass to kernel 2024-04-26T20:12:45.613 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:12:45.660 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:12:45.668 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:12:45.756 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:12:45.837 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:12:45.872 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:12:45.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:45.920 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:12:45.927 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:45 smithi148 ceph-mon[32281]: pgmap v492: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:45.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:45 smithi157 ceph-mon[32703]: pgmap v492: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:45.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:45 smithi131 ceph-mon[27919]: pgmap v492: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:12:46.041 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:12:46.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:12:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:46.235 INFO:teuthology.orchestra.run.smithi131.stdout:client25370 2024-04-26T20:12:46.241 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:12:46.250 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:46.363 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:12:46.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:12:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:46.654 INFO:teuthology.orchestra.run.smithi131.stdout:client25370 2024-04-26T20:12:46.664 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:12:46.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:12:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:46.819 INFO:teuthology.orchestra.run.smithi131.stdout:client25370 2024-04-26T20:12:46.827 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2027e7ca-0407-11ef-bc93-c7b262605968.client25370/status') 2024-04-26T20:12:46.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:12:46.985 INFO:teuthology.orchestra.run.smithi131.stdout:instance: client.25370 (3)172.21.15.131:0/43888593 2024-04-26T20:12:47.026 INFO:teuthology.orchestra.run.smithi131.stdout:blocklisted: false 2024-04-26T20:12:47.040 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records in 2024-04-26T20:12:47.046 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records out 2024-04-26T20:12:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:70 bytes copied, 7.925e-05 s, 883 kB/s 2024-04-26T20:12:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:12:47.069 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T20:12:47.165 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T20:12:47.178 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:12:47.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-26T20:12:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.437+0000 7f4a73e00640 1 -- 172.21.15.131:0/3427132510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74108510 msgr2=0x7f4a7410a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:47.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.437+0000 7f4a73e00640 1 --2- 172.21.15.131:0/3427132510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74108510 0x7f4a7410a900 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ca0 tx=0x7f4a6c02a8c0 comp rx=0 tx=0).stop 2024-04-26T20:12:47.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.437+0000 7f4a73e00640 1 -- 172.21.15.131:0/3427132510 shutdown_connections 2024-04-26T20:12:47.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.437+0000 7f4a73e00640 1 --2- 172.21.15.131:0/3427132510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74108510 0x7f4a7410a900 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:47.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.437+0000 7f4a73e00640 1 --2- 172.21.15.131:0/3427132510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74105be0 0x7f4a74107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.437+0000 7f4a73e00640 1 --2- 172.21.15.131:0/3427132510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a741032b0 0x7f4a741056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:47.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.437+0000 7f4a73e00640 1 -- 172.21.15.131:0/3427132510 >> 172.21.15.131:0/3427132510 conn(0x7f4a740fcc20 msgr2=0x7f4a740ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.438+0000 7f4a73e00640 1 -- 172.21.15.131:0/3427132510 shutdown_connections 2024-04-26T20:12:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.438+0000 7f4a73e00640 1 -- 172.21.15.131:0/3427132510 wait complete. 2024-04-26T20:12:47.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 Processor -- start 2024-04-26T20:12:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 -- start start 2024-04-26T20:12:47.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a741032b0 0x7f4a7419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74105be0 0x7f4a7419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74108510 0x7f4a741a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:47.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410e350 con 0x7f4a74105be0 2024-04-26T20:12:47.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410e1d0 con 0x7f4a74108510 2024-04-26T20:12:47.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a73e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410e4d0 con 0x7f4a741032b0 2024-04-26T20:12:47.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a72000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74105be0 0x7f4a7419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:47.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a72000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74105be0 0x7f4a7419b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50068/0 (socket says 172.21.15.131:50068) 2024-04-26T20:12:47.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a741032b0 0x7f4a7419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:47.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.439+0000 7f4a72000640 1 -- 172.21.15.131:0/288591055 learned_addr learned my addr 172.21.15.131:0/288591055 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a73400640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74108510 0x7f4a741a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:47.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a72000640 1 -- 172.21.15.131:0/288591055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a741032b0 msgr2=0x7f4a7419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a72000640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a741032b0 0x7f4a7419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:48.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a72000640 1 -- 172.21.15.131:0/288591055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74108510 msgr2=0x7f4a741a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a72000640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74108510 0x7f4a741a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:48.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a72000640 1 -- 172.21.15.131:0/288591055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a741a2800 con 0x7f4a74105be0 2024-04-26T20:12:48.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a72000640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74105be0 0x7f4a7419b120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e9a0 tx=0x7f4a6000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:48.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a6b400640 1 -- 172.21.15.131:0/288591055 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000cd80 con 0x7f4a74105be0 2024-04-26T20:12:48.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a6b400640 1 -- 172.21.15.131:0/288591055 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a600035f0 con 0x7f4a74105be0 2024-04-26T20:12:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a6b400640 1 -- 172.21.15.131:0/288591055 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60010690 con 0x7f4a74105be0 2024-04-26T20:12:48.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.440+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a741a2af0 con 0x7f4a74105be0 2024-04-26T20:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.441+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a3040 con 0x7f4a74105be0 2024-04-26T20:12:48.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.442+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7410a000 con 0x7f4a74105be0 2024-04-26T20:12:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.445+0000 7f4a6b400640 1 -- 172.21.15.131:0/288591055 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4a60004000 con 0x7f4a74105be0 2024-04-26T20:12:48.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.446+0000 7f4a6b400640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a580a3c30 0x7f4a580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.446+0000 7f4a6b400640 1 -- 172.21.15.131:0/288591055 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4a600d0c10 con 0x7f4a74105be0 2024-04-26T20:12:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.446+0000 7f4a72a00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a580a3c30 0x7f4a580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:48.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.447+0000 7f4a72a00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a580a3c30 0x7f4a580a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009bd0 tx=0x7f4a64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:48.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.451+0000 7f4a6b400640 1 -- 172.21.15.131:0/288591055 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4a600042f0 con 0x7f4a74105be0 2024-04-26T20:12:48.442 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:47 smithi148 ceph-mon[32281]: pgmap v493: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:48.448 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:47 smithi157 ceph-mon[32703]: pgmap v493: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:48.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:47.695+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4a7402dcf0 con 0x7f4a74105be0 2024-04-26T20:12:48.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:47 smithi131 ceph-mon[27919]: pgmap v493: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:12:48.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.441+0000 7f4a6b400640 1 -- 172.21.15.131:0/288591055 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f4a6009d040 con 0x7f4a74105be0 2024-04-26T20:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a580a3c30 msgr2=0x7f4a580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a580a3c30 0x7f4a580a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009bd0 tx=0x7f4a64005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:48.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74105be0 msgr2=0x7f4a7419b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:48.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74105be0 0x7f4a7419b120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e9a0 tx=0x7f4a6000ee60 comp rx=0 tx=0).stop 2024-04-26T20:12:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 shutdown_connections 2024-04-26T20:12:48.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74108510 0x7f4a741a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:48.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a74105be0 0x7f4a7419b120 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4a580a3c30 0x7f4a580a60f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 --2- 172.21.15.131:0/288591055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a741032b0 0x7f4a7419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:48.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 >> 172.21.15.131:0/288591055 conn(0x7f4a740fcc20 msgr2=0x7f4a74102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 shutdown_connections 2024-04-26T20:12:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:48.446+0000 7f4a73e00640 1 -- 172.21.15.131:0/288591055 wait complete. 2024-04-26T20:12:48.822 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T20:12:48.840 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-26T20:12:48.901 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:12:48.908 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:48 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/288591055' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T20:12:48.913 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:48 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/288591055' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T20:12:48.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:48 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/288591055' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T20:12:49.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.134+0000 7f248fe00640 1 -- 172.21.15.131:0/2779192888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2490068de0 msgr2=0x7f2490101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:49.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.134+0000 7f248fe00640 1 --2- 172.21.15.131:0/2779192888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2490068de0 0x7f2490101b70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2480009900 tx=0x7f248002a410 comp rx=0 tx=0).stop 2024-04-26T20:12:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.135+0000 7f248fe00640 1 -- 172.21.15.131:0/2779192888 shutdown_connections 2024-04-26T20:12:49.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.135+0000 7f248fe00640 1 --2- 172.21.15.131:0/2779192888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490106db0 0x7f24901091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:49.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.135+0000 7f248fe00640 1 --2- 172.21.15.131:0/2779192888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24901020b0 0x7f2490102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:49.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.135+0000 7f248fe00640 1 --2- 172.21.15.131:0/2779192888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2490068de0 0x7f2490101b70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:49.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.135+0000 7f248fe00640 1 -- 172.21.15.131:0/2779192888 >> 172.21.15.131:0/2779192888 conn(0x7f24900fbef0 msgr2=0x7f24900fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:49.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.135+0000 7f248fe00640 1 -- 172.21.15.131:0/2779192888 shutdown_connections 2024-04-26T20:12:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.135+0000 7f248fe00640 1 -- 172.21.15.131:0/2779192888 wait complete. 2024-04-26T20:12:49.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 Processor -- start 2024-04-26T20:12:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 -- start start 2024-04-26T20:12:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2490068de0 0x7f249019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:49.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24901020b0 0x7f249019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:49.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490106db0 0x7f24901a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f249010e310 con 0x7f24901020b0 2024-04-26T20:12:49.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f249010e190 con 0x7f2490068de0 2024-04-26T20:12:49.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f249010e490 con 0x7f2490106db0 2024-04-26T20:12:49.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.136+0000 7f248e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24901020b0 0x7f249019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24901020b0 0x7f249019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50088/0 (socket says 172.21.15.131:50088) 2024-04-26T20:12:49.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 -- 172.21.15.131:0/3967767778 learned_addr learned my addr 172.21.15.131:0/3967767778 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248ea00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2490068de0 0x7f249019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:49.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248f400640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490106db0 0x7f24901a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:49.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 -- 172.21.15.131:0/3967767778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490106db0 msgr2=0x7f24901a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:49.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490106db0 0x7f24901a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:49.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 -- 172.21.15.131:0/3967767778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2490068de0 msgr2=0x7f249019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:49.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2490068de0 0x7f249019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:49.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 -- 172.21.15.131:0/3967767778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24901a27e0 con 0x7f24901020b0 2024-04-26T20:12:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248f400640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490106db0 0x7f24901a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:12:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.137+0000 7f248e000640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24901020b0 0x7f249019b100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9c0 tx=0x7f247c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:49.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.138+0000 7f2487400640 1 -- 172.21.15.131:0/3967767778 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c00cde0 con 0x7f24901020b0 2024-04-26T20:12:49.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.138+0000 7f2487400640 1 -- 172.21.15.131:0/3967767778 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247c0108e0 con 0x7f24901020b0 2024-04-26T20:12:49.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.138+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24901a2ad0 con 0x7f24901020b0 2024-04-26T20:12:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.138+0000 7f2487400640 1 -- 172.21.15.131:0/3967767778 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c018960 con 0x7f24901020b0 2024-04-26T20:12:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.138+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24901a3070 con 0x7f24901020b0 2024-04-26T20:12:49.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.139+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24901a3240 con 0x7f24901020b0 2024-04-26T20:12:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.142+0000 7f2487400640 1 -- 172.21.15.131:0/3967767778 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f247c01f020 con 0x7f24901020b0 2024-04-26T20:12:50.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.143+0000 7f2487400640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24740a3cc0 0x7f24740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.143+0000 7f2487400640 1 -- 172.21.15.131:0/3967767778 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f247c0d0c60 con 0x7f24901020b0 2024-04-26T20:12:50.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.143+0000 7f248ea00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24740a3cc0 0x7f24740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.144+0000 7f248ea00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24740a3cc0 0x7f24740a6180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2480005ab0 tx=0x7f2480005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:50.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.147+0000 7f2487400640 1 -- 172.21.15.131:0/3967767778 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f247c054060 con 0x7f24901020b0 2024-04-26T20:12:50.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.351+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24901a35d0 con 0x7f24901020b0 2024-04-26T20:12:50.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.354+0000 7f2487400640 1 -- 172.21.15.131:0/3967767778 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7f247c09d090 con 0x7f24901020b0 2024-04-26T20:12:50.260 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:12:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:12:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24740a3cc0 msgr2=0x7f24740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24740a3cc0 0x7f24740a6180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2480005ab0 tx=0x7f2480005a20 comp rx=0 tx=0).stop 2024-04-26T20:12:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24901020b0 msgr2=0x7f249019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24901020b0 0x7f249019b100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9c0 tx=0x7f247c00ee80 comp rx=0 tx=0).stop 2024-04-26T20:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 shutdown_connections 2024-04-26T20:12:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24740a3cc0 0x7f24740a6180 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:50.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490106db0 0x7f24901a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24901020b0 0x7f249019b100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:50.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 --2- 172.21.15.131:0/3967767778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2490068de0 0x7f249019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:50.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 >> 172.21.15.131:0/3967767778 conn(0x7f24900fbef0 msgr2=0x7f24900fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:50.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 shutdown_connections 2024-04-26T20:12:50.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:49.357+0000 7f248fe00640 1 -- 172.21.15.131:0/3967767778 wait complete. 2024-04-26T20:12:50.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:12:50.598 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:12:50.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:49 smithi131 ceph-mon[27919]: pgmap v494: 129 pgs: 129 active+clean; 636 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T20:12:50.629 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:49 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/288591055' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T20:12:50.689 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:49 smithi131 ceph-mon[27919]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:12:50.695 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:49 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3967767778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:50.737 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:49 smithi148 ceph-mon[32281]: pgmap v494: 129 pgs: 129 active+clean; 636 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T20:12:50.744 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:49 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/288591055' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T20:12:50.752 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:49 smithi148 ceph-mon[32281]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:12:50.761 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:49 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3967767778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:50.769 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:49 smithi157 ceph-mon[32703]: pgmap v494: 129 pgs: 129 active+clean; 636 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T20:12:50.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:49 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/288591055' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T20:12:50.784 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:49 smithi157 ceph-mon[32703]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T20:12:50.860 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:49 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3967767778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:50.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 -- 172.21.15.131:0/2816875128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f983810a980 msgr2=0x7f98381112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:50.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 --2- 172.21.15.131:0/2816875128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f983810a980 0x7f98381112a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9830009ca0 tx=0x7f983002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:12:50.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 -- 172.21.15.131:0/2816875128 shutdown_connections 2024-04-26T20:12:50.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 --2- 172.21.15.131:0/2816875128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f983810a980 0x7f98381112a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:51.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 --2- 172.21.15.131:0/2816875128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838109fc0 0x7f983810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:51.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 --2- 172.21.15.131:0/2816875128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838108e70 0x7f9838109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:51.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 -- 172.21.15.131:0/2816875128 >> 172.21.15.131:0/2816875128 conn(0x7f9838077e00 msgr2=0x7f983807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:51.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 -- 172.21.15.131:0/2816875128 shutdown_connections 2024-04-26T20:12:51.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.954+0000 7f9837200640 1 -- 172.21.15.131:0/2816875128 wait complete. 2024-04-26T20:12:51.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.955+0000 7f9837200640 1 Processor -- start 2024-04-26T20:12:51.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.955+0000 7f9837200640 1 -- start start 2024-04-26T20:12:51.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9837200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9838108e70 0x7f983819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:51.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9837200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838109fc0 0x7f983819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:51.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9837200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983810a980 0x7f98381a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:51.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9837200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f98381149d0 con 0x7f9838108e70 2024-04-26T20:12:51.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9837200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9838114850 con 0x7f9838109fc0 2024-04-26T20:12:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9837200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9838114b50 con 0x7f983810a980 2024-04-26T20:12:51.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9835e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9838108e70 0x7f983819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:51.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9836800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983810a980 0x7f98381a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:51.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9836800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983810a980 0x7f98381a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54732/0 (socket says 172.21.15.131:54732) 2024-04-26T20:12:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f982fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838109fc0 0x7f983819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:51.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.956+0000 7f9836800640 1 -- 172.21.15.131:0/3480790197 learned_addr learned my addr 172.21.15.131:0/3480790197 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f9836800640 1 -- 172.21.15.131:0/3480790197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838109fc0 msgr2=0x7f983819f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f9836800640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838109fc0 0x7f983819f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:51.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f9836800640 1 -- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9838108e70 msgr2=0x7f983819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:51.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f9836800640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9838108e70 0x7f983819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:51.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f9836800640 1 -- 172.21.15.131:0/3480790197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98381a6c30 con 0x7f983810a980 2024-04-26T20:12:51.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f982fe00640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838109fc0 0x7f983819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:51.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f9836800640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983810a980 0x7f98381a65c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f98300097b0 tx=0x7f9830031770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:51.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f982ea00640 1 -- 172.21.15.131:0/3480790197 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830003b70 con 0x7f983810a980 2024-04-26T20:12:51.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f982ea00640 1 -- 172.21.15.131:0/3480790197 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9830003d10 con 0x7f983810a980 2024-04-26T20:12:51.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381a6ec0 con 0x7f983810a980 2024-04-26T20:12:51.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.957+0000 7f982ea00640 1 -- 172.21.15.131:0/3480790197 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983002fc10 con 0x7f983810a980 2024-04-26T20:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.958+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a7380 con 0x7f983810a980 2024-04-26T20:12:51.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.959+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9838110120 con 0x7f983810a980 2024-04-26T20:12:51.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.961+0000 7f982ea00640 1 -- 172.21.15.131:0/3480790197 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9830004b40 con 0x7f983810a980 2024-04-26T20:12:51.911 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:51 smithi148 ceph-mon[32281]: pgmap v495: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:12:51.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:51 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3480790197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:51.931 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:51 smithi157 ceph-mon[32703]: pgmap v495: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:12:51.952 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:51 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3480790197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.962+0000 7f982ea00640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f981c0a3c40 0x7f981c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:52.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.962+0000 7f982ea00640 1 -- 172.21.15.131:0/3480790197 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f98300f1f20 con 0x7f983810a980 2024-04-26T20:12:52.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.962+0000 7f9835e00640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f981c0a3c40 0x7f981c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:52.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.963+0000 7f9835e00640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f981c0a3c40 0x7f981c0a6100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98280095f0 tx=0x7f9828005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:50.964+0000 7f982ea00640 1 -- 172.21.15.131:0/3480790197 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f98300be460 con 0x7f983810a980 2024-04-26T20:12:52.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.166+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f983802dad0 con 0x7f983810a980 2024-04-26T20:12:52.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.167+0000 7f982ea00640 1 -- 172.21.15.131:0/3480790197 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f98300bdd60 con 0x7f983810a980 2024-04-26T20:12:52.114 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:12:52.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f981c0a3c40 msgr2=0x7f981c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:52.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f981c0a3c40 0x7f981c0a6100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98280095f0 tx=0x7f9828005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:52.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983810a980 msgr2=0x7f98381a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:52.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983810a980 0x7f98381a65c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f98300097b0 tx=0x7f9830031770 comp rx=0 tx=0).stop 2024-04-26T20:12:52.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 shutdown_connections 2024-04-26T20:12:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f981c0a3c40 0x7f981c0a6100 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:52.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983810a980 0x7f98381a65c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:52.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838109fc0 0x7f983819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 --2- 172.21.15.131:0/3480790197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9838108e70 0x7f983819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:52.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.169+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 >> 172.21.15.131:0/3480790197 conn(0x7f9838077e00 msgr2=0x7f983810d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:52.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.170+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 shutdown_connections 2024-04-26T20:12:52.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:51.170+0000 7f9837200640 1 -- 172.21.15.131:0/3480790197 wait complete. 2024-04-26T20:12:52.600 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:12:52.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:51 smithi131 ceph-mon[27919]: pgmap v495: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:12:52.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:51 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3480790197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:12:52.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 -- 172.21.15.131:0/2657514224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0104a70 msgr2=0x7f7de0104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 --2- 172.21.15.131:0/2657514224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0104a70 0x7f7de0104e90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009be0 tx=0x7f7dd002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:12:52.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 -- 172.21.15.131:0/2657514224 shutdown_connections 2024-04-26T20:12:52.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 --2- 172.21.15.131:0/2657514224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de0106620 0x7f7de010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:52.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 --2- 172.21.15.131:0/2657514224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0105c60 0x7f7de01060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:52.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 --2- 172.21.15.131:0/2657514224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0104a70 0x7f7de0104e90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:53.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 -- 172.21.15.131:0/2657514224 >> 172.21.15.131:0/2657514224 conn(0x7f7de00fffe0 msgr2=0x7f7de0102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 -- 172.21.15.131:0/2657514224 shutdown_connections 2024-04-26T20:12:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.835+0000 7f7de5800640 1 -- 172.21.15.131:0/2657514224 wait complete. 2024-04-26T20:12:53.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.836+0000 7f7de5800640 1 Processor -- start 2024-04-26T20:12:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de5800640 1 -- start start 2024-04-26T20:12:53.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de5800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de0104a70 0x7f7de019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:53.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de5800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0105c60 0x7f7de019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0106620 0x7f7de01a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de5800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0110570 con 0x7f7de0106620 2024-04-26T20:12:53.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de5800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01103f0 con 0x7f7de0105c60 2024-04-26T20:12:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de5800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01106f0 con 0x7f7de0104a70 2024-04-26T20:12:53.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0106620 0x7f7de01a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0106620 0x7f7de01a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50128/0 (socket says 172.21.15.131:50128) 2024-04-26T20:12:53.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.837+0000 7f7de4e00640 1 -- 172.21.15.131:0/277720907 learned_addr learned my addr 172.21.15.131:0/277720907 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:53.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7ddfe00640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de0104a70 0x7f7de019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:53.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7dd7400640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0105c60 0x7f7de019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7de4e00640 1 -- 172.21.15.131:0/277720907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de0104a70 msgr2=0x7f7de019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:53.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7de4e00640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de0104a70 0x7f7de019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:53.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7de4e00640 1 -- 172.21.15.131:0/277720907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0105c60 msgr2=0x7f7de019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:53.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7de4e00640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0105c60 0x7f7de019b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7de4e00640 1 -- 172.21.15.131:0/277720907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de01a28e0 con 0x7f7de0106620 2024-04-26T20:12:53.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7de4e00640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0106620 0x7f7de01a2240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800e970 tx=0x7f7dd800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:53.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.838+0000 7f7ddd600640 1 -- 172.21.15.131:0/277720907 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800ccc0 con 0x7f7de0106620 2024-04-26T20:12:53.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.839+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de01a2bd0 con 0x7f7de0106620 2024-04-26T20:12:53.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.839+0000 7f7ddd600640 1 -- 172.21.15.131:0/277720907 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd800ce60 con 0x7f7de0106620 2024-04-26T20:12:53.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.839+0000 7f7ddd600640 1 -- 172.21.15.131:0/277720907 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8010430 con 0x7f7de0106620 2024-04-26T20:12:53.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.839+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de01a30f0 con 0x7f7de0106620 2024-04-26T20:12:53.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.840+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de010be10 con 0x7f7de0106620 2024-04-26T20:12:53.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.842+0000 7f7ddd600640 1 -- 172.21.15.131:0/277720907 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7dd8004f20 con 0x7f7de0106620 2024-04-26T20:12:53.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.843+0000 7f7ddd600640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7dc40a3c40 0x7f7dc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:53.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.844+0000 7f7ddd600640 1 -- 172.21.15.131:0/277720907 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7dd80d0c40 con 0x7f7de0106620 2024-04-26T20:12:53.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.844+0000 7f7ddfe00640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7dc40a3c40 0x7f7dc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:53.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.844+0000 7f7ddfe00640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7dc40a3c40 0x7f7dc40a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7dd002acb0 tx=0x7f7dd0030970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:53.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:52.848+0000 7f7ddd600640 1 -- 172.21.15.131:0/277720907 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7dd80d7080 con 0x7f7de0106620 2024-04-26T20:12:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.048+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7de002dcd0 con 0x7f7de0106620 2024-04-26T20:12:53.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.049+0000 7f7ddd600640 1 -- 172.21.15.131:0/277720907 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f7dd809d0d0 con 0x7f7de0106620 2024-04-26T20:12:54.009 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:12:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7dc40a3c40 msgr2=0x7f7dc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:54.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7dc40a3c40 0x7f7dc40a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7dd002acb0 tx=0x7f7dd0030970 comp rx=0 tx=0).stop 2024-04-26T20:12:54.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0106620 msgr2=0x7f7de01a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:54.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0106620 0x7f7de01a2240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800e970 tx=0x7f7dd800ed70 comp rx=0 tx=0).stop 2024-04-26T20:12:54.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 shutdown_connections 2024-04-26T20:12:54.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de0106620 0x7f7de01a2240 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de0105c60 0x7f7de019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7dc40a3c40 0x7f7dc40a6100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 --2- 172.21.15.131:0/277720907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de0104a70 0x7f7de019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 >> 172.21.15.131:0/277720907 conn(0x7f7de00fffe0 msgr2=0x7f7de0108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:54.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 shutdown_connections 2024-04-26T20:12:54.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:53.052+0000 7f7de5800640 1 -- 172.21.15.131:0/277720907 wait complete. 2024-04-26T20:12:54.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:12:54.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:53 smithi131 ceph-mon[27919]: pgmap v496: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T20:12:54.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:53 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/277720907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:54.470 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:53 smithi148 ceph-mon[32281]: pgmap v496: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T20:12:54.479 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:53 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/277720907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:54.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:53 smithi157 ceph-mon[32703]: pgmap v496: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T20:12:54.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:53 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/277720907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:12:54.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 -- 172.21.15.131:0/4157065064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788103b50 msgr2=0x7f8788103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:54.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 --2- 172.21.15.131:0/4157065064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788103b50 0x7f8788103fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8770009900 tx=0x7f877002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 -- 172.21.15.131:0/4157065064 shutdown_connections 2024-04-26T20:12:54.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 --2- 172.21.15.131:0/4157065064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8788104510 0x7f878810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 --2- 172.21.15.131:0/4157065064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788103b50 0x7f8788103fd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 --2- 172.21.15.131:0/4157065064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8788102960 0x7f8788102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:54.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 -- 172.21.15.131:0/4157065064 >> 172.21.15.131:0/4157065064 conn(0x7f87880fdef0 msgr2=0x7f8788100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:54.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 -- 172.21.15.131:0/4157065064 shutdown_connections 2024-04-26T20:12:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.686+0000 7f878d600640 1 -- 172.21.15.131:0/4157065064 wait complete. 2024-04-26T20:12:54.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 Processor -- start 2024-04-26T20:12:54.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 -- start start 2024-04-26T20:12:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8788102960 0x7f878819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:54.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8788103b50 0x7f878819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:54.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788104510 0x7f87881a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:55.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f878810e490 con 0x7f8788103b50 2024-04-26T20:12:55.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f878810e310 con 0x7f8788104510 2024-04-26T20:12:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.687+0000 7f878d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f878810e610 con 0x7f8788102960 2024-04-26T20:12:55.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f877e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8788103b50 0x7f878819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:55.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8786a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8788102960 0x7f878819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:55.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8787400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788104510 0x7f87881a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:55.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f877e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8788103b50 0x7f878819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49930/0 (socket says 172.21.15.131:49930) 2024-04-26T20:12:55.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8786a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8788102960 0x7f878819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45540/0 (socket says 172.21.15.131:45540) 2024-04-26T20:12:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f877e000640 1 -- 172.21.15.131:0/2152149179 learned_addr learned my addr 172.21.15.131:0/2152149179 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:55.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8787400640 1 -- 172.21.15.131:0/2152149179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8788102960 msgr2=0x7f878819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:55.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8787400640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8788102960 0x7f878819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:55.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8787400640 1 -- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8788103b50 msgr2=0x7f878819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8787400640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8788103b50 0x7f878819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:55.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8787400640 1 -- 172.21.15.131:0/2152149179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87881a29e0 con 0x7f8788104510 2024-04-26T20:12:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.688+0000 7f8787400640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788104510 0x7f87881a2370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8780009fd0 tx=0x7f878000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:55.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.689+0000 7f877fe00640 1 -- 172.21.15.131:0/2152149179 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878000ce20 con 0x7f8788104510 2024-04-26T20:12:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.689+0000 7f877fe00640 1 -- 172.21.15.131:0/2152149179 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8780005020 con 0x7f8788104510 2024-04-26T20:12:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.689+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87881a2cd0 con 0x7f8788104510 2024-04-26T20:12:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.689+0000 7f877fe00640 1 -- 172.21.15.131:0/2152149179 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780010690 con 0x7f8788104510 2024-04-26T20:12:55.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.689+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87881a31a0 con 0x7f8788104510 2024-04-26T20:12:55.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.690+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87881a3430 con 0x7f8788104510 2024-04-26T20:12:55.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.692+0000 7f877fe00640 1 -- 172.21.15.131:0/2152149179 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f87800054d0 con 0x7f8788104510 2024-04-26T20:12:55.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.693+0000 7f877fe00640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f876c0a3cc0 0x7f876c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:55.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.693+0000 7f877fe00640 1 -- 172.21.15.131:0/2152149179 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f87800d0ff0 con 0x7f8788104510 2024-04-26T20:12:55.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.693+0000 7f8786a00640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f876c0a3cc0 0x7f876c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:55.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.694+0000 7f8786a00640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f876c0a3cc0 0x7f876c0a6180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87881039b0 tx=0x7f8778005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:55.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.698+0000 7f877fe00640 1 -- 172.21.15.131:0/2152149179 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f878009d480 con 0x7f8788104510 2024-04-26T20:12:55.797 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:55 smithi157 ceph-mon[32703]: pgmap v497: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T20:12:55.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:55 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2152149179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:55.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.905+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87881a36a0 con 0x7f8788104510 2024-04-26T20:12:55.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.906+0000 7f877fe00640 1 -- 172.21.15.131:0/2152149179 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f878009cd80 con 0x7f8788104510 2024-04-26T20:12:55.941 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:12:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.908+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f876c0a3cc0 msgr2=0x7f876c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:55.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f876c0a3cc0 0x7f876c0a6180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87881039b0 tx=0x7f8778005f90 comp rx=0 tx=0).stop 2024-04-26T20:12:56.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788104510 msgr2=0x7f87881a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788104510 0x7f87881a2370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8780009fd0 tx=0x7f878000ef10 comp rx=0 tx=0).stop 2024-04-26T20:12:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 shutdown_connections 2024-04-26T20:12:56.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788104510 0x7f87881a2370 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:56.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8788103b50 0x7f878819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f876c0a3cc0 0x7f876c0a6180 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:56.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 --2- 172.21.15.131:0/2152149179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8788102960 0x7f878819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:56.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 >> 172.21.15.131:0/2152149179 conn(0x7f87880fdef0 msgr2=0x7f8788074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:56.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 shutdown_connections 2024-04-26T20:12:56.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:54.909+0000 7f878d600640 1 -- 172.21.15.131:0/2152149179 wait complete. 2024-04-26T20:12:56.174 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:12:56.189 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:55 smithi131 ceph-mon[27919]: pgmap v497: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T20:12:56.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:55 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2152149179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:56.304 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:55 smithi148 ceph-mon[32281]: pgmap v497: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T20:12:56.422 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:55 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2152149179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:12:56.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.520+0000 7f7798000640 1 -- 172.21.15.131:0/2230812717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7790076910 msgr2=0x7f7790074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:56.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.520+0000 7f7798000640 1 --2- 172.21.15.131:0/2230812717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7790076910 0x7f7790074d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7784009900 tx=0x7f778402a480 comp rx=0 tx=0).stop 2024-04-26T20:12:56.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.521+0000 7f7798000640 1 -- 172.21.15.131:0/2230812717 shutdown_connections 2024-04-26T20:12:56.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.521+0000 7f7798000640 1 --2- 172.21.15.131:0/2230812717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f779010a800 0x7f779010cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:56.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.521+0000 7f7798000640 1 --2- 172.21.15.131:0/2230812717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790075330 0x7f7790075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:56.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.521+0000 7f7798000640 1 --2- 172.21.15.131:0/2230812717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7790076910 0x7f7790074d60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:56.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.521+0000 7f7798000640 1 -- 172.21.15.131:0/2230812717 >> 172.21.15.131:0/2230812717 conn(0x7f77900fdee0 msgr2=0x7f7790100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:56.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.521+0000 7f7798000640 1 -- 172.21.15.131:0/2230812717 shutdown_connections 2024-04-26T20:12:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.521+0000 7f7798000640 1 -- 172.21.15.131:0/2230812717 wait complete. 2024-04-26T20:12:56.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 Processor -- start 2024-04-26T20:12:56.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 -- start start 2024-04-26T20:12:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7790075330 0x7f779019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790076910 0x7f779019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:56.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f779010a800 0x7f77901a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:56.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7790110690 con 0x7f779010a800 2024-04-26T20:12:56.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7790110510 con 0x7f7790076910 2024-04-26T20:12:56.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.522+0000 7f7798000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7790110810 con 0x7f7790075330 2024-04-26T20:12:56.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f779010a800 0x7f77901a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:57.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7795600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7790075330 0x7f779019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f779010a800 0x7f77901a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49940/0 (socket says 172.21.15.131:49940) 2024-04-26T20:12:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 -- 172.21.15.131:0/920232622 learned_addr learned my addr 172.21.15.131:0/920232622 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7794c00640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790076910 0x7f779019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:57.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 -- 172.21.15.131:0/920232622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7790075330 msgr2=0x7f779019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:57.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7790075330 0x7f779019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:57.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 -- 172.21.15.131:0/920232622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790076910 msgr2=0x7f779019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790076910 0x7f779019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:57.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 -- 172.21.15.131:0/920232622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77901a2a00 con 0x7f779010a800 2024-04-26T20:12:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7794c00640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790076910 0x7f779019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.523+0000 7f7796000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f779010a800 0x7f77901a2390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f778800eb10 tx=0x7f778800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:57.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.524+0000 7f778e000640 1 -- 172.21.15.131:0/920232622 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788004b20 con 0x7f779010a800 2024-04-26T20:12:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.524+0000 7f778e000640 1 -- 172.21.15.131:0/920232622 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7788004cc0 con 0x7f779010a800 2024-04-26T20:12:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.524+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77901a2cf0 con 0x7f779010a800 2024-04-26T20:12:57.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.524+0000 7f778e000640 1 -- 172.21.15.131:0/920232622 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788010670 con 0x7f779010a800 2024-04-26T20:12:57.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.524+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77901a31e0 con 0x7f779010a800 2024-04-26T20:12:57.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.525+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77901095c0 con 0x7f779010a800 2024-04-26T20:12:57.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.527+0000 7f778e000640 1 -- 172.21.15.131:0/920232622 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7788005020 con 0x7f779010a800 2024-04-26T20:12:57.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.529+0000 7f778e000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77780a3c40 0x7f77780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:57.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.529+0000 7f778e000640 1 -- 172.21.15.131:0/920232622 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f77880d0f00 con 0x7f779010a800 2024-04-26T20:12:57.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.529+0000 7f7795600640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77780a3c40 0x7f77780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:57.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.530+0000 7f7795600640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77780a3c40 0x7f77780a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7784000c00 tx=0x7f77840314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:57.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.533+0000 7f778e000640 1 -- 172.21.15.131:0/920232622 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f778809d330 con 0x7f779010a800 2024-04-26T20:12:57.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:57 smithi157 ceph-mon[32703]: pgmap v498: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T20:12:57.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:57 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/920232622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:57.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.737+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f77900635d0 con 0x7f779010a800 2024-04-26T20:12:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.738+0000 7f778e000640 1 -- 172.21.15.131:0/920232622 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f778809cc30 con 0x7f779010a800 2024-04-26T20:12:57.874 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:12:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77780a3c40 msgr2=0x7f77780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:57.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77780a3c40 0x7f77780a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7784000c00 tx=0x7f77840314a0 comp rx=0 tx=0).stop 2024-04-26T20:12:57.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f779010a800 msgr2=0x7f77901a2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:58.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f779010a800 0x7f77901a2390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f778800eb10 tx=0x7f778800c2d0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 shutdown_connections 2024-04-26T20:12:58.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f779010a800 0x7f77901a2390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790076910 0x7f779019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77780a3c40 0x7f77780a6100 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 --2- 172.21.15.131:0/920232622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7790075330 0x7f779019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 >> 172.21.15.131:0/920232622 conn(0x7f77900fdee0 msgr2=0x7f7790106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:58.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 shutdown_connections 2024-04-26T20:12:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:56.741+0000 7f7798000640 1 -- 172.21.15.131:0/920232622 wait complete. 2024-04-26T20:12:58.347 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:12:58.362 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:57 smithi131 ceph-mon[27919]: pgmap v498: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T20:12:58.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:57 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/920232622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:58.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:57 smithi148 ceph-mon[32281]: pgmap v498: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T20:12:58.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:57 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/920232622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:12:58.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 -- 172.21.15.131:0/1245882394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 msgr2=0x7ff0e810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 --2- 172.21.15.131:0/1245882394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 0x7ff0e810a600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009860 tx=0x7ff0d002a340 comp rx=0 tx=0).stop 2024-04-26T20:12:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 -- 172.21.15.131:0/1245882394 shutdown_connections 2024-04-26T20:12:58.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 --2- 172.21.15.131:0/1245882394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e810ab40 0x7ff0e81113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 --2- 172.21.15.131:0/1245882394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 0x7ff0e810a600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 --2- 172.21.15.131:0/1245882394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e8108f90 0x7ff0e81093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:58.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 -- 172.21.15.131:0/1245882394 >> 172.21.15.131:0/1245882394 conn(0x7ff0e8077f40 msgr2=0x7ff0e807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:12:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 -- 172.21.15.131:0/1245882394 shutdown_connections 2024-04-26T20:12:58.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.598+0000 7ff0ed400640 1 -- 172.21.15.131:0/1245882394 wait complete. 2024-04-26T20:12:58.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 Processor -- start 2024-04-26T20:12:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 -- start start 2024-04-26T20:12:59.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e8108f90 0x7ff0e819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:59.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 0x7ff0e819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:59.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e810ab40 0x7ff0e81a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8114ad0 con 0x7ff0e810ab40 2024-04-26T20:12:59.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8114950 con 0x7ff0e810a180 2024-04-26T20:12:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0ed400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8114c50 con 0x7ff0e8108f90 2024-04-26T20:12:59.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0e7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e810ab40 0x7ff0e81a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:59.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0e7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e810ab40 0x7ff0e81a66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49960/0 (socket says 172.21.15.131:49960) 2024-04-26T20:12:59.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0e7200640 1 -- 172.21.15.131:0/481634823 learned_addr learned my addr 172.21.15.131:0/481634823 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:12:59.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0dde00640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 0x7ff0e819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.600+0000 7ff0e6800640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e8108f90 0x7ff0e819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:59.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0e7200640 1 -- 172.21.15.131:0/481634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e8108f90 msgr2=0x7ff0e819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:59.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0e7200640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e8108f90 0x7ff0e819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:59.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0e7200640 1 -- 172.21.15.131:0/481634823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 msgr2=0x7ff0e819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:59.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0e7200640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 0x7ff0e819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:12:59.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0e7200640 1 -- 172.21.15.131:0/481634823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e81a6d60 con 0x7ff0e810ab40 2024-04-26T20:12:59.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0dde00640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 0x7ff0e819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:12:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0e7200640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e810ab40 0x7ff0e81a66f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000efc0 tx=0x7ff0e000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:59.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.601+0000 7ff0dfe00640 1 -- 172.21.15.131:0/481634823 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0003970 con 0x7ff0e810ab40 2024-04-26T20:12:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.602+0000 7ff0dfe00640 1 -- 172.21.15.131:0/481634823 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e0004020 con 0x7ff0e810ab40 2024-04-26T20:12:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.602+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e81a6ff0 con 0x7ff0e810ab40 2024-04-26T20:12:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.602+0000 7ff0dfe00640 1 -- 172.21.15.131:0/481634823 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000b940 con 0x7ff0e810ab40 2024-04-26T20:12:59.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.602+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81a74c0 con 0x7ff0e810ab40 2024-04-26T20:12:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.603+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e81102d0 con 0x7ff0e810ab40 2024-04-26T20:12:59.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.605+0000 7ff0dfe00640 1 -- 172.21.15.131:0/481634823 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff0e001d020 con 0x7ff0e810ab40 2024-04-26T20:12:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.606+0000 7ff0dfe00640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0cc0a3c40 0x7ff0cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:12:59.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.607+0000 7ff0dfe00640 1 -- 172.21.15.131:0/481634823 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff0e00d0e70 con 0x7ff0e810ab40 2024-04-26T20:12:59.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.607+0000 7ff0e6800640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0cc0a3c40 0x7ff0cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:12:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.607+0000 7ff0e6800640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0cc0a3c40 0x7ff0cc0a6100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8109fe0 tx=0x7ff0d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:12:59.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.610+0000 7ff0dfe00640 1 -- 172.21.15.131:0/481634823 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff0e009d2a0 con 0x7ff0e810ab40 2024-04-26T20:12:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.814+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0e8063530 con 0x7ff0e810ab40 2024-04-26T20:12:59.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.816+0000 7ff0dfe00640 1 -- 172.21.15.131:0/481634823 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7ff0e009cba0 con 0x7ff0e810ab40 2024-04-26T20:12:59.906 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:59 smithi148 ceph-mon[32281]: pgmap v499: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:12:59.914 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:12:59 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/481634823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:59.959 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:59 smithi157 ceph-mon[32703]: pgmap v499: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:12:59.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:12:59 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/481634823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:12:59.974 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:12:59.981 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:12:59.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0cc0a3c40 msgr2=0x7ff0cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:12:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0cc0a3c40 0x7ff0cc0a6100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8109fe0 tx=0x7ff0d8005f90 comp rx=0 tx=0).stop 2024-04-26T20:13:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e810ab40 msgr2=0x7ff0e81a66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e810ab40 0x7ff0e81a66f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000efc0 tx=0x7ff0e000c800 comp rx=0 tx=0).stop 2024-04-26T20:13:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 shutdown_connections 2024-04-26T20:13:00.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e810ab40 0x7ff0e81a66f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:00.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e810a180 0x7ff0e819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:00.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0cc0a3c40 0x7ff0cc0a6100 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:00.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 --2- 172.21.15.131:0/481634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e8108f90 0x7ff0e819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.819+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 >> 172.21.15.131:0/481634823 conn(0x7ff0e8077f40 msgr2=0x7ff0e810d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.820+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 shutdown_connections 2024-04-26T20:13:00.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:12:58.820+0000 7ff0ed400640 1 -- 172.21.15.131:0/481634823 wait complete. 2024-04-26T20:13:00.292 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:13:00.395 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:13:00.408 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:59 smithi131 ceph-mon[27919]: pgmap v499: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T20:13:00.421 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:12:59 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/481634823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:13:00.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 -- 172.21.15.131:0/2331819091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 msgr2=0x7fc5a0075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:00.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2331819091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 0x7fc5a0075810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009860 tx=0x7fc58c02a340 comp rx=0 tx=0).stop 2024-04-26T20:13:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 -- 172.21.15.131:0/2331819091 shutdown_connections 2024-04-26T20:13:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2331819091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a010a780 0x7fc5a010cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:00.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2331819091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 0x7fc5a0075810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:00.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2331819091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 0x7fc5a0074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:00.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 -- 172.21.15.131:0/2331819091 >> 172.21.15.131:0/2331819091 conn(0x7fc5a00fde30 msgr2=0x7fc5a0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:00.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 -- 172.21.15.131:0/2331819091 shutdown_connections 2024-04-26T20:13:00.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.655+0000 7fc5a6800640 1 -- 172.21.15.131:0/2331819091 wait complete. 2024-04-26T20:13:00.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 Processor -- start 2024-04-26T20:13:00.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 -- start start 2024-04-26T20:13:00.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 0x7fc5a019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 0x7fc5a019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:00.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a010a780 0x7fc5a01a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:01.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0110560 con 0x7fc5a0076970 2024-04-26T20:13:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01103e0 con 0x7fc5a0075390 2024-04-26T20:13:01.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.656+0000 7fc5a6800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01106e0 con 0x7fc5a010a780 2024-04-26T20:13:01.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 0x7fc5a019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:01.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 0x7fc5a019b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49984/0 (socket says 172.21.15.131:49984) 2024-04-26T20:13:01.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 -- 172.21.15.131:0/2146477094 learned_addr learned my addr 172.21.15.131:0/2146477094 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a5400640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 0x7fc5a019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a5e00640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a010a780 0x7fc5a01a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:01.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 -- 172.21.15.131:0/2146477094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a010a780 msgr2=0x7fc5a01a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a010a780 0x7fc5a01a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:01.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 -- 172.21.15.131:0/2146477094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 msgr2=0x7fc5a019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:01.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 0x7fc5a019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 -- 172.21.15.131:0/2146477094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a01a28b0 con 0x7fc5a0076970 2024-04-26T20:13:01.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.657+0000 7fc5a4a00640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 0x7fc5a019b1b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0057c0 tx=0x7fc58c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.658+0000 7fc59e000640 1 -- 172.21.15.131:0/2146477094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c037040 con 0x7fc5a0076970 2024-04-26T20:13:01.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.658+0000 7fc59e000640 1 -- 172.21.15.131:0/2146477094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc58c030bd0 con 0x7fc5a0076970 2024-04-26T20:13:01.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.658+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a01a2b40 con 0x7fc5a0076970 2024-04-26T20:13:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.658+0000 7fc59e000640 1 -- 172.21.15.131:0/2146477094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c005280 con 0x7fc5a0076970 2024-04-26T20:13:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.658+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a01a2fd0 con 0x7fc5a0076970 2024-04-26T20:13:01.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.659+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a01094b0 con 0x7fc5a0076970 2024-04-26T20:13:01.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.661+0000 7fc59e000640 1 -- 172.21.15.131:0/2146477094 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc58c043020 con 0x7fc5a0076970 2024-04-26T20:13:01.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.662+0000 7fc59e000640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5840a3c40 0x7fc5840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.662+0000 7fc59e000640 1 -- 172.21.15.131:0/2146477094 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc58c0efee0 con 0x7fc5a0076970 2024-04-26T20:13:01.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.663+0000 7fc5a5400640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5840a3c40 0x7fc5840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.663+0000 7fc5a5400640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5840a3c40 0x7fc5840a6100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5a019bb40 tx=0x7fc590009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.669+0000 7fc59e000640 1 -- 172.21.15.131:0/2146477094 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc58c0bc420 con 0x7fc5a0076970 2024-04-26T20:13:01.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.876+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5a0063530 con 0x7fc5a0076970 2024-04-26T20:13:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.877+0000 7fc59e000640 1 -- 172.21.15.131:0/2146477094 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fc58c0bbd20 con 0x7fc5a0076970 2024-04-26T20:13:01.798 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:13:01.842 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:01 smithi157 ceph-mon[32703]: pgmap v500: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T20:13:01.850 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:01 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2146477094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:13:01.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5840a3c40 msgr2=0x7fc5840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:01.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5840a3c40 0x7fc5840a6100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5a019bb40 tx=0x7fc590009260 comp rx=0 tx=0).stop 2024-04-26T20:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 msgr2=0x7fc5a019b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:01.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 0x7fc5a019b1b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0057c0 tx=0x7fc58c032da0 comp rx=0 tx=0).stop 2024-04-26T20:13:01.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 shutdown_connections 2024-04-26T20:13:02.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc5840a3c40 0x7fc5840a6100 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a010a780 0x7fc5a01a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a0076970 0x7fc5a019b1b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 --2- 172.21.15.131:0/2146477094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a0075390 0x7fc5a019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 >> 172.21.15.131:0/2146477094 conn(0x7fc5a00fde30 msgr2=0x7fc5a0106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:02.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 shutdown_connections 2024-04-26T20:13:02.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:00.880+0000 7fc5a6800640 1 -- 172.21.15.131:0/2146477094 wait complete. 2024-04-26T20:13:02.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:13:02.264 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:01 smithi131 ceph-mon[27919]: pgmap v500: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T20:13:02.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:01 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2146477094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:13:02.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:01 smithi148 ceph-mon[32281]: pgmap v500: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T20:13:02.420 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:01 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2146477094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:13:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 -- 172.21.15.131:0/2286396344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 msgr2=0x7fd3f410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:02.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 --2- 172.21.15.131:0/2286396344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 0x7fd3f410a440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009900 tx=0x7fd3e402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 -- 172.21.15.131:0/2286396344 shutdown_connections 2024-04-26T20:13:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 --2- 172.21.15.131:0/2286396344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3f410a980 0x7fd3f41112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 --2- 172.21.15.131:0/2286396344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 0x7fd3f410a440 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 --2- 172.21.15.131:0/2286396344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3f4108e70 0x7fd3f4109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:02.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 -- 172.21.15.131:0/2286396344 >> 172.21.15.131:0/2286396344 conn(0x7fd3f4077e00 msgr2=0x7fd3f407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 -- 172.21.15.131:0/2286396344 shutdown_connections 2024-04-26T20:13:02.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.534+0000 7fd3fa400640 1 -- 172.21.15.131:0/2286396344 wait complete. 2024-04-26T20:13:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 Processor -- start 2024-04-26T20:13:02.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 -- start start 2024-04-26T20:13:02.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3f4108e70 0x7fd3f419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:02.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 0x7fd3f419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:02.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3f410a980 0x7fd3f41a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41149a0 con 0x7fd3f4109fc0 2024-04-26T20:13:03.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4114820 con 0x7fd3f4108e70 2024-04-26T20:13:03.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.535+0000 7fd3fa400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4114b20 con 0x7fd3f410a980 2024-04-26T20:13:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 0x7fd3f419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:03.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3f4108e70 0x7fd3f419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:03.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 0x7fd3f419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50006/0 (socket says 172.21.15.131:50006) 2024-04-26T20:13:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3f410a980 0x7fd3f41a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:03.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f3e00640 1 -- 172.21.15.131:0/1061912100 learned_addr learned my addr 172.21.15.131:0/1061912100 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:03.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9000640 1 -- 172.21.15.131:0/1061912100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3f410a980 msgr2=0x7fd3f41a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:03.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9000640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3f410a980 0x7fd3f41a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9000640 1 -- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 msgr2=0x7fd3f419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:03.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9000640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 0x7fd3f419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:03.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9000640 1 -- 172.21.15.131:0/1061912100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41a6c70 con 0x7fd3f4108e70 2024-04-26T20:13:03.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.536+0000 7fd3f9a00640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3f410a980 0x7fd3f41a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:13:03.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.537+0000 7fd3f9000640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3f4108e70 0x7fd3f419f050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9f0 tx=0x7fd3e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.537+0000 7fd3f1600640 1 -- 172.21.15.131:0/1061912100 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800cd80 con 0x7fd3f4108e70 2024-04-26T20:13:03.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.537+0000 7fd3f1600640 1 -- 172.21.15.131:0/1061912100 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e80035f0 con 0x7fd3f4108e70 2024-04-26T20:13:03.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.537+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41a6f60 con 0x7fd3f4108e70 2024-04-26T20:13:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.537+0000 7fd3f1600640 1 -- 172.21.15.131:0/1061912100 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8010690 con 0x7fd3f4108e70 2024-04-26T20:13:03.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.537+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a73c0 con 0x7fd3f4108e70 2024-04-26T20:13:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.538+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f4110180 con 0x7fd3f4108e70 2024-04-26T20:13:03.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.540+0000 7fd3f1600640 1 -- 172.21.15.131:0/1061912100 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd3e80037a0 con 0x7fd3f4108e70 2024-04-26T20:13:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.541+0000 7fd3f1600640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:03.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.541+0000 7fd3f1600640 1 -- 172.21.15.131:0/1061912100 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd3e80d0c60 con 0x7fd3f4108e70 2024-04-26T20:13:03.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.541+0000 7fd3f3e00640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:03.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.542+0000 7fd3f3e00640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd3f41a0560 tx=0x7fd3e40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:03.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.546+0000 7fd3f1600640 1 -- 172.21.15.131:0/1061912100 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd3e809d090 con 0x7fd3f4108e70 2024-04-26T20:13:03.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.750+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd3f402dd60 con 0x7fd3f4108e70 2024-04-26T20:13:03.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.751+0000 7fd3f1600640 1 -- 172.21.15.131:0/1061912100 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fd3e809c990 con 0x7fd3f4108e70 2024-04-26T20:13:03.659 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:13:03.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.754+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd3dc0a3c40 msgr2=0x7fd3dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:03.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.754+0000 7fd3fa400640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd3f41a0560 tx=0x7fd3e40314a0 comp rx=0 tx=0).stop 2024-04-26T20:13:03.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.754+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3f4108e70 msgr2=0x7fd3f419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.754+0000 7fd3fa400640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3f4108e70 0x7fd3f419f050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9f0 tx=0x7fd3e800eeb0 comp rx=0 tx=0).stop 2024-04-26T20:13:03.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 shutdown_connections 2024-04-26T20:13:03.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:03.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3f410a980 0x7fd3f41a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:04.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3f4109fc0 0x7fd3f419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:04.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 --2- 172.21.15.131:0/1061912100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3f4108e70 0x7fd3f419f050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:04.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 >> 172.21.15.131:0/1061912100 conn(0x7fd3f4077e00 msgr2=0x7fd3f410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:04.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 shutdown_connections 2024-04-26T20:13:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:02.755+0000 7fd3fa400640 1 -- 172.21.15.131:0/1061912100 wait complete. 2024-04-26T20:13:04.179 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:13:04.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:03 smithi157 ceph-mon[32703]: pgmap v501: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:04.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:03 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1061912100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:13:04.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:03 smithi148 ceph-mon[32281]: pgmap v501: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:04.223 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:03 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1061912100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:13:04.236 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:03 smithi131 ceph-mon[27919]: pgmap v501: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:04.247 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:03 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1061912100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:13:04.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 -- 172.21.15.131:0/3993329662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410ab40 msgr2=0x7fe0741113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 --2- 172.21.15.131:0/3993329662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410ab40 0x7fe0741113d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009ca0 tx=0x7fe06c02a8c0 comp rx=0 tx=0).stop 2024-04-26T20:13:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 -- 172.21.15.131:0/3993329662 shutdown_connections 2024-04-26T20:13:04.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 --2- 172.21.15.131:0/3993329662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410ab40 0x7fe0741113d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:04.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 --2- 172.21.15.131:0/3993329662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe07410a180 0x7fe07410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 --2- 172.21.15.131:0/3993329662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe074108f90 0x7fe0741093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 -- 172.21.15.131:0/3993329662 >> 172.21.15.131:0/3993329662 conn(0x7fe074077f30 msgr2=0x7fe07407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 -- 172.21.15.131:0/3993329662 shutdown_connections 2024-04-26T20:13:04.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.592+0000 7fe07a000640 1 -- 172.21.15.131:0/3993329662 wait complete. 2024-04-26T20:13:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.593+0000 7fe07a000640 1 Processor -- start 2024-04-26T20:13:04.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.593+0000 7fe07a000640 1 -- start start 2024-04-26T20:13:04.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.593+0000 7fe07a000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074108f90 0x7fe07419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe07a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410a180 0x7fe07419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe07a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07410ab40 0x7fe0741a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:04.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe07a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe074114b20 con 0x7fe07410a180 2024-04-26T20:13:04.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe07a000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0741149a0 con 0x7fe074108f90 2024-04-26T20:13:04.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe07a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe074114ca0 con 0x7fe07410ab40 2024-04-26T20:13:05.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074108f90 0x7fe07419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:05.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe06be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410a180 0x7fe07419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:05.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074108f90 0x7fe07419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:33306/0 (socket says 172.21.15.131:33306) 2024-04-26T20:13:05.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 -- 172.21.15.131:0/3858615757 learned_addr learned my addr 172.21.15.131:0/3858615757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:05.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073e00640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07410ab40 0x7fe0741a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:05.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 -- 172.21.15.131:0/3858615757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07410ab40 msgr2=0x7fe0741a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:05.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07410ab40 0x7fe0741a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 -- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410a180 msgr2=0x7fe07419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410a180 0x7fe07419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe073400640 1 -- 172.21.15.131:0/3858615757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0741a6d50 con 0x7fe074108f90 2024-04-26T20:13:05.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.594+0000 7fe06be00640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410a180 0x7fe07419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:13:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.595+0000 7fe073400640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074108f90 0x7fe07419f130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe06400d950 tx=0x7fe06400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:05.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.595+0000 7fe070c00640 1 -- 172.21.15.131:0/3858615757 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064017020 con 0x7fe074108f90 2024-04-26T20:13:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.595+0000 7fe070c00640 1 -- 172.21.15.131:0/3858615757 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe064009c30 con 0x7fe074108f90 2024-04-26T20:13:05.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.595+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0741a6fe0 con 0x7fe074108f90 2024-04-26T20:13:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.595+0000 7fe070c00640 1 -- 172.21.15.131:0/3858615757 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06400f600 con 0x7fe074108f90 2024-04-26T20:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.595+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0741a74a0 con 0x7fe074108f90 2024-04-26T20:13:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.596+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe074110250 con 0x7fe074108f90 2024-04-26T20:13:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.598+0000 7fe070c00640 1 -- 172.21.15.131:0/3858615757 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe06401a020 con 0x7fe074108f90 2024-04-26T20:13:05.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.599+0000 7fe070c00640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe0580a3c40 0x7fe0580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:05.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.600+0000 7fe070c00640 1 -- 172.21.15.131:0/3858615757 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe0640d08b0 con 0x7fe074108f90 2024-04-26T20:13:05.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.600+0000 7fe06be00640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe0580a3c40 0x7fe0580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:05.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.600+0000 7fe06be00640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe0580a3c40 0x7fe0580a6100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe0741a0640 tx=0x7fe05c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:05.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.604+0000 7fe070c00640 1 -- 172.21.15.131:0/3858615757 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe064021020 con 0x7fe074108f90 2024-04-26T20:13:05.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.806+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe0740635d0 con 0x7fe074108f90 2024-04-26T20:13:05.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.807+0000 7fe070c00640 1 -- 172.21.15.131:0/3858615757 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fe06409cce0 con 0x7fe074108f90 2024-04-26T20:13:05.813 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:13:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe0580a3c40 msgr2=0x7fe0580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:05.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe0580a3c40 0x7fe0580a6100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe0741a0640 tx=0x7fe05c00b410 comp rx=0 tx=0).stop 2024-04-26T20:13:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074108f90 msgr2=0x7fe07419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:05.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074108f90 0x7fe07419f130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe06400d950 tx=0x7fe06400de10 comp rx=0 tx=0).stop 2024-04-26T20:13:05.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 shutdown_connections 2024-04-26T20:13:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe0580a3c40 0x7fe0580a6100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07410ab40 0x7fe0741a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:05.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07410a180 0x7fe07419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 --2- 172.21.15.131:0/3858615757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074108f90 0x7fe07419f130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:05.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.809+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 >> 172.21.15.131:0/3858615757 conn(0x7fe074077f30 msgr2=0x7fe07410d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:06.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.810+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 shutdown_connections 2024-04-26T20:13:06.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:04.810+0000 7fe07a000640 1 -- 172.21.15.131:0/3858615757 wait complete. 2024-04-26T20:13:06.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:13:06.089 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:05 smithi131 ceph-mon[27919]: pgmap v502: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:06.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:05 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3858615757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:13:06.137 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:05 smithi157 ceph-mon[32703]: pgmap v502: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:06.144 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:05 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3858615757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:13:06.153 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:05 smithi148 ceph-mon[32281]: pgmap v502: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:06.159 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:05 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3858615757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:13:06.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.343+0000 7f9b47200640 1 -- 172.21.15.131:0/2252883705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 msgr2=0x7f9b481112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:06.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.343+0000 7f9b47200640 1 --2- 172.21.15.131:0/2252883705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 0x7f9b481112a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ca0 tx=0x7f9b4002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:13:06.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.344+0000 7f9b47200640 1 -- 172.21.15.131:0/2252883705 shutdown_connections 2024-04-26T20:13:06.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.344+0000 7f9b47200640 1 --2- 172.21.15.131:0/2252883705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 0x7f9b481112a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:06.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.344+0000 7f9b47200640 1 --2- 172.21.15.131:0/2252883705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48109fc0 0x7f9b4810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:06.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.344+0000 7f9b47200640 1 --2- 172.21.15.131:0/2252883705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b48108e70 0x7f9b48109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:06.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.344+0000 7f9b47200640 1 -- 172.21.15.131:0/2252883705 >> 172.21.15.131:0/2252883705 conn(0x7f9b48077e00 msgr2=0x7f9b4807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:06.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.344+0000 7f9b47200640 1 -- 172.21.15.131:0/2252883705 shutdown_connections 2024-04-26T20:13:06.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.344+0000 7f9b47200640 1 -- 172.21.15.131:0/2252883705 wait complete. 2024-04-26T20:13:06.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 Processor -- start 2024-04-26T20:13:06.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 -- start start 2024-04-26T20:13:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48108e70 0x7f9b4819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:06.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b48109fc0 0x7f9b4819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:06.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 0x7f9b481a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:06.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48114a20 con 0x7f9b4810a980 2024-04-26T20:13:06.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481148a0 con 0x7f9b48109fc0 2024-04-26T20:13:06.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b47200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48114ba0 con 0x7f9b48108e70 2024-04-26T20:13:06.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.345+0000 7f9b46800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 0x7f9b481a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:06.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b45e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48108e70 0x7f9b4819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:06.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b46800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 0x7f9b481a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50104/0 (socket says 172.21.15.131:50104) 2024-04-26T20:13:06.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b46800640 1 -- 172.21.15.131:0/3162828194 learned_addr learned my addr 172.21.15.131:0/3162828194 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:06.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b3fe00640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b48109fc0 0x7f9b4819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b45e00640 1 -- 172.21.15.131:0/3162828194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b48109fc0 msgr2=0x7f9b4819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:07.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b45e00640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b48109fc0 0x7f9b4819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:07.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b45e00640 1 -- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 msgr2=0x7f9b481a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:07.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b45e00640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 0x7f9b481a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b45e00640 1 -- 172.21.15.131:0/3162828194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b481a6c00 con 0x7f9b48108e70 2024-04-26T20:13:07.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b45e00640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48108e70 0x7f9b4819efe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800e9f0 tx=0x7f9b3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:07.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b3ea00640 1 -- 172.21.15.131:0/3162828194 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800ccc0 con 0x7f9b48108e70 2024-04-26T20:13:07.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.346+0000 7f9b3ea00640 1 -- 172.21.15.131:0/3162828194 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b380108e0 con 0x7f9b48108e70 2024-04-26T20:13:07.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.347+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b481a6ef0 con 0x7f9b48108e70 2024-04-26T20:13:07.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.347+0000 7f9b3ea00640 1 -- 172.21.15.131:0/3162828194 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38018880 con 0x7f9b48108e70 2024-04-26T20:13:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.347+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481a7360 con 0x7f9b48108e70 2024-04-26T20:13:07.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.348+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b481100b0 con 0x7f9b48108e70 2024-04-26T20:13:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.350+0000 7f9b3ea00640 1 -- 172.21.15.131:0/3162828194 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9b38010430 con 0x7f9b48108e70 2024-04-26T20:13:07.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.352+0000 7f9b3ea00640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b280a3c40 0x7f9b280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:07.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.352+0000 7f9b3ea00640 1 -- 172.21.15.131:0/3162828194 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9b380d11a0 con 0x7f9b48108e70 2024-04-26T20:13:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.352+0000 7f9b3fe00640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b280a3c40 0x7f9b280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.352+0000 7f9b3fe00640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b280a3c40 0x7f9b280a6100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b481a04f0 tx=0x7f9b30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:07.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.354+0000 7f9b3ea00640 1 -- 172.21.15.131:0/3162828194 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9b3809cf80 con 0x7f9b48108e70 2024-04-26T20:13:07.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.559+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9b4802db70 con 0x7f9b48108e70 2024-04-26T20:13:07.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.561+0000 7f9b3ea00640 1 -- 172.21.15.131:0/3162828194 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f9b380a1280 con 0x7f9b48108e70 2024-04-26T20:13:07.507 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:13:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b280a3c40 msgr2=0x7f9b280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b280a3c40 0x7f9b280a6100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b481a04f0 tx=0x7f9b30005f90 comp rx=0 tx=0).stop 2024-04-26T20:13:07.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48108e70 msgr2=0x7f9b4819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48108e70 0x7f9b4819efe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800e9f0 tx=0x7f9b3800eeb0 comp rx=0 tx=0).stop 2024-04-26T20:13:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 shutdown_connections 2024-04-26T20:13:07.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b4810a980 0x7f9b481a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b48109fc0 0x7f9b4819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b280a3c40 0x7f9b280a6100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:07.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 --2- 172.21.15.131:0/3162828194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48108e70 0x7f9b4819efe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 >> 172.21.15.131:0/3162828194 conn(0x7f9b48077e00 msgr2=0x7f9b4810d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:07.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 shutdown_connections 2024-04-26T20:13:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:06.563+0000 7f9b47200640 1 -- 172.21.15.131:0/3162828194 wait complete. 2024-04-26T20:13:07.654 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-26T20:13:07.761 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:13:07.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:07 smithi131 ceph-mon[27919]: pgmap v503: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:07.871 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:07 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3162828194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:13:07.907 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:07 smithi148 ceph-mon[32281]: pgmap v503: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:07.912 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:07 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3162828194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:13:07.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:07 smithi157 ceph-mon[32703]: pgmap v503: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:07.985 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:07 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3162828194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:13:08.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 -- 172.21.15.131:0/974602201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 msgr2=0x7f96f8101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 --2- 172.21.15.131:0/974602201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f8101c70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009900 tx=0x7f96e802a410 comp rx=0 tx=0).stop 2024-04-26T20:13:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 -- 172.21.15.131:0/974602201 shutdown_connections 2024-04-26T20:13:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 --2- 172.21.15.131:0/974602201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f8103400 0x7f96f8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:08.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 --2- 172.21.15.131:0/974602201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f8102a40 0x7f96f8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:08.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 --2- 172.21.15.131:0/974602201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f8101c70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:08.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 -- 172.21.15.131:0/974602201 >> 172.21.15.131:0/974602201 conn(0x7f96f80fcdc0 msgr2=0x7f96f80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 -- 172.21.15.131:0/974602201 shutdown_connections 2024-04-26T20:13:08.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.117+0000 7f96fd000640 1 -- 172.21.15.131:0/974602201 wait complete. 2024-04-26T20:13:08.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.118+0000 7f96fd000640 1 Processor -- start 2024-04-26T20:13:08.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.118+0000 7f96fd000640 1 -- start start 2024-04-26T20:13:08.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96fd000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96fd000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f8102a40 0x7f96f819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:08.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96fd000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f8103400 0x7f96f81a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:08.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96fd000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96f810d3c0 con 0x7f96f8101850 2024-04-26T20:13:08.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96fd000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96f810d240 con 0x7f96f8102a40 2024-04-26T20:13:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96fd000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96f810d540 con 0x7f96f8103400 2024-04-26T20:13:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50132/0 (socket says 172.21.15.131:50132) 2024-04-26T20:13:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96f5e00640 1 -- 172.21.15.131:0/3304447946 learned_addr learned my addr 172.21.15.131:0/3304447946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:08.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96f5400640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f8102a40 0x7f96f819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:08.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.119+0000 7f96f6800640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f8103400 0x7f96f81a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:08.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96f5e00640 1 -- 172.21.15.131:0/3304447946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f8103400 msgr2=0x7f96f81a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96f5e00640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f8103400 0x7f96f81a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:08.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96f5e00640 1 -- 172.21.15.131:0/3304447946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f8102a40 msgr2=0x7f96f819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:08.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96f5e00640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f8102a40 0x7f96f819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:08.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96f5e00640 1 -- 172.21.15.131:0/3304447946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f81a2a20 con 0x7f96f8101850 2024-04-26T20:13:08.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96f5e00640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f819ae00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96e8031500 tx=0x7f96e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:08.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96eea00640 1 -- 172.21.15.131:0/3304447946 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e803f070 con 0x7f96f8101850 2024-04-26T20:13:08.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96eea00640 1 -- 172.21.15.131:0/3304447946 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e8031c50 con 0x7f96f8101850 2024-04-26T20:13:08.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f81a2cb0 con 0x7f96f8101850 2024-04-26T20:13:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96eea00640 1 -- 172.21.15.131:0/3304447946 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8035030 con 0x7f96f8101850 2024-04-26T20:13:08.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.120+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f81a31a0 con 0x7f96f8101850 2024-04-26T20:13:08.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.121+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f81a3370 con 0x7f96f8101850 2024-04-26T20:13:08.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.123+0000 7f96eea00640 1 -- 172.21.15.131:0/3304447946 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f96e8042070 con 0x7f96f8101850 2024-04-26T20:13:08.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.126+0000 7f96eea00640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f96d80a3c40 0x7f96d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:09.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.126+0000 7f96eea00640 1 -- 172.21.15.131:0/3304447946 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f96e80f10e0 con 0x7f96f8101850 2024-04-26T20:13:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.126+0000 7f96f5400640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f96d80a3c40 0x7f96d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:09.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.126+0000 7f96f5400640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f96d80a3c40 0x7f96d80a6100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f96f819c310 tx=0x7f96e0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:09.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.129+0000 7f96eea00640 1 -- 172.21.15.131:0/3304447946 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f96e80bcec0 con 0x7f96f8101850 2024-04-26T20:13:09.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:08 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3304447946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:13:09.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:08 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3304447946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:13:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.371+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96f81a3700 con 0x7f96f8101850 2024-04-26T20:13:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.372+0000 7f96eea00640 1 -- 172.21.15.131:0/3304447946 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7f96e80c11c0 con 0x7f96f8101850 2024-04-26T20:13:09.158 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:13:09.293 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:12:47.696526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24571,24557]},"id":1}]} 2024-04-26T20:13:09.303 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-04-26T20:13:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f96d80a3c40 msgr2=0x7f96d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f96d80a3c40 0x7f96d80a6100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f96f819c310 tx=0x7f96e0009500 comp rx=0 tx=0).stop 2024-04-26T20:13:09.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 msgr2=0x7f96f819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:09.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f819ae00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96e8031500 tx=0x7f96e8003c10 comp rx=0 tx=0).stop 2024-04-26T20:13:09.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 shutdown_connections 2024-04-26T20:13:09.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f96d80a3c40 0x7f96d80a6100 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:09.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f8103400 0x7f96f81a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:09.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f8102a40 0x7f96f819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 --2- 172.21.15.131:0/3304447946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f8101850 0x7f96f819ae00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.375+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 >> 172.21.15.131:0/3304447946 conn(0x7f96f80fcdc0 msgr2=0x7f96f80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:09.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.376+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 shutdown_connections 2024-04-26T20:13:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:08.376+0000 7f96fd000640 1 -- 172.21.15.131:0/3304447946 wait complete. 2024-04-26T20:13:09.561 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-26T20:13:09.569 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-26T20:13:09.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:13:09.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:08 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3304447946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:13:09.799 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:09 smithi131 ceph-mon[27919]: pgmap v504: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:09.851 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:09 smithi148 ceph-mon[32281]: pgmap v504: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:09.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:09 smithi157 ceph-mon[32703]: pgmap v504: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:10.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 -- 172.21.15.131:0/3284312885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c076970 msgr2=0x7f4a0c074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:10.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 --2- 172.21.15.131:0/3284312885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c076970 0x7f4a0c074dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009930 tx=0x7f49fc02a420 comp rx=0 tx=0).stop 2024-04-26T20:13:10.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 -- 172.21.15.131:0/3284312885 shutdown_connections 2024-04-26T20:13:10.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 --2- 172.21.15.131:0/3284312885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c10ed40 0x7f4a0c111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:10.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 --2- 172.21.15.131:0/3284312885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a0c075390 0x7f4a0c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:10.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 --2- 172.21.15.131:0/3284312885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c076970 0x7f4a0c074dc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 -- 172.21.15.131:0/3284312885 >> 172.21.15.131:0/3284312885 conn(0x7f4a0c0fdcc0 msgr2=0x7f4a0c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 -- 172.21.15.131:0/3284312885 shutdown_connections 2024-04-26T20:13:10.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.058+0000 7f4a12400640 1 -- 172.21.15.131:0/3284312885 wait complete. 2024-04-26T20:13:10.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.059+0000 7f4a12400640 1 Processor -- start 2024-04-26T20:13:10.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.059+0000 7f4a12400640 1 -- start start 2024-04-26T20:13:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a12400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c075390 0x7f4a0c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:10.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a12400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c076970 0x7f4a0c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:10.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a12400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a0c10ed40 0x7f4a0c1a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a12400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c114890 con 0x7f4a0c075390 2024-04-26T20:13:10.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a12400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c114710 con 0x7f4a0c076970 2024-04-26T20:13:10.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a12400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c114a10 con 0x7f4a0c10ed40 2024-04-26T20:13:10.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a11000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c075390 0x7f4a0c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:10.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a03e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c076970 0x7f4a0c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:10.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a11a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a0c10ed40 0x7f4a0c1a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:10.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a03e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c076970 0x7f4a0c19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:33352/0 (socket says 172.21.15.131:33352) 2024-04-26T20:13:10.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.060+0000 7f4a03e00640 1 -- 172.21.15.131:0/133107184 learned_addr learned my addr 172.21.15.131:0/133107184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:10.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a03e00640 1 -- 172.21.15.131:0/133107184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a0c10ed40 msgr2=0x7f4a0c1a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:10.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a03e00640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a0c10ed40 0x7f4a0c1a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a03e00640 1 -- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c075390 msgr2=0x7f4a0c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:10.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a03e00640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c075390 0x7f4a0c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:10.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a03e00640 1 -- 172.21.15.131:0/133107184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1a6be0 con 0x7f4a0c076970 2024-04-26T20:13:10.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a03e00640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c076970 0x7f4a0c19f500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49f400e9f0 tx=0x7f49f400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:10.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a0a000640 1 -- 172.21.15.131:0/133107184 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f400cd80 con 0x7f4a0c076970 2024-04-26T20:13:10.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c1a6ed0 con 0x7f4a0c076970 2024-04-26T20:13:10.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a0a000640 1 -- 172.21.15.131:0/133107184 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f40054d0 con 0x7f4a0c076970 2024-04-26T20:13:10.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a0a000640 1 -- 172.21.15.131:0/133107184 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4010690 con 0x7f4a0c076970 2024-04-26T20:13:10.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.061+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a7340 con 0x7f4a0c076970 2024-04-26T20:13:10.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.062+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c104f10 con 0x7f4a0c076970 2024-04-26T20:13:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.065+0000 7f4a0a000640 1 -- 172.21.15.131:0/133107184 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f49f4004e90 con 0x7f4a0c076970 2024-04-26T20:13:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.066+0000 7f4a0a000640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49f00a3c40 0x7f49f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:11.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.066+0000 7f4a0a000640 1 -- 172.21.15.131:0/133107184 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f49f40d0dc0 con 0x7f4a0c076970 2024-04-26T20:13:11.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.066+0000 7f4a11000640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49f00a3c40 0x7f49f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:11.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.067+0000 7f4a11000640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49f00a3c40 0x7f49f00a6100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49fc006010 tx=0x7f49fc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:11.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.070+0000 7f4a0a000640 1 -- 172.21.15.131:0/133107184 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f49f409d1f0 con 0x7f4a0c076970 2024-04-26T20:13:11.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.273+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a0c062d70 con 0x7f49f00a3c40 2024-04-26T20:13:11.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.289+0000 7f4a0a000640 1 -- 172.21.15.131:0/133107184 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340369 (secure 0 0 0) 0x7f4a0c062d70 con 0x7f49f00a3c40 2024-04-26T20:13:11.145 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:13:11.157 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-26T20:13:11.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49f00a3c40 msgr2=0x7f49f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:11.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49f00a3c40 0x7f49f00a6100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49fc006010 tx=0x7f49fc0314a0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c076970 msgr2=0x7f4a0c19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:11.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c076970 0x7f4a0c19f500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49f400e9f0 tx=0x7f49f400edf0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 shutdown_connections 2024-04-26T20:13:11.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49f00a3c40 0x7f49f00a6100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a0c10ed40 0x7f4a0c1a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c076970 0x7f4a0c19f500 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 --2- 172.21.15.131:0/133107184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a0c075390 0x7f4a0c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.295+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 >> 172.21.15.131:0/133107184 conn(0x7f4a0c0fdcc0 msgr2=0x7f4a0c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:11.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.296+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 shutdown_connections 2024-04-26T20:13:11.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:10.296+0000 7f4a12400640 1 -- 172.21.15.131:0/133107184 wait complete. 2024-04-26T20:13:11.465 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-04-26T20:13:09.747450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655652,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1685,"num_read_kb":958,"num_write":357,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368480,"kb_used_data":15140,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573344,"statfs":{"total":1151940427776,"available":1151563104256,"internally_reserved":0,"allocated":15503360,"data_stored":8593694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006389"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.827058+0000","last_change":"2024-04-26T20:07:34.771977+0000","last_active":"2024-04-26T20:09:39.827058+0000","last_peered":"2024-04-26T20:09:39.827058+0000","last_clean":"2024-04-26T20:09:39.827058+0000","last_became_active":"2024-04-26T20:07:34.771658+0000","last_became_peered":"2024-04-26T20:07:34.771658+0000","last_unstale":"2024-04-26T20:09:39.827058+0000","last_undegraded":"2024-04-26T20:09:39.827058+0000","last_fullsized":"2024-04-26T20:09:39.827058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:38.419057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044979+0000","last_change":"2024-04-26T20:07:34.768774+0000","last_active":"2024-04-26T20:09:43.044979+0000","last_peered":"2024-04-26T20:09:43.044979+0000","last_clean":"2024-04-26T20:09:43.044979+0000","last_became_active":"2024-04-26T20:07:34.768383+0000","last_became_peered":"2024-04-26T20:07:34.768383+0000","last_unstale":"2024-04-26T20:09:43.044979+0000","last_undegraded":"2024-04-26T20:09:43.044979+0000","last_fullsized":"2024-04-26T20:09:43.044979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:43.813261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044955+0000","last_change":"2024-04-26T20:07:34.761796+0000","last_active":"2024-04-26T20:09:43.044955+0000","last_peered":"2024-04-26T20:09:43.044955+0000","last_clean":"2024-04-26T20:09:43.044955+0000","last_became_active":"2024-04-26T20:07:34.761403+0000","last_became_peered":"2024-04-26T20:07:34.761403+0000","last_unstale":"2024-04-26T20:09:43.044955+0000","last_undegraded":"2024-04-26T20:09:43.044955+0000","last_fullsized":"2024-04-26T20:09:43.044955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:06.832304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.012166+0000","last_change":"2024-04-26T20:07:34.756728+0000","last_active":"2024-04-26T20:09:37.012166+0000","last_peered":"2024-04-26T20:09:37.012166+0000","last_clean":"2024-04-26T20:09:37.012166+0000","last_became_active":"2024-04-26T20:07:34.756406+0000","last_became_peered":"2024-04-26T20:07:34.756406+0000","last_unstale":"2024-04-26T20:09:37.012166+0000","last_undegraded":"2024-04-26T20:09:37.012166+0000","last_fullsized":"2024-04-26T20:09:37.012166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:27.753674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457110+0000","last_change":"2024-04-26T20:07:34.758243+0000","last_active":"2024-04-26T20:10:05.457110+0000","last_peered":"2024-04-26T20:10:05.457110+0000","last_clean":"2024-04-26T20:10:05.457110+0000","last_became_active":"2024-04-26T20:07:34.757868+0000","last_became_peered":"2024-04-26T20:07:34.757868+0000","last_unstale":"2024-04-26T20:10:05.457110+0000","last_undegraded":"2024-04-26T20:10:05.457110+0000","last_fullsized":"2024-04-26T20:10:05.457110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:53.361835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.971598+0000","last_change":"2024-04-26T20:08:15.585311+0000","last_active":"2024-04-26T20:10:07.971598+0000","last_peered":"2024-04-26T20:10:07.971598+0000","last_clean":"2024-04-26T20:10:07.971598+0000","last_became_active":"2024-04-26T20:08:15.584982+0000","last_became_peered":"2024-04-26T20:08:15.584982+0000","last_unstale":"2024-04-26T20:10:07.971598+0000","last_undegraded":"2024-04-26T20:10:07.971598+0000","last_fullsized":"2024-04-26T20:10:07.971598+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:56.828218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:08.269506+0000","last_change":"2024-04-26T20:07:34.757529+0000","last_active":"2024-04-26T20:10:08.269506+0000","last_peered":"2024-04-26T20:10:08.269506+0000","last_clean":"2024-04-26T20:10:08.269506+0000","last_became_active":"2024-04-26T20:07:34.757166+0000","last_became_peered":"2024-04-26T20:07:34.757166+0000","last_unstale":"2024-04-26T20:10:08.269506+0000","last_undegraded":"2024-04-26T20:10:08.269506+0000","last_fullsized":"2024-04-26T20:10:08.269506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:01.309698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:04.581136+0000","last_change":"2024-04-26T20:07:34.765352+0000","last_active":"2024-04-26T20:10:04.581136+0000","last_peered":"2024-04-26T20:10:04.581136+0000","last_clean":"2024-04-26T20:10:04.581136+0000","last_became_active":"2024-04-26T20:07:34.758400+0000","last_became_peered":"2024-04-26T20:07:34.758400+0000","last_unstale":"2024-04-26T20:10:04.581136+0000","last_undegraded":"2024-04-26T20:10:04.581136+0000","last_fullsized":"2024-04-26T20:10:04.581136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:43.940746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.186777+0000","last_change":"2024-04-26T20:07:34.754413+0000","last_active":"2024-04-26T20:10:03.186777+0000","last_peered":"2024-04-26T20:10:03.186777+0000","last_clean":"2024-04-26T20:10:03.186777+0000","last_became_active":"2024-04-26T20:07:34.754073+0000","last_became_peered":"2024-04-26T20:07:34.754073+0000","last_unstale":"2024-04-26T20:10:03.186777+0000","last_undegraded":"2024-04-26T20:10:03.186777+0000","last_fullsized":"2024-04-26T20:10:03.186777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:25.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.971090+0000","last_change":"2024-04-26T20:07:34.748739+0000","last_active":"2024-04-26T20:10:02.971090+0000","last_peered":"2024-04-26T20:10:02.971090+0000","last_clean":"2024-04-26T20:10:02.971090+0000","last_became_active":"2024-04-26T20:07:34.748378+0000","last_became_peered":"2024-04-26T20:07:34.748378+0000","last_unstale":"2024-04-26T20:10:02.971090+0000","last_undegraded":"2024-04-26T20:10:02.971090+0000","last_fullsized":"2024-04-26T20:10:02.971090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:13.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.268839+0000","last_change":"2024-04-26T20:07:34.747402+0000","last_active":"2024-04-26T20:10:03.268839+0000","last_peered":"2024-04-26T20:10:03.268839+0000","last_clean":"2024-04-26T20:10:03.268839+0000","last_became_active":"2024-04-26T20:07:34.746965+0000","last_became_peered":"2024-04-26T20:07:34.746965+0000","last_unstale":"2024-04-26T20:10:03.268839+0000","last_undegraded":"2024-04-26T20:10:03.268839+0000","last_fullsized":"2024-04-26T20:10:03.268839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:32.066286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.367770+0000","last_change":"2024-04-26T20:09:16.366157+0000","last_active":"2024-04-26T20:09:16.367770+0000","last_peered":"2024-04-26T20:09:16.367770+0000","last_clean":"2024-04-26T20:09:16.367770+0000","last_became_active":"2024-04-26T20:09:16.365132+0000","last_became_peered":"2024-04-26T20:09:16.365132+0000","last_unstale":"2024-04-26T20:09:16.367770+0000","last_undegraded":"2024-04-26T20:09:16.367770+0000","last_fullsized":"2024-04-26T20:09:16.367770+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.559981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.646861+0000","last_change":"2024-04-26T20:07:34.766542+0000","last_active":"2024-04-26T20:10:02.646861+0000","last_peered":"2024-04-26T20:10:02.646861+0000","last_clean":"2024-04-26T20:10:02.646861+0000","last_became_active":"2024-04-26T20:07:34.766185+0000","last_became_peered":"2024-04-26T20:07:34.766185+0000","last_unstale":"2024-04-26T20:10:02.646861+0000","last_undegraded":"2024-04-26T20:10:02.646861+0000","last_fullsized":"2024-04-26T20:10:02.646861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:34.499145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.047069+0000","last_change":"2024-04-26T20:07:34.762610+0000","last_active":"2024-04-26T20:09:58.047069+0000","last_peered":"2024-04-26T20:09:58.047069+0000","last_clean":"2024-04-26T20:09:58.047069+0000","last_became_active":"2024-04-26T20:07:34.762100+0000","last_became_peered":"2024-04-26T20:07:34.762100+0000","last_unstale":"2024-04-26T20:09:58.047069+0000","last_undegraded":"2024-04-26T20:09:58.047069+0000","last_fullsized":"2024-04-26T20:09:58.047069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:18.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.047033+0000","last_change":"2024-04-26T20:07:34.766754+0000","last_active":"2024-04-26T20:09:58.047033+0000","last_peered":"2024-04-26T20:09:58.047033+0000","last_clean":"2024-04-26T20:09:58.047033+0000","last_became_active":"2024-04-26T20:07:34.762204+0000","last_became_peered":"2024-04-26T20:07:34.762204+0000","last_unstale":"2024-04-26T20:09:58.047033+0000","last_undegraded":"2024-04-26T20:09:58.047033+0000","last_fullsized":"2024-04-26T20:09:58.047033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:25.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.186093+0000","last_change":"2024-04-26T20:07:34.769739+0000","last_active":"2024-04-26T20:09:58.186093+0000","last_peered":"2024-04-26T20:09:58.186093+0000","last_clean":"2024-04-26T20:09:58.186093+0000","last_became_active":"2024-04-26T20:07:34.769383+0000","last_became_peered":"2024-04-26T20:07:34.769383+0000","last_unstale":"2024-04-26T20:09:58.186093+0000","last_undegraded":"2024-04-26T20:09:58.186093+0000","last_fullsized":"2024-04-26T20:09:58.186093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:08.771590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:54.828723+0000","last_change":"2024-04-26T20:07:34.772010+0000","last_active":"2024-04-26T20:09:54.828723+0000","last_peered":"2024-04-26T20:09:54.828723+0000","last_clean":"2024-04-26T20:09:54.828723+0000","last_became_active":"2024-04-26T20:07:34.771682+0000","last_became_peered":"2024-04-26T20:07:34.771682+0000","last_unstale":"2024-04-26T20:09:54.828723+0000","last_undegraded":"2024-04-26T20:09:54.828723+0000","last_fullsized":"2024-04-26T20:09:54.828723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:57.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.211446+0000","last_change":"2024-04-26T20:07:34.757547+0000","last_active":"2024-04-26T20:09:55.211446+0000","last_peered":"2024-04-26T20:09:55.211446+0000","last_clean":"2024-04-26T20:09:55.211446+0000","last_became_active":"2024-04-26T20:07:34.757180+0000","last_became_peered":"2024-04-26T20:07:34.757180+0000","last_unstale":"2024-04-26T20:09:55.211446+0000","last_undegraded":"2024-04-26T20:09:55.211446+0000","last_fullsized":"2024-04-26T20:09:55.211446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:32.365944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.610166+0000","last_change":"2024-04-26T20:07:34.767363+0000","last_active":"2024-04-26T20:09:52.610166+0000","last_peered":"2024-04-26T20:09:52.610166+0000","last_clean":"2024-04-26T20:09:52.610166+0000","last_became_active":"2024-04-26T20:07:34.766909+0000","last_became_peered":"2024-04-26T20:07:34.766909+0000","last_unstale":"2024-04-26T20:09:52.610166+0000","last_undegraded":"2024-04-26T20:09:52.610166+0000","last_fullsized":"2024-04-26T20:09:52.610166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:32:31.500141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.612056+0000","last_change":"2024-04-26T20:07:38.823959+0000","last_active":"2024-04-26T20:10:07.612056+0000","last_peered":"2024-04-26T20:10:07.612056+0000","last_clean":"2024-04-26T20:10:07.612056+0000","last_became_active":"2024-04-26T20:07:38.823524+0000","last_became_peered":"2024-04-26T20:07:38.823524+0000","last_unstale":"2024-04-26T20:10:07.612056+0000","last_undegraded":"2024-04-26T20:10:07.612056+0000","last_fullsized":"2024-04-26T20:10:07.612056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:23.348376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'9","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:35.712160+0000","last_change":"2024-04-26T20:07:31.721324+0000","last_active":"2024-04-26T20:09:35.712160+0000","last_peered":"2024-04-26T20:09:35.712160+0000","last_clean":"2024-04-26T20:09:35.712160+0000","last_became_active":"2024-04-26T20:07:31.720140+0000","last_became_peered":"2024-04-26T20:07:31.720140+0000","last_unstale":"2024-04-26T20:09:35.712160+0000","last_undegraded":"2024-04-26T20:09:35.712160+0000","last_fullsized":"2024-04-26T20:09:35.712160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:27.636660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:34.575396+0000","last_change":"2024-04-26T20:07:34.760387+0000","last_active":"2024-04-26T20:09:34.575396+0000","last_peered":"2024-04-26T20:09:34.575396+0000","last_clean":"2024-04-26T20:09:34.575396+0000","last_became_active":"2024-04-26T20:07:34.760042+0000","last_became_peered":"2024-04-26T20:07:34.760042+0000","last_unstale":"2024-04-26T20:09:34.575396+0000","last_undegraded":"2024-04-26T20:09:34.575396+0000","last_fullsized":"2024-04-26T20:09:34.575396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:39.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.015378+0000","last_change":"2024-04-26T20:08:15.587181+0000","last_active":"2024-04-26T20:10:07.015378+0000","last_peered":"2024-04-26T20:10:07.015378+0000","last_clean":"2024-04-26T20:10:07.015378+0000","last_became_active":"2024-04-26T20:08:15.586847+0000","last_became_peered":"2024-04-26T20:08:15.586847+0000","last_unstale":"2024-04-26T20:10:07.015378+0000","last_undegraded":"2024-04-26T20:10:07.015378+0000","last_fullsized":"2024-04-26T20:10:07.015378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:12.495523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:34.575467+0000","last_change":"2024-04-26T20:07:31.711623+0000","last_active":"2024-04-26T20:09:34.575467+0000","last_peered":"2024-04-26T20:09:34.575467+0000","last_clean":"2024-04-26T20:09:34.575467+0000","last_became_active":"2024-04-26T20:07:31.710488+0000","last_became_peered":"2024-04-26T20:07:31.710488+0000","last_unstale":"2024-04-26T20:09:34.575467+0000","last_undegraded":"2024-04-26T20:09:34.575467+0000","last_fullsized":"2024-04-26T20:09:34.575467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:04.747170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.045052+0000","last_change":"2024-04-26T20:09:15.801039+0000","last_active":"2024-04-26T20:09:43.045052+0000","last_peered":"2024-04-26T20:09:43.045052+0000","last_clean":"2024-04-26T20:09:43.045052+0000","last_became_active":"2024-04-26T20:09:15.800049+0000","last_became_peered":"2024-04-26T20:09:15.800049+0000","last_unstale":"2024-04-26T20:09:43.045052+0000","last_undegraded":"2024-04-26T20:09:43.045052+0000","last_fullsized":"2024-04-26T20:09:43.045052+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:49.168625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,9],"acting":[0,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.016057+0000","last_change":"2024-04-26T20:07:38.822443+0000","last_active":"2024-04-26T20:10:12.016057+0000","last_peered":"2024-04-26T20:10:12.016057+0000","last_clean":"2024-04-26T20:10:12.016057+0000","last_became_active":"2024-04-26T20:07:38.822046+0000","last_became_peered":"2024-04-26T20:07:38.822046+0000","last_unstale":"2024-04-26T20:10:12.016057+0000","last_undegraded":"2024-04-26T20:10:12.016057+0000","last_fullsized":"2024-04-26T20:10:12.016057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:26.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457125+0000","last_change":"2024-04-26T20:07:31.715319+0000","last_active":"2024-04-26T20:10:05.457125+0000","last_peered":"2024-04-26T20:10:05.457125+0000","last_clean":"2024-04-26T20:10:05.457125+0000","last_became_active":"2024-04-26T20:07:31.714915+0000","last_became_peered":"2024-04-26T20:07:31.714915+0000","last_unstale":"2024-04-26T20:10:05.457125+0000","last_undegraded":"2024-04-26T20:10:05.457125+0000","last_fullsized":"2024-04-26T20:10:05.457125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:27.898652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:08.187267+0000","last_change":"2024-04-26T20:07:34.753422+0000","last_active":"2024-04-26T20:10:08.187267+0000","last_peered":"2024-04-26T20:10:08.187267+0000","last_clean":"2024-04-26T20:10:08.187267+0000","last_became_active":"2024-04-26T20:07:34.753009+0000","last_became_peered":"2024-04-26T20:07:34.753009+0000","last_unstale":"2024-04-26T20:10:08.187267+0000","last_undegraded":"2024-04-26T20:10:08.187267+0000","last_fullsized":"2024-04-26T20:10:08.187267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.369715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.016037+0000","last_change":"2024-04-26T20:07:38.830239+0000","last_active":"2024-04-26T20:10:12.016037+0000","last_peered":"2024-04-26T20:10:12.016037+0000","last_clean":"2024-04-26T20:10:12.016037+0000","last_became_active":"2024-04-26T20:07:38.829509+0000","last_became_peered":"2024-04-26T20:07:38.829509+0000","last_unstale":"2024-04-26T20:10:12.016037+0000","last_undegraded":"2024-04-26T20:10:12.016037+0000","last_fullsized":"2024-04-26T20:10:12.016037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:53.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788907+0000","last_change":"2024-04-26T20:08:14.576129+0000","last_active":"2024-04-26T20:09:15.788907+0000","last_peered":"2024-04-26T20:09:15.788907+0000","last_clean":"2024-04-26T20:09:15.788907+0000","last_became_active":"2024-04-26T20:08:14.573168+0000","last_became_peered":"2024-04-26T20:08:14.573168+0000","last_unstale":"2024-04-26T20:09:15.788907+0000","last_undegraded":"2024-04-26T20:09:15.788907+0000","last_fullsized":"2024-04-26T20:09:15.788907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:38.234154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645639+0000","last_change":"2024-04-26T20:07:34.749270+0000","last_active":"2024-04-26T20:10:07.645639+0000","last_peered":"2024-04-26T20:10:07.645639+0000","last_clean":"2024-04-26T20:10:07.645639+0000","last_became_active":"2024-04-26T20:07:34.748585+0000","last_became_peered":"2024-04-26T20:07:34.748585+0000","last_unstale":"2024-04-26T20:10:07.645639+0000","last_undegraded":"2024-04-26T20:10:07.645639+0000","last_fullsized":"2024-04-26T20:10:07.645639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:03.091720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.612582+0000","last_change":"2024-04-26T20:07:38.806247+0000","last_active":"2024-04-26T20:10:12.612582+0000","last_peered":"2024-04-26T20:10:12.612582+0000","last_clean":"2024-04-26T20:10:12.612582+0000","last_became_active":"2024-04-26T20:07:38.805748+0000","last_became_peered":"2024-04-26T20:07:38.805748+0000","last_unstale":"2024-04-26T20:10:12.612582+0000","last_undegraded":"2024-04-26T20:10:12.612582+0000","last_fullsized":"2024-04-26T20:10:12.612582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:16.962494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":308,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:06.626389+0000","last_change":"2024-04-26T20:07:31.713618+0000","last_active":"2024-04-26T20:13:06.626389+0000","last_peered":"2024-04-26T20:13:06.626389+0000","last_clean":"2024-04-26T20:13:06.626389+0000","last_became_active":"2024-04-26T20:07:31.713259+0000","last_became_peered":"2024-04-26T20:07:31.713259+0000","last_unstale":"2024-04-26T20:13:06.626389+0000","last_undegraded":"2024-04-26T20:13:06.626389+0000","last_fullsized":"2024-04-26T20:13:06.626389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:25.215087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":259,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457088+0000","last_change":"2024-04-26T20:08:14.585688+0000","last_active":"2024-04-26T20:10:05.457088+0000","last_peered":"2024-04-26T20:10:05.457088+0000","last_clean":"2024-04-26T20:10:05.457088+0000","last_became_active":"2024-04-26T20:08:14.585337+0000","last_became_peered":"2024-04-26T20:08:14.585337+0000","last_unstale":"2024-04-26T20:10:05.457088+0000","last_undegraded":"2024-04-26T20:10:05.457088+0000","last_fullsized":"2024-04-26T20:10:05.457088+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:03.515457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.612549+0000","last_change":"2024-04-26T20:07:38.829772+0000","last_active":"2024-04-26T20:10:12.612549+0000","last_peered":"2024-04-26T20:10:12.612549+0000","last_clean":"2024-04-26T20:10:12.612549+0000","last_became_active":"2024-04-26T20:07:38.829327+0000","last_became_peered":"2024-04-26T20:07:38.829327+0000","last_unstale":"2024-04-26T20:10:12.612549+0000","last_undegraded":"2024-04-26T20:10:12.612549+0000","last_fullsized":"2024-04-26T20:10:12.612549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:37.252863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.971049+0000","last_change":"2024-04-26T20:07:31.728654+0000","last_active":"2024-04-26T20:10:02.971049+0000","last_peered":"2024-04-26T20:10:02.971049+0000","last_clean":"2024-04-26T20:10:02.971049+0000","last_became_active":"2024-04-26T20:07:31.728278+0000","last_became_peered":"2024-04-26T20:07:31.728278+0000","last_unstale":"2024-04-26T20:10:02.971049+0000","last_undegraded":"2024-04-26T20:10:02.971049+0000","last_fullsized":"2024-04-26T20:10:02.971049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645617+0000","last_change":"2024-04-26T20:07:34.739575+0000","last_active":"2024-04-26T20:10:07.645617+0000","last_peered":"2024-04-26T20:10:07.645617+0000","last_clean":"2024-04-26T20:10:07.645617+0000","last_became_active":"2024-04-26T20:07:34.739264+0000","last_became_peered":"2024-04-26T20:07:34.739264+0000","last_unstale":"2024-04-26T20:10:07.645617+0000","last_undegraded":"2024-04-26T20:10:07.645617+0000","last_fullsized":"2024-04-26T20:10:07.645617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:25.130064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827749+0000","last_change":"2024-04-26T20:07:38.822669+0000","last_active":"2024-04-26T20:09:44.827749+0000","last_peered":"2024-04-26T20:09:44.827749+0000","last_clean":"2024-04-26T20:09:44.827749+0000","last_became_active":"2024-04-26T20:07:38.822197+0000","last_became_peered":"2024-04-26T20:07:38.822197+0000","last_unstale":"2024-04-26T20:09:44.827749+0000","last_undegraded":"2024-04-26T20:09:44.827749+0000","last_fullsized":"2024-04-26T20:09:44.827749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:56.782794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"95'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.646035+0000","last_change":"2024-04-26T20:07:31.723527+0000","last_active":"2024-04-26T20:09:57.646035+0000","last_peered":"2024-04-26T20:09:57.646035+0000","last_clean":"2024-04-26T20:09:57.646035+0000","last_became_active":"2024-04-26T20:07:31.722574+0000","last_became_peered":"2024-04-26T20:07:31.722574+0000","last_unstale":"2024-04-26T20:09:57.646035+0000","last_undegraded":"2024-04-26T20:09:57.646035+0000","last_fullsized":"2024-04-26T20:09:57.646035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.521613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.580024+0000","last_change":"2024-04-26T20:07:34.771459+0000","last_active":"2024-04-26T20:09:59.580024+0000","last_peered":"2024-04-26T20:09:59.580024+0000","last_clean":"2024-04-26T20:09:59.580024+0000","last_became_active":"2024-04-26T20:07:34.771124+0000","last_became_peered":"2024-04-26T20:07:34.771124+0000","last_unstale":"2024-04-26T20:09:59.580024+0000","last_undegraded":"2024-04-26T20:09:59.580024+0000","last_fullsized":"2024-04-26T20:09:59.580024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.560440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.043976+0000","last_change":"2024-04-26T20:07:38.838858+0000","last_active":"2024-04-26T20:09:38.043976+0000","last_peered":"2024-04-26T20:09:38.043976+0000","last_clean":"2024-04-26T20:09:38.043976+0000","last_became_active":"2024-04-26T20:07:38.838125+0000","last_became_peered":"2024-04-26T20:07:38.838125+0000","last_unstale":"2024-04-26T20:09:38.043976+0000","last_undegraded":"2024-04-26T20:09:38.043976+0000","last_fullsized":"2024-04-26T20:09:38.043976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:01.856633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.644435+0000","last_change":"2024-04-26T20:07:31.710690+0000","last_active":"2024-04-26T20:09:57.644435+0000","last_peered":"2024-04-26T20:09:57.644435+0000","last_clean":"2024-04-26T20:09:57.644435+0000","last_became_active":"2024-04-26T20:07:31.710353+0000","last_became_peered":"2024-04-26T20:07:31.710353+0000","last_unstale":"2024-04-26T20:09:57.644435+0000","last_undegraded":"2024-04-26T20:09:57.644435+0000","last_fullsized":"2024-04-26T20:09:57.644435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:47.442673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715632+0000","last_change":"2024-04-26T20:07:34.776023+0000","last_active":"2024-04-26T20:10:00.715632+0000","last_peered":"2024-04-26T20:10:00.715632+0000","last_clean":"2024-04-26T20:10:00.715632+0000","last_became_active":"2024-04-26T20:07:34.775571+0000","last_became_peered":"2024-04-26T20:07:34.775571+0000","last_unstale":"2024-04-26T20:10:00.715632+0000","last_undegraded":"2024-04-26T20:10:00.715632+0000","last_fullsized":"2024-04-26T20:10:00.715632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:40.573046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.580074+0000","last_change":"2024-04-26T20:07:38.820636+0000","last_active":"2024-04-26T20:09:59.580074+0000","last_peered":"2024-04-26T20:09:59.580074+0000","last_clean":"2024-04-26T20:09:59.580074+0000","last_became_active":"2024-04-26T20:07:38.819390+0000","last_became_peered":"2024-04-26T20:07:38.819390+0000","last_unstale":"2024-04-26T20:09:59.580074+0000","last_undegraded":"2024-04-26T20:09:59.580074+0000","last_fullsized":"2024-04-26T20:09:59.580074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:58.793479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'4","reported_seq":300,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:06.625753+0000","last_change":"2024-04-26T20:07:31.724192+0000","last_active":"2024-04-26T20:13:06.625753+0000","last_peered":"2024-04-26T20:13:06.625753+0000","last_clean":"2024-04-26T20:13:06.625753+0000","last_became_active":"2024-04-26T20:07:31.723816+0000","last_became_peered":"2024-04-26T20:07:31.723816+0000","last_unstale":"2024-04-26T20:13:06.625753+0000","last_undegraded":"2024-04-26T20:13:06.625753+0000","last_fullsized":"2024-04-26T20:13:06.625753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:31.707282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":255,"num_read_kb":255,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.186757+0000","last_change":"2024-04-26T20:07:34.759027+0000","last_active":"2024-04-26T20:10:03.186757+0000","last_peered":"2024-04-26T20:10:03.186757+0000","last_clean":"2024-04-26T20:10:03.186757+0000","last_became_active":"2024-04-26T20:07:34.758689+0000","last_became_peered":"2024-04-26T20:07:34.758689+0000","last_unstale":"2024-04-26T20:10:03.186757+0000","last_undegraded":"2024-04-26T20:10:03.186757+0000","last_fullsized":"2024-04-26T20:10:03.186757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:29.110222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715598+0000","last_change":"2024-04-26T20:07:38.819000+0000","last_active":"2024-04-26T20:10:00.715598+0000","last_peered":"2024-04-26T20:10:00.715598+0000","last_clean":"2024-04-26T20:10:00.715598+0000","last_became_active":"2024-04-26T20:07:38.818526+0000","last_became_peered":"2024-04-26T20:07:38.818526+0000","last_unstale":"2024-04-26T20:10:00.715598+0000","last_undegraded":"2024-04-26T20:10:00.715598+0000","last_fullsized":"2024-04-26T20:10:00.715598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:28.830096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268278+0000","last_change":"2024-04-26T20:07:31.712952+0000","last_active":"2024-04-26T20:09:58.268278+0000","last_peered":"2024-04-26T20:09:58.268278+0000","last_clean":"2024-04-26T20:09:58.268278+0000","last_became_active":"2024-04-26T20:07:31.712081+0000","last_became_peered":"2024-04-26T20:07:31.712081+0000","last_unstale":"2024-04-26T20:09:58.268278+0000","last_undegraded":"2024-04-26T20:09:58.268278+0000","last_fullsized":"2024-04-26T20:09:58.268278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:46.923536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.047921+0000","last_change":"2024-04-26T20:09:16.367690+0000","last_active":"2024-04-26T20:10:03.047921+0000","last_peered":"2024-04-26T20:10:03.047921+0000","last_clean":"2024-04-26T20:10:03.047921+0000","last_became_active":"2024-04-26T20:09:16.366496+0000","last_became_peered":"2024-04-26T20:09:16.366496+0000","last_unstale":"2024-04-26T20:10:03.047921+0000","last_undegraded":"2024-04-26T20:10:03.047921+0000","last_fullsized":"2024-04-26T20:10:03.047921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:36.154062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.014903+0000","last_change":"2024-04-26T20:07:38.822520+0000","last_active":"2024-04-26T20:10:02.014903+0000","last_peered":"2024-04-26T20:10:02.014903+0000","last_clean":"2024-04-26T20:10:02.014903+0000","last_became_active":"2024-04-26T20:07:38.822056+0000","last_became_peered":"2024-04-26T20:07:38.822056+0000","last_unstale":"2024-04-26T20:10:02.014903+0000","last_undegraded":"2024-04-26T20:10:02.014903+0000","last_fullsized":"2024-04-26T20:10:02.014903+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.324996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"95'4","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.837386+0000","last_change":"2024-04-26T20:07:31.724015+0000","last_active":"2024-04-26T20:09:55.837386+0000","last_peered":"2024-04-26T20:09:55.837386+0000","last_clean":"2024-04-26T20:09:55.837386+0000","last_became_active":"2024-04-26T20:07:31.723686+0000","last_became_peered":"2024-04-26T20:07:31.723686+0000","last_unstale":"2024-04-26T20:09:55.837386+0000","last_undegraded":"2024-04-26T20:09:55.837386+0000","last_fullsized":"2024-04-26T20:09:55.837386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:55.925280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.186136+0000","last_change":"2024-04-26T20:07:34.750276+0000","last_active":"2024-04-26T20:09:58.186136+0000","last_peered":"2024-04-26T20:09:58.186136+0000","last_clean":"2024-04-26T20:09:58.186136+0000","last_became_active":"2024-04-26T20:07:34.749716+0000","last_became_peered":"2024-04-26T20:07:34.749716+0000","last_unstale":"2024-04-26T20:09:58.186136+0000","last_undegraded":"2024-04-26T20:09:58.186136+0000","last_fullsized":"2024-04-26T20:09:58.186136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:22.429780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715645+0000","last_change":"2024-04-26T20:07:38.815636+0000","last_active":"2024-04-26T20:10:00.715645+0000","last_peered":"2024-04-26T20:10:00.715645+0000","last_clean":"2024-04-26T20:10:00.715645+0000","last_became_active":"2024-04-26T20:07:38.815278+0000","last_became_peered":"2024-04-26T20:07:38.815278+0000","last_unstale":"2024-04-26T20:10:00.715645+0000","last_undegraded":"2024-04-26T20:10:00.715645+0000","last_fullsized":"2024-04-26T20:10:00.715645+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:27.522132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'36","reported_seq":598,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:07.627554+0000","last_change":"2024-04-26T20:07:31.711393+0000","last_active":"2024-04-26T20:13:07.627554+0000","last_peered":"2024-04-26T20:13:07.627554+0000","last_clean":"2024-04-26T20:13:07.627554+0000","last_became_active":"2024-04-26T20:07:31.710998+0000","last_became_peered":"2024-04-26T20:07:31.710998+0000","last_unstale":"2024-04-26T20:13:07.627554+0000","last_undegraded":"2024-04-26T20:13:07.627554+0000","last_fullsized":"2024-04-26T20:13:07.627554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:00.674950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60739,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":521,"num_read_kb":320,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.676036+0000","last_change":"2024-04-26T20:07:34.752774+0000","last_active":"2024-04-26T20:09:58.676036+0000","last_peered":"2024-04-26T20:09:58.676036+0000","last_clean":"2024-04-26T20:09:58.676036+0000","last_became_active":"2024-04-26T20:07:34.752471+0000","last_became_peered":"2024-04-26T20:07:34.752471+0000","last_unstale":"2024-04-26T20:09:58.676036+0000","last_undegraded":"2024-04-26T20:09:58.676036+0000","last_fullsized":"2024-04-26T20:09:58.676036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.646231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645658+0000","last_change":"2024-04-26T20:07:38.830322+0000","last_active":"2024-04-26T20:10:07.645658+0000","last_peered":"2024-04-26T20:10:07.645658+0000","last_clean":"2024-04-26T20:10:07.645658+0000","last_became_active":"2024-04-26T20:07:38.829910+0000","last_became_peered":"2024-04-26T20:07:38.829910+0000","last_unstale":"2024-04-26T20:10:07.645658+0000","last_undegraded":"2024-04-26T20:10:07.645658+0000","last_fullsized":"2024-04-26T20:10:07.645658+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:01.979793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"95'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.610124+0000","last_change":"2024-04-26T20:07:31.712054+0000","last_active":"2024-04-26T20:09:52.610124+0000","last_peered":"2024-04-26T20:09:52.610124+0000","last_clean":"2024-04-26T20:09:52.610124+0000","last_became_active":"2024-04-26T20:07:31.711618+0000","last_became_peered":"2024-04-26T20:07:31.711618+0000","last_unstale":"2024-04-26T20:09:52.610124+0000","last_undegraded":"2024-04-26T20:09:52.610124+0000","last_fullsized":"2024-04-26T20:09:52.610124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:46.465777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.646073+0000","last_change":"2024-04-26T20:07:34.772993+0000","last_active":"2024-04-26T20:09:57.646073+0000","last_peered":"2024-04-26T20:09:57.646073+0000","last_clean":"2024-04-26T20:09:57.646073+0000","last_became_active":"2024-04-26T20:07:34.772587+0000","last_became_peered":"2024-04-26T20:07:34.772587+0000","last_unstale":"2024-04-26T20:09:57.646073+0000","last_undegraded":"2024-04-26T20:09:57.646073+0000","last_fullsized":"2024-04-26T20:09:57.646073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:38.459869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.829340+0000","last_change":"2024-04-26T20:08:15.588307+0000","last_active":"2024-04-26T20:09:59.829340+0000","last_peered":"2024-04-26T20:09:59.829340+0000","last_clean":"2024-04-26T20:09:59.829340+0000","last_became_active":"2024-04-26T20:08:15.587978+0000","last_became_peered":"2024-04-26T20:08:15.587978+0000","last_unstale":"2024-04-26T20:09:59.829340+0000","last_undegraded":"2024-04-26T20:09:59.829340+0000","last_fullsized":"2024-04-26T20:09:59.829340+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:36.491892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643976+0000","last_change":"2024-04-26T20:07:31.720938+0000","last_active":"2024-04-26T20:09:52.643976+0000","last_peered":"2024-04-26T20:09:52.643976+0000","last_clean":"2024-04-26T20:09:52.643976+0000","last_became_active":"2024-04-26T20:07:31.720579+0000","last_became_peered":"2024-04-26T20:07:31.720579+0000","last_unstale":"2024-04-26T20:09:52.643976+0000","last_undegraded":"2024-04-26T20:09:52.643976+0000","last_fullsized":"2024-04-26T20:09:52.643976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:55.737976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268260+0000","last_change":"2024-04-26T20:07:34.764802+0000","last_active":"2024-04-26T20:09:58.268260+0000","last_peered":"2024-04-26T20:09:58.268260+0000","last_clean":"2024-04-26T20:09:58.268260+0000","last_became_active":"2024-04-26T20:07:34.763468+0000","last_became_peered":"2024-04-26T20:07:34.763468+0000","last_unstale":"2024-04-26T20:09:58.268260+0000","last_undegraded":"2024-04-26T20:09:58.268260+0000","last_fullsized":"2024-04-26T20:09:58.268260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:23.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.716233+0000","last_change":"2024-04-26T20:07:38.830392+0000","last_active":"2024-04-26T20:10:05.716233+0000","last_peered":"2024-04-26T20:10:05.716233+0000","last_clean":"2024-04-26T20:10:05.716233+0000","last_became_active":"2024-04-26T20:07:38.822964+0000","last_became_peered":"2024-04-26T20:07:38.822964+0000","last_unstale":"2024-04-26T20:10:05.716233+0000","last_undegraded":"2024-04-26T20:10:05.716233+0000","last_fullsized":"2024-04-26T20:10:05.716233+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:17.992368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'3","reported_seq":300,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:05.825283+0000","last_change":"2024-04-26T20:07:31.710567+0000","last_active":"2024-04-26T20:13:05.825283+0000","last_peered":"2024-04-26T20:13:05.825283+0000","last_clean":"2024-04-26T20:13:05.825283+0000","last_became_active":"2024-04-26T20:07:31.710197+0000","last_became_peered":"2024-04-26T20:07:31.710197+0000","last_unstale":"2024-04-26T20:13:05.825283+0000","last_undegraded":"2024-04-26T20:13:05.825283+0000","last_fullsized":"2024-04-26T20:13:05.825283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:10.648320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":257,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268229+0000","last_change":"2024-04-26T20:07:34.743237+0000","last_active":"2024-04-26T20:09:58.268229+0000","last_peered":"2024-04-26T20:09:58.268229+0000","last_clean":"2024-04-26T20:09:58.268229+0000","last_became_active":"2024-04-26T20:07:34.742146+0000","last_became_peered":"2024-04-26T20:07:34.742146+0000","last_unstale":"2024-04-26T20:09:58.268229+0000","last_undegraded":"2024-04-26T20:09:58.268229+0000","last_fullsized":"2024-04-26T20:09:58.268229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:15.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.647738+0000","last_change":"2024-04-26T20:08:15.591942+0000","last_active":"2024-04-26T20:10:07.647738+0000","last_peered":"2024-04-26T20:10:07.647738+0000","last_clean":"2024-04-26T20:10:07.647738+0000","last_became_active":"2024-04-26T20:08:15.590842+0000","last_became_peered":"2024-04-26T20:08:15.590842+0000","last_unstale":"2024-04-26T20:10:07.647738+0000","last_undegraded":"2024-04-26T20:10:07.647738+0000","last_fullsized":"2024-04-26T20:10:07.647738+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:44.344152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"95'15","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.456016+0000","last_change":"2024-04-26T20:07:31.716877+0000","last_active":"2024-04-26T20:09:55.456016+0000","last_peered":"2024-04-26T20:09:55.456016+0000","last_clean":"2024-04-26T20:09:55.456016+0000","last_became_active":"2024-04-26T20:07:31.716560+0000","last_became_peered":"2024-04-26T20:07:31.716560+0000","last_unstale":"2024-04-26T20:09:55.456016+0000","last_undegraded":"2024-04-26T20:09:55.456016+0000","last_fullsized":"2024-04-26T20:09:55.456016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:08:47.680991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.837358+0000","last_change":"2024-04-26T20:07:34.749893+0000","last_active":"2024-04-26T20:09:55.837358+0000","last_peered":"2024-04-26T20:09:55.837358+0000","last_clean":"2024-04-26T20:09:55.837358+0000","last_became_active":"2024-04-26T20:07:34.748535+0000","last_became_peered":"2024-04-26T20:07:34.748535+0000","last_unstale":"2024-04-26T20:09:55.837358+0000","last_undegraded":"2024-04-26T20:09:55.837358+0000","last_fullsized":"2024-04-26T20:09:55.837358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:32.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643963+0000","last_change":"2024-04-26T20:07:38.824046+0000","last_active":"2024-04-26T20:09:52.643963+0000","last_peered":"2024-04-26T20:09:52.643963+0000","last_clean":"2024-04-26T20:09:52.643963+0000","last_became_active":"2024-04-26T20:07:38.823549+0000","last_became_peered":"2024-04-26T20:07:38.823549+0000","last_unstale":"2024-04-26T20:09:52.643963+0000","last_undegraded":"2024-04-26T20:09:52.643963+0000","last_fullsized":"2024-04-26T20:09:52.643963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:26.218201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:50.455410+0000","last_change":"2024-04-26T20:07:31.711968+0000","last_active":"2024-04-26T20:09:50.455410+0000","last_peered":"2024-04-26T20:09:50.455410+0000","last_clean":"2024-04-26T20:09:50.455410+0000","last_became_active":"2024-04-26T20:07:31.710982+0000","last_became_peered":"2024-04-26T20:07:31.710982+0000","last_unstale":"2024-04-26T20:09:50.455410+0000","last_undegraded":"2024-04-26T20:09:50.455410+0000","last_fullsized":"2024-04-26T20:09:50.455410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:29.526536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.828290+0000","last_change":"2024-04-26T20:07:34.758101+0000","last_active":"2024-04-26T20:09:49.828290+0000","last_peered":"2024-04-26T20:09:49.828290+0000","last_clean":"2024-04-26T20:09:49.828290+0000","last_became_active":"2024-04-26T20:07:34.757747+0000","last_became_peered":"2024-04-26T20:07:34.757747+0000","last_unstale":"2024-04-26T20:09:49.828290+0000","last_undegraded":"2024-04-26T20:09:49.828290+0000","last_fullsized":"2024-04-26T20:09:49.828290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.337040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:45.454881+0000","last_change":"2024-04-26T20:07:39.312976+0000","last_active":"2024-04-26T20:09:45.454881+0000","last_peered":"2024-04-26T20:09:45.454881+0000","last_clean":"2024-04-26T20:09:45.454881+0000","last_became_active":"2024-04-26T20:07:39.312443+0000","last_became_peered":"2024-04-26T20:07:39.312443+0000","last_unstale":"2024-04-26T20:09:45.454881+0000","last_undegraded":"2024-04-26T20:09:45.454881+0000","last_fullsized":"2024-04-26T20:09:45.454881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:23.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:35.712065+0000","last_change":"2024-04-26T20:08:14.587592+0000","last_active":"2024-04-26T20:09:35.712065+0000","last_peered":"2024-04-26T20:09:35.712065+0000","last_clean":"2024-04-26T20:09:35.712065+0000","last_became_active":"2024-04-26T20:08:14.587157+0000","last_became_peered":"2024-04-26T20:08:14.587157+0000","last_unstale":"2024-04-26T20:09:35.712065+0000","last_undegraded":"2024-04-26T20:09:35.712065+0000","last_fullsized":"2024-04-26T20:09:35.712065+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:22.184416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.969101+0000","last_change":"2024-04-26T20:07:34.764376+0000","last_active":"2024-04-26T20:09:42.969101+0000","last_peered":"2024-04-26T20:09:42.969101+0000","last_clean":"2024-04-26T20:09:42.969101+0000","last_became_active":"2024-04-26T20:07:34.764040+0000","last_became_peered":"2024-04-26T20:07:34.764040+0000","last_unstale":"2024-04-26T20:09:42.969101+0000","last_undegraded":"2024-04-26T20:09:42.969101+0000","last_fullsized":"2024-04-26T20:09:42.969101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:43.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644412+0000","last_change":"2024-04-26T20:07:38.837075+0000","last_active":"2024-04-26T20:09:47.644412+0000","last_peered":"2024-04-26T20:09:47.644412+0000","last_clean":"2024-04-26T20:09:47.644412+0000","last_became_active":"2024-04-26T20:07:38.835912+0000","last_became_peered":"2024-04-26T20:07:38.835912+0000","last_unstale":"2024-04-26T20:09:47.644412+0000","last_undegraded":"2024-04-26T20:09:47.644412+0000","last_fullsized":"2024-04-26T20:09:47.644412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.853262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.266109+0000","last_change":"2024-04-26T20:07:31.728087+0000","last_active":"2024-04-26T20:09:38.266109+0000","last_peered":"2024-04-26T20:09:38.266109+0000","last_clean":"2024-04-26T20:09:38.266109+0000","last_became_active":"2024-04-26T20:07:31.727688+0000","last_became_peered":"2024-04-26T20:07:31.727688+0000","last_unstale":"2024-04-26T20:09:38.266109+0000","last_undegraded":"2024-04-26T20:09:38.266109+0000","last_fullsized":"2024-04-26T20:09:38.266109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:16.220049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.969049+0000","last_change":"2024-04-26T20:07:34.765161+0000","last_active":"2024-04-26T20:09:42.969049+0000","last_peered":"2024-04-26T20:09:42.969049+0000","last_clean":"2024-04-26T20:09:42.969049+0000","last_became_active":"2024-04-26T20:07:34.764789+0000","last_became_peered":"2024-04-26T20:07:34.764789+0000","last_unstale":"2024-04-26T20:09:42.969049+0000","last_undegraded":"2024-04-26T20:09:42.969049+0000","last_fullsized":"2024-04-26T20:09:42.969049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:52.417157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.864999+0000","last_change":"2024-04-26T20:07:39.313159+0000","last_active":"2024-04-26T20:09:49.864999+0000","last_peered":"2024-04-26T20:09:49.864999+0000","last_clean":"2024-04-26T20:09:49.864999+0000","last_became_active":"2024-04-26T20:07:39.312637+0000","last_became_peered":"2024-04-26T20:07:39.312637+0000","last_unstale":"2024-04-26T20:09:49.864999+0000","last_undegraded":"2024-04-26T20:09:49.864999+0000","last_fullsized":"2024-04-26T20:09:49.864999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:10.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.642222+0000","last_change":"2024-04-26T20:08:14.586352+0000","last_active":"2024-04-26T20:09:37.642222+0000","last_peered":"2024-04-26T20:09:37.642222+0000","last_clean":"2024-04-26T20:09:37.642222+0000","last_became_active":"2024-04-26T20:08:14.585975+0000","last_became_peered":"2024-04-26T20:08:14.585975+0000","last_unstale":"2024-04-26T20:09:37.642222+0000","last_undegraded":"2024-04-26T20:09:37.642222+0000","last_fullsized":"2024-04-26T20:09:37.642222+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:27.676683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'4","reported_seq":298,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:05.824458+0000","last_change":"2024-04-26T20:07:31.726272+0000","last_active":"2024-04-26T20:13:05.824458+0000","last_peered":"2024-04-26T20:13:05.824458+0000","last_clean":"2024-04-26T20:13:05.824458+0000","last_became_active":"2024-04-26T20:07:31.725889+0000","last_became_peered":"2024-04-26T20:07:31.725889+0000","last_unstale":"2024-04-26T20:13:05.824458+0000","last_undegraded":"2024-04-26T20:13:05.824458+0000","last_fullsized":"2024-04-26T20:13:05.824458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:03.091138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.266670+0000","last_change":"2024-04-26T20:07:38.815171+0000","last_active":"2024-04-26T20:09:43.266670+0000","last_peered":"2024-04-26T20:09:43.266670+0000","last_clean":"2024-04-26T20:09:43.266670+0000","last_became_active":"2024-04-26T20:07:38.814769+0000","last_became_peered":"2024-04-26T20:07:38.814769+0000","last_unstale":"2024-04-26T20:09:43.266670+0000","last_undegraded":"2024-04-26T20:09:43.266670+0000","last_fullsized":"2024-04-26T20:09:43.266670+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.506176+0000","last_change":"2024-04-26T20:07:34.766538+0000","last_active":"2024-04-26T20:09:38.506176+0000","last_peered":"2024-04-26T20:09:38.506176+0000","last_clean":"2024-04-26T20:09:38.506176+0000","last_became_active":"2024-04-26T20:07:34.761651+0000","last_became_peered":"2024-04-26T20:07:34.761651+0000","last_unstale":"2024-04-26T20:09:38.506176+0000","last_undegraded":"2024-04-26T20:09:38.506176+0000","last_fullsized":"2024-04-26T20:09:38.506176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:11.213046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'3","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.012132+0000","last_change":"2024-04-26T20:07:31.720351+0000","last_active":"2024-04-26T20:09:37.012132+0000","last_peered":"2024-04-26T20:09:37.012132+0000","last_clean":"2024-04-26T20:09:37.012132+0000","last_became_active":"2024-04-26T20:07:31.719966+0000","last_became_peered":"2024-04-26T20:07:31.719966+0000","last_unstale":"2024-04-26T20:09:37.012132+0000","last_undegraded":"2024-04-26T20:09:37.012132+0000","last_fullsized":"2024-04-26T20:09:37.012132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:17.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:28.271807+0000","last_change":"2024-04-26T20:05:56.337783+0000","last_active":"2024-04-26T20:10:28.271807+0000","last_peered":"2024-04-26T20:10:28.271807+0000","last_clean":"2024-04-26T20:10:28.271807+0000","last_became_active":"2024-04-26T20:05:56.337301+0000","last_became_peered":"2024-04-26T20:05:56.337301+0000","last_unstale":"2024-04-26T20:10:28.271807+0000","last_undegraded":"2024-04-26T20:10:28.271807+0000","last_fullsized":"2024-04-26T20:10:28.271807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_clean_scrub_stamp":"2024-04-26T20:01:58.332403+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:45.808606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643997+0000","last_change":"2024-04-26T20:07:39.313210+0000","last_active":"2024-04-26T20:09:52.643997+0000","last_peered":"2024-04-26T20:09:52.643997+0000","last_clean":"2024-04-26T20:09:52.643997+0000","last_became_active":"2024-04-26T20:07:39.312575+0000","last_became_peered":"2024-04-26T20:07:39.312575+0000","last_unstale":"2024-04-26T20:09:52.643997+0000","last_undegraded":"2024-04-26T20:09:52.643997+0000","last_fullsized":"2024-04-26T20:09:52.643997+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:02.877951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.643273+0000","last_change":"2024-04-26T20:07:34.759474+0000","last_active":"2024-04-26T20:09:42.643273+0000","last_peered":"2024-04-26T20:09:42.643273+0000","last_clean":"2024-04-26T20:09:42.643273+0000","last_became_active":"2024-04-26T20:07:34.759127+0000","last_became_peered":"2024-04-26T20:07:34.759127+0000","last_unstale":"2024-04-26T20:09:42.643273+0000","last_undegraded":"2024-04-26T20:09:42.643273+0000","last_fullsized":"2024-04-26T20:09:42.643273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.874866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:33.265636+0000","last_change":"2024-04-26T20:07:31.710633+0000","last_active":"2024-04-26T20:09:33.265636+0000","last_peered":"2024-04-26T20:09:33.265636+0000","last_clean":"2024-04-26T20:09:33.265636+0000","last_became_active":"2024-04-26T20:07:31.710290+0000","last_became_peered":"2024-04-26T20:07:31.710290+0000","last_unstale":"2024-04-26T20:09:33.265636+0000","last_undegraded":"2024-04-26T20:09:33.265636+0000","last_fullsized":"2024-04-26T20:09:33.265636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:18.114770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827720+0000","last_change":"2024-04-26T20:07:38.829768+0000","last_active":"2024-04-26T20:09:44.827720+0000","last_peered":"2024-04-26T20:09:44.827720+0000","last_clean":"2024-04-26T20:09:44.827720+0000","last_became_active":"2024-04-26T20:07:38.829364+0000","last_became_peered":"2024-04-26T20:07:38.829364+0000","last_unstale":"2024-04-26T20:09:44.827720+0000","last_undegraded":"2024-04-26T20:09:44.827720+0000","last_fullsized":"2024-04-26T20:09:44.827720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:56.887062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:40.454431+0000","last_change":"2024-04-26T20:07:31.720296+0000","last_active":"2024-04-26T20:09:40.454431+0000","last_peered":"2024-04-26T20:09:40.454431+0000","last_clean":"2024-04-26T20:09:40.454431+0000","last_became_active":"2024-04-26T20:07:31.719918+0000","last_became_peered":"2024-04-26T20:07:31.719918+0000","last_unstale":"2024-04-26T20:09:40.454431+0000","last_undegraded":"2024-04-26T20:09:40.454431+0000","last_fullsized":"2024-04-26T20:09:40.454431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:43.783104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.643331+0000","last_change":"2024-04-26T20:07:34.770634+0000","last_active":"2024-04-26T20:09:42.643331+0000","last_peered":"2024-04-26T20:09:42.643331+0000","last_clean":"2024-04-26T20:09:42.643331+0000","last_became_active":"2024-04-26T20:07:34.770170+0000","last_became_peered":"2024-04-26T20:07:34.770170+0000","last_unstale":"2024-04-26T20:09:42.643331+0000","last_undegraded":"2024-04-26T20:09:42.643331+0000","last_fullsized":"2024-04-26T20:09:42.643331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:21.528735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.576313+0000","last_change":"2024-04-26T20:07:39.317345+0000","last_active":"2024-04-26T20:09:39.576313+0000","last_peered":"2024-04-26T20:09:39.576313+0000","last_clean":"2024-04-26T20:09:39.576313+0000","last_became_active":"2024-04-26T20:07:39.315998+0000","last_became_peered":"2024-04-26T20:07:39.315998+0000","last_unstale":"2024-04-26T20:09:39.576313+0000","last_undegraded":"2024-04-26T20:09:39.576313+0000","last_fullsized":"2024-04-26T20:09:39.576313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:18.715213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'10","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:14.756534+0000","last_change":"2024-04-26T20:07:31.729195+0000","last_active":"2024-04-26T20:10:14.756534+0000","last_peered":"2024-04-26T20:10:14.756534+0000","last_clean":"2024-04-26T20:10:14.756534+0000","last_became_active":"2024-04-26T20:07:31.728746+0000","last_became_peered":"2024-04-26T20:07:31.728746+0000","last_unstale":"2024-04-26T20:10:14.756534+0000","last_undegraded":"2024-04-26T20:10:14.756534+0000","last_fullsized":"2024-04-26T20:10:14.756534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:34.443091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577252+0000","last_change":"2024-04-26T20:07:34.765013+0000","last_active":"2024-04-26T20:09:44.577252+0000","last_peered":"2024-04-26T20:09:44.577252+0000","last_clean":"2024-04-26T20:09:44.577252+0000","last_became_active":"2024-04-26T20:07:34.764597+0000","last_became_peered":"2024-04-26T20:07:34.764597+0000","last_unstale":"2024-04-26T20:09:44.577252+0000","last_undegraded":"2024-04-26T20:09:44.577252+0000","last_fullsized":"2024-04-26T20:09:44.577252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:51.513026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609504+0000","last_change":"2024-04-26T20:07:38.805910+0000","last_active":"2024-04-26T20:09:47.609504+0000","last_peered":"2024-04-26T20:09:47.609504+0000","last_clean":"2024-04-26T20:09:47.609504+0000","last_became_active":"2024-04-26T20:07:38.804580+0000","last_became_peered":"2024-04-26T20:07:38.804580+0000","last_unstale":"2024-04-26T20:09:47.609504+0000","last_undegraded":"2024-04-26T20:09:47.609504+0000","last_fullsized":"2024-04-26T20:09:47.609504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:41.321084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.012912+0000","last_change":"2024-04-26T20:07:31.915771+0000","last_active":"2024-04-26T20:09:42.012912+0000","last_peered":"2024-04-26T20:09:42.012912+0000","last_clean":"2024-04-26T20:09:42.012912+0000","last_became_active":"2024-04-26T20:07:31.915422+0000","last_became_peered":"2024-04-26T20:07:31.915422+0000","last_unstale":"2024-04-26T20:09:42.012912+0000","last_undegraded":"2024-04-26T20:09:42.012912+0000","last_fullsized":"2024-04-26T20:09:42.012912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:33.309270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.045699+0000","last_change":"2024-04-26T20:09:16.089155+0000","last_active":"2024-04-26T20:09:48.045699+0000","last_peered":"2024-04-26T20:09:48.045699+0000","last_clean":"2024-04-26T20:09:48.045699+0000","last_became_active":"2024-04-26T20:09:16.088425+0000","last_became_peered":"2024-04-26T20:09:16.088425+0000","last_unstale":"2024-04-26T20:09:48.045699+0000","last_undegraded":"2024-04-26T20:09:48.045699+0000","last_fullsized":"2024-04-26T20:09:48.045699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:00.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432128+0000","last_change":"2024-04-26T20:07:39.313079+0000","last_active":"2024-04-26T20:09:56.432128+0000","last_peered":"2024-04-26T20:09:56.432128+0000","last_clean":"2024-04-26T20:09:56.432128+0000","last_became_active":"2024-04-26T20:07:39.312603+0000","last_became_peered":"2024-04-26T20:07:39.312603+0000","last_unstale":"2024-04-26T20:09:56.432128+0000","last_undegraded":"2024-04-26T20:09:56.432128+0000","last_fullsized":"2024-04-26T20:09:56.432128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:45.717375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.184665+0000","last_change":"2024-04-26T20:07:31.715373+0000","last_active":"2024-04-26T20:09:43.184665+0000","last_peered":"2024-04-26T20:09:43.184665+0000","last_clean":"2024-04-26T20:09:43.184665+0000","last_became_active":"2024-04-26T20:07:31.714973+0000","last_became_peered":"2024-04-26T20:07:31.714973+0000","last_unstale":"2024-04-26T20:09:43.184665+0000","last_undegraded":"2024-04-26T20:09:43.184665+0000","last_fullsized":"2024-04-26T20:09:43.184665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.772671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:45.454845+0000","last_change":"2024-04-26T20:07:34.753619+0000","last_active":"2024-04-26T20:09:45.454845+0000","last_peered":"2024-04-26T20:09:45.454845+0000","last_clean":"2024-04-26T20:09:45.454845+0000","last_became_active":"2024-04-26T20:07:34.753263+0000","last_became_peered":"2024-04-26T20:07:34.753263+0000","last_unstale":"2024-04-26T20:09:45.454845+0000","last_undegraded":"2024-04-26T20:09:45.454845+0000","last_fullsized":"2024-04-26T20:09:45.454845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.358153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.715184+0000","last_change":"2024-04-26T20:08:15.840243+0000","last_active":"2024-04-26T20:09:55.715184+0000","last_peered":"2024-04-26T20:09:55.715184+0000","last_clean":"2024-04-26T20:09:55.715184+0000","last_became_active":"2024-04-26T20:08:15.838891+0000","last_became_peered":"2024-04-26T20:08:15.838891+0000","last_unstale":"2024-04-26T20:09:55.715184+0000","last_undegraded":"2024-04-26T20:09:55.715184+0000","last_fullsized":"2024-04-26T20:09:55.715184+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:15.018199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577186+0000","last_change":"2024-04-26T20:07:31.722516+0000","last_active":"2024-04-26T20:09:44.577186+0000","last_peered":"2024-04-26T20:09:44.577186+0000","last_clean":"2024-04-26T20:09:44.577186+0000","last_became_active":"2024-04-26T20:07:31.722096+0000","last_became_peered":"2024-04-26T20:07:31.722096+0000","last_unstale":"2024-04-26T20:09:44.577186+0000","last_undegraded":"2024-04-26T20:09:44.577186+0000","last_fullsized":"2024-04-26T20:09:44.577186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:57.693947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827697+0000","last_change":"2024-04-26T20:07:34.768754+0000","last_active":"2024-04-26T20:09:44.827697+0000","last_peered":"2024-04-26T20:09:44.827697+0000","last_clean":"2024-04-26T20:09:44.827697+0000","last_became_active":"2024-04-26T20:07:34.768405+0000","last_became_peered":"2024-04-26T20:07:34.768405+0000","last_unstale":"2024-04-26T20:09:44.827697+0000","last_undegraded":"2024-04-26T20:09:44.827697+0000","last_fullsized":"2024-04-26T20:09:44.827697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:08.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432095+0000","last_change":"2024-04-26T20:07:38.828804+0000","last_active":"2024-04-26T20:09:56.432095+0000","last_peered":"2024-04-26T20:09:56.432095+0000","last_clean":"2024-04-26T20:09:56.432095+0000","last_became_active":"2024-04-26T20:07:38.828452+0000","last_became_peered":"2024-04-26T20:07:38.828452+0000","last_unstale":"2024-04-26T20:09:56.432095+0000","last_undegraded":"2024-04-26T20:09:56.432095+0000","last_fullsized":"2024-04-26T20:09:56.432095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:29.905102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:11:52.622917+0000","last_change":"2024-04-26T20:07:31.710696+0000","last_active":"2024-04-26T20:11:52.622917+0000","last_peered":"2024-04-26T20:11:52.622917+0000","last_clean":"2024-04-26T20:11:52.622917+0000","last_became_active":"2024-04-26T20:07:31.710039+0000","last_became_peered":"2024-04-26T20:07:31.710039+0000","last_unstale":"2024-04-26T20:11:52.622917+0000","last_undegraded":"2024-04-26T20:11:52.622917+0000","last_fullsized":"2024-04-26T20:11:52.622917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:50.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.802095+0000","last_change":"2024-04-26T20:09:15.798031+0000","last_active":"2024-04-26T20:09:15.802095+0000","last_peered":"2024-04-26T20:09:15.802095+0000","last_clean":"2024-04-26T20:09:15.802095+0000","last_became_active":"2024-04-26T20:09:15.797535+0000","last_became_peered":"2024-04-26T20:09:15.797535+0000","last_unstale":"2024-04-26T20:09:15.802095+0000","last_undegraded":"2024-04-26T20:09:15.802095+0000","last_fullsized":"2024-04-26T20:09:15.802095+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:03.059618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432117+0000","last_change":"2024-04-26T20:07:39.312800+0000","last_active":"2024-04-26T20:09:56.432117+0000","last_peered":"2024-04-26T20:09:56.432117+0000","last_clean":"2024-04-26T20:09:56.432117+0000","last_became_active":"2024-04-26T20:07:39.312309+0000","last_became_peered":"2024-04-26T20:07:39.312309+0000","last_unstale":"2024-04-26T20:09:56.432117+0000","last_undegraded":"2024-04-26T20:09:56.432117+0000","last_fullsized":"2024-04-26T20:09:56.432117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:02.013468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:50.455381+0000","last_change":"2024-04-26T20:07:31.715509+0000","last_active":"2024-04-26T20:09:50.455381+0000","last_peered":"2024-04-26T20:09:50.455381+0000","last_clean":"2024-04-26T20:09:50.455381+0000","last_became_active":"2024-04-26T20:07:31.714606+0000","last_became_peered":"2024-04-26T20:07:31.714606+0000","last_unstale":"2024-04-26T20:09:50.455381+0000","last_undegraded":"2024-04-26T20:09:50.455381+0000","last_fullsized":"2024-04-26T20:09:50.455381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:03.979681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.267172+0000","last_change":"2024-04-26T20:07:34.763771+0000","last_active":"2024-04-26T20:09:48.267172+0000","last_peered":"2024-04-26T20:09:48.267172+0000","last_clean":"2024-04-26T20:09:48.267172+0000","last_became_active":"2024-04-26T20:07:34.763422+0000","last_became_peered":"2024-04-26T20:07:34.763422+0000","last_unstale":"2024-04-26T20:09:48.267172+0000","last_undegraded":"2024-04-26T20:09:48.267172+0000","last_fullsized":"2024-04-26T20:09:48.267172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:27.224024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.185094+0000","last_change":"2024-04-26T20:07:38.830156+0000","last_active":"2024-04-26T20:09:48.185094+0000","last_peered":"2024-04-26T20:09:48.185094+0000","last_clean":"2024-04-26T20:09:48.185094+0000","last_became_active":"2024-04-26T20:07:38.829530+0000","last_became_peered":"2024-04-26T20:07:38.829530+0000","last_unstale":"2024-04-26T20:09:48.185094+0000","last_undegraded":"2024-04-26T20:09:48.185094+0000","last_fullsized":"2024-04-26T20:09:48.185094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:21.720974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609555+0000","last_change":"2024-04-26T20:07:31.915638+0000","last_active":"2024-04-26T20:09:47.609555+0000","last_peered":"2024-04-26T20:09:47.609555+0000","last_clean":"2024-04-26T20:09:47.609555+0000","last_became_active":"2024-04-26T20:07:31.915266+0000","last_became_peered":"2024-04-26T20:07:31.915266+0000","last_unstale":"2024-04-26T20:09:47.609555+0000","last_undegraded":"2024-04-26T20:09:47.609555+0000","last_fullsized":"2024-04-26T20:09:47.609555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:51.452828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267679+0000","last_change":"2024-04-26T20:07:34.754476+0000","last_active":"2024-04-26T20:09:53.267679+0000","last_peered":"2024-04-26T20:09:53.267679+0000","last_clean":"2024-04-26T20:09:53.267679+0000","last_became_active":"2024-04-26T20:07:34.754054+0000","last_became_peered":"2024-04-26T20:07:34.754054+0000","last_unstale":"2024-04-26T20:09:53.267679+0000","last_undegraded":"2024-04-26T20:09:53.267679+0000","last_fullsized":"2024-04-26T20:09:53.267679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.497083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:51.675511+0000","last_change":"2024-04-26T20:07:38.835265+0000","last_active":"2024-04-26T20:09:51.675511+0000","last_peered":"2024-04-26T20:09:51.675511+0000","last_clean":"2024-04-26T20:09:51.675511+0000","last_became_active":"2024-04-26T20:07:38.833873+0000","last_became_peered":"2024-04-26T20:07:38.833873+0000","last_unstale":"2024-04-26T20:09:51.675511+0000","last_undegraded":"2024-04-26T20:09:51.675511+0000","last_fullsized":"2024-04-26T20:09:51.675511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:17.850236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.643274+0000","last_change":"2024-04-26T20:07:31.725576+0000","last_active":"2024-04-26T20:09:47.643274+0000","last_peered":"2024-04-26T20:09:47.643274+0000","last_clean":"2024-04-26T20:09:47.643274+0000","last_became_active":"2024-04-26T20:07:31.725166+0000","last_became_peered":"2024-04-26T20:07:31.725166+0000","last_unstale":"2024-04-26T20:09:47.643274+0000","last_undegraded":"2024-04-26T20:09:47.643274+0000","last_fullsized":"2024-04-26T20:09:47.643274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:04.539215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267647+0000","last_change":"2024-04-26T20:07:34.748263+0000","last_active":"2024-04-26T20:09:53.267647+0000","last_peered":"2024-04-26T20:09:53.267647+0000","last_clean":"2024-04-26T20:09:53.267647+0000","last_became_active":"2024-04-26T20:07:34.747889+0000","last_became_peered":"2024-04-26T20:07:34.747889+0000","last_unstale":"2024-04-26T20:09:53.267647+0000","last_undegraded":"2024-04-26T20:09:53.267647+0000","last_fullsized":"2024-04-26T20:09:53.267647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.257405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643935+0000","last_change":"2024-04-26T20:07:38.803999+0000","last_active":"2024-04-26T20:09:52.643935+0000","last_peered":"2024-04-26T20:09:52.643935+0000","last_clean":"2024-04-26T20:09:52.643935+0000","last_became_active":"2024-04-26T20:07:38.803585+0000","last_became_peered":"2024-04-26T20:07:38.803585+0000","last_unstale":"2024-04-26T20:09:52.643935+0000","last_undegraded":"2024-04-26T20:09:52.643935+0000","last_fullsized":"2024-04-26T20:09:52.643935+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:58.863511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644544+0000","last_change":"2024-04-26T20:07:31.730445+0000","last_active":"2024-04-26T20:09:47.644544+0000","last_peered":"2024-04-26T20:09:47.644544+0000","last_clean":"2024-04-26T20:09:47.644544+0000","last_became_active":"2024-04-26T20:07:31.729944+0000","last_became_peered":"2024-04-26T20:07:31.729944+0000","last_unstale":"2024-04-26T20:09:47.644544+0000","last_undegraded":"2024-04-26T20:09:47.644544+0000","last_fullsized":"2024-04-26T20:09:47.644544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:06.274097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267728+0000","last_change":"2024-04-26T20:07:34.755449+0000","last_active":"2024-04-26T20:09:53.267728+0000","last_peered":"2024-04-26T20:09:53.267728+0000","last_clean":"2024-04-26T20:09:53.267728+0000","last_became_active":"2024-04-26T20:07:34.755100+0000","last_became_peered":"2024-04-26T20:07:34.755100+0000","last_unstale":"2024-04-26T20:09:53.267728+0000","last_undegraded":"2024-04-26T20:09:53.267728+0000","last_fullsized":"2024-04-26T20:09:53.267728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.774751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.827010+0000","last_change":"2024-04-26T20:09:16.177597+0000","last_active":"2024-04-26T20:09:39.827010+0000","last_peered":"2024-04-26T20:09:39.827010+0000","last_clean":"2024-04-26T20:09:39.827010+0000","last_became_active":"2024-04-26T20:09:16.177150+0000","last_became_peered":"2024-04-26T20:09:16.177150+0000","last_unstale":"2024-04-26T20:09:39.827010+0000","last_undegraded":"2024-04-26T20:09:39.827010+0000","last_fullsized":"2024-04-26T20:09:39.827010+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:50.910457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,5],"acting":[11,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.608818+0000","last_change":"2024-04-26T20:07:34.747723+0000","last_active":"2024-04-26T20:09:42.608818+0000","last_peered":"2024-04-26T20:09:42.608818+0000","last_clean":"2024-04-26T20:09:42.608818+0000","last_became_active":"2024-04-26T20:07:34.747411+0000","last_became_peered":"2024-04-26T20:07:34.747411+0000","last_unstale":"2024-04-26T20:09:42.608818+0000","last_undegraded":"2024-04-26T20:09:42.608818+0000","last_fullsized":"2024-04-26T20:09:42.608818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.608845+0000","last_change":"2024-04-26T20:07:34.773012+0000","last_active":"2024-04-26T20:09:42.608845+0000","last_peered":"2024-04-26T20:09:42.608845+0000","last_clean":"2024-04-26T20:09:42.608845+0000","last_became_active":"2024-04-26T20:07:34.772735+0000","last_became_peered":"2024-04-26T20:07:34.772735+0000","last_unstale":"2024-04-26T20:09:42.608845+0000","last_undegraded":"2024-04-26T20:09:42.608845+0000","last_fullsized":"2024-04-26T20:09:42.608845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:08.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044906+0000","last_change":"2024-04-26T20:07:34.762977+0000","last_active":"2024-04-26T20:09:43.044906+0000","last_peered":"2024-04-26T20:09:43.044906+0000","last_clean":"2024-04-26T20:09:43.044906+0000","last_became_active":"2024-04-26T20:07:34.762530+0000","last_became_peered":"2024-04-26T20:07:34.762530+0000","last_unstale":"2024-04-26T20:09:43.044906+0000","last_undegraded":"2024-04-26T20:09:43.044906+0000","last_fullsized":"2024-04-26T20:09:43.044906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:24.811825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644495+0000","last_change":"2024-04-26T20:07:34.769986+0000","last_active":"2024-04-26T20:09:47.644495+0000","last_peered":"2024-04-26T20:09:47.644495+0000","last_clean":"2024-04-26T20:09:47.644495+0000","last_became_active":"2024-04-26T20:07:34.769591+0000","last_became_peered":"2024-04-26T20:07:34.769591+0000","last_unstale":"2024-04-26T20:09:47.644495+0000","last_undegraded":"2024-04-26T20:09:47.644495+0000","last_fullsized":"2024-04-26T20:09:47.644495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:27.466675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827735+0000","last_change":"2024-04-26T20:07:34.754733+0000","last_active":"2024-04-26T20:09:44.827735+0000","last_peered":"2024-04-26T20:09:44.827735+0000","last_clean":"2024-04-26T20:09:44.827735+0000","last_became_active":"2024-04-26T20:07:34.754374+0000","last_became_peered":"2024-04-26T20:07:34.754374+0000","last_unstale":"2024-04-26T20:09:44.827735+0000","last_undegraded":"2024-04-26T20:09:44.827735+0000","last_fullsized":"2024-04-26T20:09:44.827735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:02.732025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609576+0000","last_change":"2024-04-26T20:07:34.750329+0000","last_active":"2024-04-26T20:09:47.609576+0000","last_peered":"2024-04-26T20:09:47.609576+0000","last_clean":"2024-04-26T20:09:47.609576+0000","last_became_active":"2024-04-26T20:07:34.749970+0000","last_became_peered":"2024-04-26T20:07:34.749970+0000","last_unstale":"2024-04-26T20:09:47.609576+0000","last_undegraded":"2024-04-26T20:09:47.609576+0000","last_fullsized":"2024-04-26T20:09:47.609576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:50.375128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577278+0000","last_change":"2024-04-26T20:07:34.750623+0000","last_active":"2024-04-26T20:09:44.577278+0000","last_peered":"2024-04-26T20:09:44.577278+0000","last_clean":"2024-04-26T20:09:44.577278+0000","last_became_active":"2024-04-26T20:07:34.750314+0000","last_became_peered":"2024-04-26T20:07:34.750314+0000","last_unstale":"2024-04-26T20:09:44.577278+0000","last_undegraded":"2024-04-26T20:09:44.577278+0000","last_fullsized":"2024-04-26T20:09:44.577278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:16.773986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.643225+0000","last_change":"2024-04-26T20:07:34.755474+0000","last_active":"2024-04-26T20:09:47.643225+0000","last_peered":"2024-04-26T20:09:47.643225+0000","last_clean":"2024-04-26T20:09:47.643225+0000","last_became_active":"2024-04-26T20:07:34.755133+0000","last_became_peered":"2024-04-26T20:07:34.755133+0000","last_unstale":"2024-04-26T20:09:47.643225+0000","last_undegraded":"2024-04-26T20:09:47.643225+0000","last_fullsized":"2024-04-26T20:09:47.643225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:02.861643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.578236+0000","last_change":"2024-04-26T20:07:34.752727+0000","last_active":"2024-04-26T20:09:49.578236+0000","last_peered":"2024-04-26T20:09:49.578236+0000","last_clean":"2024-04-26T20:09:49.578236+0000","last_became_active":"2024-04-26T20:07:34.752420+0000","last_became_peered":"2024-04-26T20:07:34.752420+0000","last_unstale":"2024-04-26T20:09:49.578236+0000","last_undegraded":"2024-04-26T20:09:49.578236+0000","last_fullsized":"2024-04-26T20:09:49.578236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:45.076837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.185630+0000","last_change":"2024-04-26T20:07:34.775116+0000","last_active":"2024-04-26T20:09:53.185630+0000","last_peered":"2024-04-26T20:09:53.185630+0000","last_clean":"2024-04-26T20:09:53.185630+0000","last_became_active":"2024-04-26T20:07:34.774710+0000","last_became_peered":"2024-04-26T20:07:34.774710+0000","last_unstale":"2024-04-26T20:09:53.185630+0000","last_undegraded":"2024-04-26T20:09:53.185630+0000","last_fullsized":"2024-04-26T20:09:53.185630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:53.203918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.645416+0000","last_change":"2024-04-26T20:07:34.773245+0000","last_active":"2024-04-26T20:09:52.645416+0000","last_peered":"2024-04-26T20:09:52.645416+0000","last_clean":"2024-04-26T20:09:52.645416+0000","last_became_active":"2024-04-26T20:07:34.772879+0000","last_became_peered":"2024-04-26T20:07:34.772879+0000","last_unstale":"2024-04-26T20:09:52.645416+0000","last_undegraded":"2024-04-26T20:09:52.645416+0000","last_fullsized":"2024-04-26T20:09:52.645416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:54.450454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.828260+0000","last_change":"2024-04-26T20:07:34.753716+0000","last_active":"2024-04-26T20:09:49.828260+0000","last_peered":"2024-04-26T20:09:49.828260+0000","last_clean":"2024-04-26T20:09:49.828260+0000","last_became_active":"2024-04-26T20:07:34.753378+0000","last_became_peered":"2024-04-26T20:07:34.753378+0000","last_unstale":"2024-04-26T20:09:49.828260+0000","last_undegraded":"2024-04-26T20:09:49.828260+0000","last_fullsized":"2024-04-26T20:09:49.828260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:47.165305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65050,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1559,"num_read_kb":849,"num_write":174,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383764,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":554234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645401,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1802240,"data_stored":1202241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808928,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":558566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070566,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1130496,"data_stored":554390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332203,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1150609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495729,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":549646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":610295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":551034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411465,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40668,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704484,"statfs":{"total":95995035648,"available":95953391616,"internally_reserved":0,"allocated":1200128,"data_stored":612346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":549840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920893,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":554296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757367,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1146197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:13:11.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:13:11.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 -- 172.21.15.131:0/551332654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 msgr2=0x7f5f6810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:11.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/551332654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 0x7f5f6810cfd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009ca0 tx=0x7f5f6002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 -- 172.21.15.131:0/551332654 shutdown_connections 2024-04-26T20:13:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/551332654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 0x7f5f6810cfd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/551332654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f68105d80 0x7f5f68106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/551332654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f68104b90 0x7f5f68104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:11.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 -- 172.21.15.131:0/551332654 >> 172.21.15.131:0/551332654 conn(0x7f5f68100120 msgr2=0x7f5f68102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:11.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 -- 172.21.15.131:0/551332654 shutdown_connections 2024-04-26T20:13:11.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.711+0000 7f5f6fa00640 1 -- 172.21.15.131:0/551332654 wait complete. 2024-04-26T20:13:12.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:11 smithi157 ceph-mon[32703]: pgmap v505: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:12.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:11 smithi157 ceph-mon[32703]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:12.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:11 smithi148 ceph-mon[32281]: pgmap v505: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:11 smithi148 ceph-mon[32281]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:12.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 Processor -- start 2024-04-26T20:13:12.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 -- start start 2024-04-26T20:13:12.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f68104b90 0x7f5f6819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:12.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f68105d80 0x7f5f6819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 0x7f5f681a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:12.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68110710 con 0x7f5f68106740 2024-04-26T20:13:12.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68110590 con 0x7f5f68105d80 2024-04-26T20:13:12.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.712+0000 7f5f6fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68110890 con 0x7f5f68104b90 2024-04-26T20:13:12.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f6da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 0x7f5f681a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:12.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f67e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f68105d80 0x7f5f6819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:12.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f6da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 0x7f5f681a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50170/0 (socket says 172.21.15.131:50170) 2024-04-26T20:13:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f6da00640 1 -- 172.21.15.131:0/1850840731 learned_addr learned my addr 172.21.15.131:0/1850840731 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:12.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f6d000640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f68104b90 0x7f5f6819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:12.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f67e00640 1 -- 172.21.15.131:0/1850840731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f68104b90 msgr2=0x7f5f6819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:13.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f67e00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f68104b90 0x7f5f6819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f67e00640 1 -- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 msgr2=0x7f5f681a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f67e00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 0x7f5f681a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.713+0000 7f5f67e00640 1 -- 172.21.15.131:0/1850840731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681a2960 con 0x7f5f68105d80 2024-04-26T20:13:13.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.714+0000 7f5f67e00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f68105d80 0x7f5f6819b290 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400e9f0 tx=0x7f5f5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:13.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.714+0000 7f5f65600640 1 -- 172.21.15.131:0/1850840731 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400ccc0 con 0x7f5f68105d80 2024-04-26T20:13:13.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.714+0000 7f5f65600640 1 -- 172.21.15.131:0/1850840731 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f540108e0 con 0x7f5f68105d80 2024-04-26T20:13:13.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.714+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f681a2c50 con 0x7f5f68105d80 2024-04-26T20:13:13.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.714+0000 7f5f65600640 1 -- 172.21.15.131:0/1850840731 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54018880 con 0x7f5f68105d80 2024-04-26T20:13:13.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.714+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a31a0 con 0x7f5f68105d80 2024-04-26T20:13:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.715+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6810beb0 con 0x7f5f68105d80 2024-04-26T20:13:13.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.717+0000 7f5f65600640 1 -- 172.21.15.131:0/1850840731 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5f54010430 con 0x7f5f68105d80 2024-04-26T20:13:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.718+0000 7f5f65600640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5f500a3cc0 0x7f5f500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:13.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.719+0000 7f5f65600640 1 -- 172.21.15.131:0/1850840731 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5f540d1150 con 0x7f5f68105d80 2024-04-26T20:13:13.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.719+0000 7f5f6d000640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5f500a3cc0 0x7f5f500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:13.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.719+0000 7f5f6d000640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5f500a3cc0 0x7f5f500a6180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f6819c220 tx=0x7f5f5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:13.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.721+0000 7f5f65600640 1 -- 172.21.15.131:0/1850840731 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5f5409cf30 con 0x7f5f68105d80 2024-04-26T20:13:13.853 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:11 smithi131 ceph-mon[27919]: pgmap v505: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:13.914 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:11 smithi131 ceph-mon[27919]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:13.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.920+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f68062d70 con 0x7f5f500a3cc0 2024-04-26T20:13:14.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.940+0000 7f5f65600640 1 -- 172.21.15.131:0/1850840731 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340369 (secure 0 0 0) 0x7f5f68062d70 con 0x7f5f500a3cc0 2024-04-26T20:13:14.029 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:13:14.111 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-26T20:13:14.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5f500a3cc0 msgr2=0x7f5f500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5f500a3cc0 0x7f5f500a6180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f6819c220 tx=0x7f5f5c005f90 comp rx=0 tx=0).stop 2024-04-26T20:13:14.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f68105d80 msgr2=0x7f5f6819b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f68105d80 0x7f5f6819b290 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400e9f0 tx=0x7f5f5400eeb0 comp rx=0 tx=0).stop 2024-04-26T20:13:14.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 shutdown_connections 2024-04-26T20:13:14.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f68106740 0x7f5f681a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:14.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f68105d80 0x7f5f6819b290 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5f500a3cc0 0x7f5f500a6180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:14.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 --2- 172.21.15.131:0/1850840731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f68104b90 0x7f5f6819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 >> 172.21.15.131:0/1850840731 conn(0x7f5f68100120 msgr2=0x7f5f68109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:14.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 shutdown_connections 2024-04-26T20:13:14.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:11.948+0000 7f5f6fa00640 1 -- 172.21.15.131:0/1850840731 wait complete. 2024-04-26T20:13:14.819 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:13 smithi157 ceph-mon[32703]: pgmap v506: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:14.893 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:13 smithi157 ceph-mon[32703]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:14.901 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:13 smithi148 ceph-mon[32281]: pgmap v506: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:14.945 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:13 smithi148 ceph-mon[32281]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:15.005 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-04-26T20:13:11.748484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655652,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1685,"num_read_kb":958,"num_write":357,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368480,"kb_used_data":15140,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573344,"statfs":{"total":1151940427776,"available":1151563104256,"internally_reserved":0,"allocated":15503360,"data_stored":8593694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006415"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.827058+0000","last_change":"2024-04-26T20:07:34.771977+0000","last_active":"2024-04-26T20:09:39.827058+0000","last_peered":"2024-04-26T20:09:39.827058+0000","last_clean":"2024-04-26T20:09:39.827058+0000","last_became_active":"2024-04-26T20:07:34.771658+0000","last_became_peered":"2024-04-26T20:07:34.771658+0000","last_unstale":"2024-04-26T20:09:39.827058+0000","last_undegraded":"2024-04-26T20:09:39.827058+0000","last_fullsized":"2024-04-26T20:09:39.827058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:38.419057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044979+0000","last_change":"2024-04-26T20:07:34.768774+0000","last_active":"2024-04-26T20:09:43.044979+0000","last_peered":"2024-04-26T20:09:43.044979+0000","last_clean":"2024-04-26T20:09:43.044979+0000","last_became_active":"2024-04-26T20:07:34.768383+0000","last_became_peered":"2024-04-26T20:07:34.768383+0000","last_unstale":"2024-04-26T20:09:43.044979+0000","last_undegraded":"2024-04-26T20:09:43.044979+0000","last_fullsized":"2024-04-26T20:09:43.044979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:43.813261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044955+0000","last_change":"2024-04-26T20:07:34.761796+0000","last_active":"2024-04-26T20:09:43.044955+0000","last_peered":"2024-04-26T20:09:43.044955+0000","last_clean":"2024-04-26T20:09:43.044955+0000","last_became_active":"2024-04-26T20:07:34.761403+0000","last_became_peered":"2024-04-26T20:07:34.761403+0000","last_unstale":"2024-04-26T20:09:43.044955+0000","last_undegraded":"2024-04-26T20:09:43.044955+0000","last_fullsized":"2024-04-26T20:09:43.044955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:06.832304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.012166+0000","last_change":"2024-04-26T20:07:34.756728+0000","last_active":"2024-04-26T20:09:37.012166+0000","last_peered":"2024-04-26T20:09:37.012166+0000","last_clean":"2024-04-26T20:09:37.012166+0000","last_became_active":"2024-04-26T20:07:34.756406+0000","last_became_peered":"2024-04-26T20:07:34.756406+0000","last_unstale":"2024-04-26T20:09:37.012166+0000","last_undegraded":"2024-04-26T20:09:37.012166+0000","last_fullsized":"2024-04-26T20:09:37.012166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:27.753674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457110+0000","last_change":"2024-04-26T20:07:34.758243+0000","last_active":"2024-04-26T20:10:05.457110+0000","last_peered":"2024-04-26T20:10:05.457110+0000","last_clean":"2024-04-26T20:10:05.457110+0000","last_became_active":"2024-04-26T20:07:34.757868+0000","last_became_peered":"2024-04-26T20:07:34.757868+0000","last_unstale":"2024-04-26T20:10:05.457110+0000","last_undegraded":"2024-04-26T20:10:05.457110+0000","last_fullsized":"2024-04-26T20:10:05.457110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:53.361835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.971598+0000","last_change":"2024-04-26T20:08:15.585311+0000","last_active":"2024-04-26T20:10:07.971598+0000","last_peered":"2024-04-26T20:10:07.971598+0000","last_clean":"2024-04-26T20:10:07.971598+0000","last_became_active":"2024-04-26T20:08:15.584982+0000","last_became_peered":"2024-04-26T20:08:15.584982+0000","last_unstale":"2024-04-26T20:10:07.971598+0000","last_undegraded":"2024-04-26T20:10:07.971598+0000","last_fullsized":"2024-04-26T20:10:07.971598+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:56.828218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:08.269506+0000","last_change":"2024-04-26T20:07:34.757529+0000","last_active":"2024-04-26T20:10:08.269506+0000","last_peered":"2024-04-26T20:10:08.269506+0000","last_clean":"2024-04-26T20:10:08.269506+0000","last_became_active":"2024-04-26T20:07:34.757166+0000","last_became_peered":"2024-04-26T20:07:34.757166+0000","last_unstale":"2024-04-26T20:10:08.269506+0000","last_undegraded":"2024-04-26T20:10:08.269506+0000","last_fullsized":"2024-04-26T20:10:08.269506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:01.309698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:04.581136+0000","last_change":"2024-04-26T20:07:34.765352+0000","last_active":"2024-04-26T20:10:04.581136+0000","last_peered":"2024-04-26T20:10:04.581136+0000","last_clean":"2024-04-26T20:10:04.581136+0000","last_became_active":"2024-04-26T20:07:34.758400+0000","last_became_peered":"2024-04-26T20:07:34.758400+0000","last_unstale":"2024-04-26T20:10:04.581136+0000","last_undegraded":"2024-04-26T20:10:04.581136+0000","last_fullsized":"2024-04-26T20:10:04.581136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:43.940746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.186777+0000","last_change":"2024-04-26T20:07:34.754413+0000","last_active":"2024-04-26T20:10:03.186777+0000","last_peered":"2024-04-26T20:10:03.186777+0000","last_clean":"2024-04-26T20:10:03.186777+0000","last_became_active":"2024-04-26T20:07:34.754073+0000","last_became_peered":"2024-04-26T20:07:34.754073+0000","last_unstale":"2024-04-26T20:10:03.186777+0000","last_undegraded":"2024-04-26T20:10:03.186777+0000","last_fullsized":"2024-04-26T20:10:03.186777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:25.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.971090+0000","last_change":"2024-04-26T20:07:34.748739+0000","last_active":"2024-04-26T20:10:02.971090+0000","last_peered":"2024-04-26T20:10:02.971090+0000","last_clean":"2024-04-26T20:10:02.971090+0000","last_became_active":"2024-04-26T20:07:34.748378+0000","last_became_peered":"2024-04-26T20:07:34.748378+0000","last_unstale":"2024-04-26T20:10:02.971090+0000","last_undegraded":"2024-04-26T20:10:02.971090+0000","last_fullsized":"2024-04-26T20:10:02.971090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:13.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.268839+0000","last_change":"2024-04-26T20:07:34.747402+0000","last_active":"2024-04-26T20:10:03.268839+0000","last_peered":"2024-04-26T20:10:03.268839+0000","last_clean":"2024-04-26T20:10:03.268839+0000","last_became_active":"2024-04-26T20:07:34.746965+0000","last_became_peered":"2024-04-26T20:07:34.746965+0000","last_unstale":"2024-04-26T20:10:03.268839+0000","last_undegraded":"2024-04-26T20:10:03.268839+0000","last_fullsized":"2024-04-26T20:10:03.268839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:32.066286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.367770+0000","last_change":"2024-04-26T20:09:16.366157+0000","last_active":"2024-04-26T20:09:16.367770+0000","last_peered":"2024-04-26T20:09:16.367770+0000","last_clean":"2024-04-26T20:09:16.367770+0000","last_became_active":"2024-04-26T20:09:16.365132+0000","last_became_peered":"2024-04-26T20:09:16.365132+0000","last_unstale":"2024-04-26T20:09:16.367770+0000","last_undegraded":"2024-04-26T20:09:16.367770+0000","last_fullsized":"2024-04-26T20:09:16.367770+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.559981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.646861+0000","last_change":"2024-04-26T20:07:34.766542+0000","last_active":"2024-04-26T20:10:02.646861+0000","last_peered":"2024-04-26T20:10:02.646861+0000","last_clean":"2024-04-26T20:10:02.646861+0000","last_became_active":"2024-04-26T20:07:34.766185+0000","last_became_peered":"2024-04-26T20:07:34.766185+0000","last_unstale":"2024-04-26T20:10:02.646861+0000","last_undegraded":"2024-04-26T20:10:02.646861+0000","last_fullsized":"2024-04-26T20:10:02.646861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:34.499145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.047069+0000","last_change":"2024-04-26T20:07:34.762610+0000","last_active":"2024-04-26T20:09:58.047069+0000","last_peered":"2024-04-26T20:09:58.047069+0000","last_clean":"2024-04-26T20:09:58.047069+0000","last_became_active":"2024-04-26T20:07:34.762100+0000","last_became_peered":"2024-04-26T20:07:34.762100+0000","last_unstale":"2024-04-26T20:09:58.047069+0000","last_undegraded":"2024-04-26T20:09:58.047069+0000","last_fullsized":"2024-04-26T20:09:58.047069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:18.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.047033+0000","last_change":"2024-04-26T20:07:34.766754+0000","last_active":"2024-04-26T20:09:58.047033+0000","last_peered":"2024-04-26T20:09:58.047033+0000","last_clean":"2024-04-26T20:09:58.047033+0000","last_became_active":"2024-04-26T20:07:34.762204+0000","last_became_peered":"2024-04-26T20:07:34.762204+0000","last_unstale":"2024-04-26T20:09:58.047033+0000","last_undegraded":"2024-04-26T20:09:58.047033+0000","last_fullsized":"2024-04-26T20:09:58.047033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:25.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.186093+0000","last_change":"2024-04-26T20:07:34.769739+0000","last_active":"2024-04-26T20:09:58.186093+0000","last_peered":"2024-04-26T20:09:58.186093+0000","last_clean":"2024-04-26T20:09:58.186093+0000","last_became_active":"2024-04-26T20:07:34.769383+0000","last_became_peered":"2024-04-26T20:07:34.769383+0000","last_unstale":"2024-04-26T20:09:58.186093+0000","last_undegraded":"2024-04-26T20:09:58.186093+0000","last_fullsized":"2024-04-26T20:09:58.186093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:08.771590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:54.828723+0000","last_change":"2024-04-26T20:07:34.772010+0000","last_active":"2024-04-26T20:09:54.828723+0000","last_peered":"2024-04-26T20:09:54.828723+0000","last_clean":"2024-04-26T20:09:54.828723+0000","last_became_active":"2024-04-26T20:07:34.771682+0000","last_became_peered":"2024-04-26T20:07:34.771682+0000","last_unstale":"2024-04-26T20:09:54.828723+0000","last_undegraded":"2024-04-26T20:09:54.828723+0000","last_fullsized":"2024-04-26T20:09:54.828723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:57.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.211446+0000","last_change":"2024-04-26T20:07:34.757547+0000","last_active":"2024-04-26T20:09:55.211446+0000","last_peered":"2024-04-26T20:09:55.211446+0000","last_clean":"2024-04-26T20:09:55.211446+0000","last_became_active":"2024-04-26T20:07:34.757180+0000","last_became_peered":"2024-04-26T20:07:34.757180+0000","last_unstale":"2024-04-26T20:09:55.211446+0000","last_undegraded":"2024-04-26T20:09:55.211446+0000","last_fullsized":"2024-04-26T20:09:55.211446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:32.365944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.610166+0000","last_change":"2024-04-26T20:07:34.767363+0000","last_active":"2024-04-26T20:09:52.610166+0000","last_peered":"2024-04-26T20:09:52.610166+0000","last_clean":"2024-04-26T20:09:52.610166+0000","last_became_active":"2024-04-26T20:07:34.766909+0000","last_became_peered":"2024-04-26T20:07:34.766909+0000","last_unstale":"2024-04-26T20:09:52.610166+0000","last_undegraded":"2024-04-26T20:09:52.610166+0000","last_fullsized":"2024-04-26T20:09:52.610166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:32:31.500141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.612056+0000","last_change":"2024-04-26T20:07:38.823959+0000","last_active":"2024-04-26T20:10:07.612056+0000","last_peered":"2024-04-26T20:10:07.612056+0000","last_clean":"2024-04-26T20:10:07.612056+0000","last_became_active":"2024-04-26T20:07:38.823524+0000","last_became_peered":"2024-04-26T20:07:38.823524+0000","last_unstale":"2024-04-26T20:10:07.612056+0000","last_undegraded":"2024-04-26T20:10:07.612056+0000","last_fullsized":"2024-04-26T20:10:07.612056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:23.348376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'9","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:35.712160+0000","last_change":"2024-04-26T20:07:31.721324+0000","last_active":"2024-04-26T20:09:35.712160+0000","last_peered":"2024-04-26T20:09:35.712160+0000","last_clean":"2024-04-26T20:09:35.712160+0000","last_became_active":"2024-04-26T20:07:31.720140+0000","last_became_peered":"2024-04-26T20:07:31.720140+0000","last_unstale":"2024-04-26T20:09:35.712160+0000","last_undegraded":"2024-04-26T20:09:35.712160+0000","last_fullsized":"2024-04-26T20:09:35.712160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:27.636660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:34.575396+0000","last_change":"2024-04-26T20:07:34.760387+0000","last_active":"2024-04-26T20:09:34.575396+0000","last_peered":"2024-04-26T20:09:34.575396+0000","last_clean":"2024-04-26T20:09:34.575396+0000","last_became_active":"2024-04-26T20:07:34.760042+0000","last_became_peered":"2024-04-26T20:07:34.760042+0000","last_unstale":"2024-04-26T20:09:34.575396+0000","last_undegraded":"2024-04-26T20:09:34.575396+0000","last_fullsized":"2024-04-26T20:09:34.575396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:39.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.015378+0000","last_change":"2024-04-26T20:08:15.587181+0000","last_active":"2024-04-26T20:10:07.015378+0000","last_peered":"2024-04-26T20:10:07.015378+0000","last_clean":"2024-04-26T20:10:07.015378+0000","last_became_active":"2024-04-26T20:08:15.586847+0000","last_became_peered":"2024-04-26T20:08:15.586847+0000","last_unstale":"2024-04-26T20:10:07.015378+0000","last_undegraded":"2024-04-26T20:10:07.015378+0000","last_fullsized":"2024-04-26T20:10:07.015378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:12.495523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:34.575467+0000","last_change":"2024-04-26T20:07:31.711623+0000","last_active":"2024-04-26T20:09:34.575467+0000","last_peered":"2024-04-26T20:09:34.575467+0000","last_clean":"2024-04-26T20:09:34.575467+0000","last_became_active":"2024-04-26T20:07:31.710488+0000","last_became_peered":"2024-04-26T20:07:31.710488+0000","last_unstale":"2024-04-26T20:09:34.575467+0000","last_undegraded":"2024-04-26T20:09:34.575467+0000","last_fullsized":"2024-04-26T20:09:34.575467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:04.747170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.045052+0000","last_change":"2024-04-26T20:09:15.801039+0000","last_active":"2024-04-26T20:09:43.045052+0000","last_peered":"2024-04-26T20:09:43.045052+0000","last_clean":"2024-04-26T20:09:43.045052+0000","last_became_active":"2024-04-26T20:09:15.800049+0000","last_became_peered":"2024-04-26T20:09:15.800049+0000","last_unstale":"2024-04-26T20:09:43.045052+0000","last_undegraded":"2024-04-26T20:09:43.045052+0000","last_fullsized":"2024-04-26T20:09:43.045052+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:49.168625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,9],"acting":[0,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.016057+0000","last_change":"2024-04-26T20:07:38.822443+0000","last_active":"2024-04-26T20:10:12.016057+0000","last_peered":"2024-04-26T20:10:12.016057+0000","last_clean":"2024-04-26T20:10:12.016057+0000","last_became_active":"2024-04-26T20:07:38.822046+0000","last_became_peered":"2024-04-26T20:07:38.822046+0000","last_unstale":"2024-04-26T20:10:12.016057+0000","last_undegraded":"2024-04-26T20:10:12.016057+0000","last_fullsized":"2024-04-26T20:10:12.016057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:26.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457125+0000","last_change":"2024-04-26T20:07:31.715319+0000","last_active":"2024-04-26T20:10:05.457125+0000","last_peered":"2024-04-26T20:10:05.457125+0000","last_clean":"2024-04-26T20:10:05.457125+0000","last_became_active":"2024-04-26T20:07:31.714915+0000","last_became_peered":"2024-04-26T20:07:31.714915+0000","last_unstale":"2024-04-26T20:10:05.457125+0000","last_undegraded":"2024-04-26T20:10:05.457125+0000","last_fullsized":"2024-04-26T20:10:05.457125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:27.898652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:08.187267+0000","last_change":"2024-04-26T20:07:34.753422+0000","last_active":"2024-04-26T20:10:08.187267+0000","last_peered":"2024-04-26T20:10:08.187267+0000","last_clean":"2024-04-26T20:10:08.187267+0000","last_became_active":"2024-04-26T20:07:34.753009+0000","last_became_peered":"2024-04-26T20:07:34.753009+0000","last_unstale":"2024-04-26T20:10:08.187267+0000","last_undegraded":"2024-04-26T20:10:08.187267+0000","last_fullsized":"2024-04-26T20:10:08.187267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.369715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.016037+0000","last_change":"2024-04-26T20:07:38.830239+0000","last_active":"2024-04-26T20:10:12.016037+0000","last_peered":"2024-04-26T20:10:12.016037+0000","last_clean":"2024-04-26T20:10:12.016037+0000","last_became_active":"2024-04-26T20:07:38.829509+0000","last_became_peered":"2024-04-26T20:07:38.829509+0000","last_unstale":"2024-04-26T20:10:12.016037+0000","last_undegraded":"2024-04-26T20:10:12.016037+0000","last_fullsized":"2024-04-26T20:10:12.016037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:53.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.788907+0000","last_change":"2024-04-26T20:08:14.576129+0000","last_active":"2024-04-26T20:09:15.788907+0000","last_peered":"2024-04-26T20:09:15.788907+0000","last_clean":"2024-04-26T20:09:15.788907+0000","last_became_active":"2024-04-26T20:08:14.573168+0000","last_became_peered":"2024-04-26T20:08:14.573168+0000","last_unstale":"2024-04-26T20:09:15.788907+0000","last_undegraded":"2024-04-26T20:09:15.788907+0000","last_fullsized":"2024-04-26T20:09:15.788907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:38.234154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645639+0000","last_change":"2024-04-26T20:07:34.749270+0000","last_active":"2024-04-26T20:10:07.645639+0000","last_peered":"2024-04-26T20:10:07.645639+0000","last_clean":"2024-04-26T20:10:07.645639+0000","last_became_active":"2024-04-26T20:07:34.748585+0000","last_became_peered":"2024-04-26T20:07:34.748585+0000","last_unstale":"2024-04-26T20:10:07.645639+0000","last_undegraded":"2024-04-26T20:10:07.645639+0000","last_fullsized":"2024-04-26T20:10:07.645639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:03.091720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.612582+0000","last_change":"2024-04-26T20:07:38.806247+0000","last_active":"2024-04-26T20:10:12.612582+0000","last_peered":"2024-04-26T20:10:12.612582+0000","last_clean":"2024-04-26T20:10:12.612582+0000","last_became_active":"2024-04-26T20:07:38.805748+0000","last_became_peered":"2024-04-26T20:07:38.805748+0000","last_unstale":"2024-04-26T20:10:12.612582+0000","last_undegraded":"2024-04-26T20:10:12.612582+0000","last_fullsized":"2024-04-26T20:10:12.612582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:16.962494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":308,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:06.626389+0000","last_change":"2024-04-26T20:07:31.713618+0000","last_active":"2024-04-26T20:13:06.626389+0000","last_peered":"2024-04-26T20:13:06.626389+0000","last_clean":"2024-04-26T20:13:06.626389+0000","last_became_active":"2024-04-26T20:07:31.713259+0000","last_became_peered":"2024-04-26T20:07:31.713259+0000","last_unstale":"2024-04-26T20:13:06.626389+0000","last_undegraded":"2024-04-26T20:13:06.626389+0000","last_fullsized":"2024-04-26T20:13:06.626389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:25.215087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":259,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457088+0000","last_change":"2024-04-26T20:08:14.585688+0000","last_active":"2024-04-26T20:10:05.457088+0000","last_peered":"2024-04-26T20:10:05.457088+0000","last_clean":"2024-04-26T20:10:05.457088+0000","last_became_active":"2024-04-26T20:08:14.585337+0000","last_became_peered":"2024-04-26T20:08:14.585337+0000","last_unstale":"2024-04-26T20:10:05.457088+0000","last_undegraded":"2024-04-26T20:10:05.457088+0000","last_fullsized":"2024-04-26T20:10:05.457088+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:03.515457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.612549+0000","last_change":"2024-04-26T20:07:38.829772+0000","last_active":"2024-04-26T20:10:12.612549+0000","last_peered":"2024-04-26T20:10:12.612549+0000","last_clean":"2024-04-26T20:10:12.612549+0000","last_became_active":"2024-04-26T20:07:38.829327+0000","last_became_peered":"2024-04-26T20:07:38.829327+0000","last_unstale":"2024-04-26T20:10:12.612549+0000","last_undegraded":"2024-04-26T20:10:12.612549+0000","last_fullsized":"2024-04-26T20:10:12.612549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:37.252863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.971049+0000","last_change":"2024-04-26T20:07:31.728654+0000","last_active":"2024-04-26T20:10:02.971049+0000","last_peered":"2024-04-26T20:10:02.971049+0000","last_clean":"2024-04-26T20:10:02.971049+0000","last_became_active":"2024-04-26T20:07:31.728278+0000","last_became_peered":"2024-04-26T20:07:31.728278+0000","last_unstale":"2024-04-26T20:10:02.971049+0000","last_undegraded":"2024-04-26T20:10:02.971049+0000","last_fullsized":"2024-04-26T20:10:02.971049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645617+0000","last_change":"2024-04-26T20:07:34.739575+0000","last_active":"2024-04-26T20:10:07.645617+0000","last_peered":"2024-04-26T20:10:07.645617+0000","last_clean":"2024-04-26T20:10:07.645617+0000","last_became_active":"2024-04-26T20:07:34.739264+0000","last_became_peered":"2024-04-26T20:07:34.739264+0000","last_unstale":"2024-04-26T20:10:07.645617+0000","last_undegraded":"2024-04-26T20:10:07.645617+0000","last_fullsized":"2024-04-26T20:10:07.645617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:25.130064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827749+0000","last_change":"2024-04-26T20:07:38.822669+0000","last_active":"2024-04-26T20:09:44.827749+0000","last_peered":"2024-04-26T20:09:44.827749+0000","last_clean":"2024-04-26T20:09:44.827749+0000","last_became_active":"2024-04-26T20:07:38.822197+0000","last_became_peered":"2024-04-26T20:07:38.822197+0000","last_unstale":"2024-04-26T20:09:44.827749+0000","last_undegraded":"2024-04-26T20:09:44.827749+0000","last_fullsized":"2024-04-26T20:09:44.827749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:56.782794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"95'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.646035+0000","last_change":"2024-04-26T20:07:31.723527+0000","last_active":"2024-04-26T20:09:57.646035+0000","last_peered":"2024-04-26T20:09:57.646035+0000","last_clean":"2024-04-26T20:09:57.646035+0000","last_became_active":"2024-04-26T20:07:31.722574+0000","last_became_peered":"2024-04-26T20:07:31.722574+0000","last_unstale":"2024-04-26T20:09:57.646035+0000","last_undegraded":"2024-04-26T20:09:57.646035+0000","last_fullsized":"2024-04-26T20:09:57.646035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.521613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.580024+0000","last_change":"2024-04-26T20:07:34.771459+0000","last_active":"2024-04-26T20:09:59.580024+0000","last_peered":"2024-04-26T20:09:59.580024+0000","last_clean":"2024-04-26T20:09:59.580024+0000","last_became_active":"2024-04-26T20:07:34.771124+0000","last_became_peered":"2024-04-26T20:07:34.771124+0000","last_unstale":"2024-04-26T20:09:59.580024+0000","last_undegraded":"2024-04-26T20:09:59.580024+0000","last_fullsized":"2024-04-26T20:09:59.580024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.560440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.043976+0000","last_change":"2024-04-26T20:07:38.838858+0000","last_active":"2024-04-26T20:09:38.043976+0000","last_peered":"2024-04-26T20:09:38.043976+0000","last_clean":"2024-04-26T20:09:38.043976+0000","last_became_active":"2024-04-26T20:07:38.838125+0000","last_became_peered":"2024-04-26T20:07:38.838125+0000","last_unstale":"2024-04-26T20:09:38.043976+0000","last_undegraded":"2024-04-26T20:09:38.043976+0000","last_fullsized":"2024-04-26T20:09:38.043976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:01.856633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.644435+0000","last_change":"2024-04-26T20:07:31.710690+0000","last_active":"2024-04-26T20:09:57.644435+0000","last_peered":"2024-04-26T20:09:57.644435+0000","last_clean":"2024-04-26T20:09:57.644435+0000","last_became_active":"2024-04-26T20:07:31.710353+0000","last_became_peered":"2024-04-26T20:07:31.710353+0000","last_unstale":"2024-04-26T20:09:57.644435+0000","last_undegraded":"2024-04-26T20:09:57.644435+0000","last_fullsized":"2024-04-26T20:09:57.644435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:47.442673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715632+0000","last_change":"2024-04-26T20:07:34.776023+0000","last_active":"2024-04-26T20:10:00.715632+0000","last_peered":"2024-04-26T20:10:00.715632+0000","last_clean":"2024-04-26T20:10:00.715632+0000","last_became_active":"2024-04-26T20:07:34.775571+0000","last_became_peered":"2024-04-26T20:07:34.775571+0000","last_unstale":"2024-04-26T20:10:00.715632+0000","last_undegraded":"2024-04-26T20:10:00.715632+0000","last_fullsized":"2024-04-26T20:10:00.715632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:40.573046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.580074+0000","last_change":"2024-04-26T20:07:38.820636+0000","last_active":"2024-04-26T20:09:59.580074+0000","last_peered":"2024-04-26T20:09:59.580074+0000","last_clean":"2024-04-26T20:09:59.580074+0000","last_became_active":"2024-04-26T20:07:38.819390+0000","last_became_peered":"2024-04-26T20:07:38.819390+0000","last_unstale":"2024-04-26T20:09:59.580074+0000","last_undegraded":"2024-04-26T20:09:59.580074+0000","last_fullsized":"2024-04-26T20:09:59.580074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:58.793479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'4","reported_seq":300,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:06.625753+0000","last_change":"2024-04-26T20:07:31.724192+0000","last_active":"2024-04-26T20:13:06.625753+0000","last_peered":"2024-04-26T20:13:06.625753+0000","last_clean":"2024-04-26T20:13:06.625753+0000","last_became_active":"2024-04-26T20:07:31.723816+0000","last_became_peered":"2024-04-26T20:07:31.723816+0000","last_unstale":"2024-04-26T20:13:06.625753+0000","last_undegraded":"2024-04-26T20:13:06.625753+0000","last_fullsized":"2024-04-26T20:13:06.625753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:31.707282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":255,"num_read_kb":255,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.186757+0000","last_change":"2024-04-26T20:07:34.759027+0000","last_active":"2024-04-26T20:10:03.186757+0000","last_peered":"2024-04-26T20:10:03.186757+0000","last_clean":"2024-04-26T20:10:03.186757+0000","last_became_active":"2024-04-26T20:07:34.758689+0000","last_became_peered":"2024-04-26T20:07:34.758689+0000","last_unstale":"2024-04-26T20:10:03.186757+0000","last_undegraded":"2024-04-26T20:10:03.186757+0000","last_fullsized":"2024-04-26T20:10:03.186757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:29.110222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715598+0000","last_change":"2024-04-26T20:07:38.819000+0000","last_active":"2024-04-26T20:10:00.715598+0000","last_peered":"2024-04-26T20:10:00.715598+0000","last_clean":"2024-04-26T20:10:00.715598+0000","last_became_active":"2024-04-26T20:07:38.818526+0000","last_became_peered":"2024-04-26T20:07:38.818526+0000","last_unstale":"2024-04-26T20:10:00.715598+0000","last_undegraded":"2024-04-26T20:10:00.715598+0000","last_fullsized":"2024-04-26T20:10:00.715598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:28.830096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268278+0000","last_change":"2024-04-26T20:07:31.712952+0000","last_active":"2024-04-26T20:09:58.268278+0000","last_peered":"2024-04-26T20:09:58.268278+0000","last_clean":"2024-04-26T20:09:58.268278+0000","last_became_active":"2024-04-26T20:07:31.712081+0000","last_became_peered":"2024-04-26T20:07:31.712081+0000","last_unstale":"2024-04-26T20:09:58.268278+0000","last_undegraded":"2024-04-26T20:09:58.268278+0000","last_fullsized":"2024-04-26T20:09:58.268278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:46.923536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.047921+0000","last_change":"2024-04-26T20:09:16.367690+0000","last_active":"2024-04-26T20:10:03.047921+0000","last_peered":"2024-04-26T20:10:03.047921+0000","last_clean":"2024-04-26T20:10:03.047921+0000","last_became_active":"2024-04-26T20:09:16.366496+0000","last_became_peered":"2024-04-26T20:09:16.366496+0000","last_unstale":"2024-04-26T20:10:03.047921+0000","last_undegraded":"2024-04-26T20:10:03.047921+0000","last_fullsized":"2024-04-26T20:10:03.047921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:36.154062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.014903+0000","last_change":"2024-04-26T20:07:38.822520+0000","last_active":"2024-04-26T20:10:02.014903+0000","last_peered":"2024-04-26T20:10:02.014903+0000","last_clean":"2024-04-26T20:10:02.014903+0000","last_became_active":"2024-04-26T20:07:38.822056+0000","last_became_peered":"2024-04-26T20:07:38.822056+0000","last_unstale":"2024-04-26T20:10:02.014903+0000","last_undegraded":"2024-04-26T20:10:02.014903+0000","last_fullsized":"2024-04-26T20:10:02.014903+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.324996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"95'4","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.837386+0000","last_change":"2024-04-26T20:07:31.724015+0000","last_active":"2024-04-26T20:09:55.837386+0000","last_peered":"2024-04-26T20:09:55.837386+0000","last_clean":"2024-04-26T20:09:55.837386+0000","last_became_active":"2024-04-26T20:07:31.723686+0000","last_became_peered":"2024-04-26T20:07:31.723686+0000","last_unstale":"2024-04-26T20:09:55.837386+0000","last_undegraded":"2024-04-26T20:09:55.837386+0000","last_fullsized":"2024-04-26T20:09:55.837386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:55.925280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.186136+0000","last_change":"2024-04-26T20:07:34.750276+0000","last_active":"2024-04-26T20:09:58.186136+0000","last_peered":"2024-04-26T20:09:58.186136+0000","last_clean":"2024-04-26T20:09:58.186136+0000","last_became_active":"2024-04-26T20:07:34.749716+0000","last_became_peered":"2024-04-26T20:07:34.749716+0000","last_unstale":"2024-04-26T20:09:58.186136+0000","last_undegraded":"2024-04-26T20:09:58.186136+0000","last_fullsized":"2024-04-26T20:09:58.186136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:22.429780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715645+0000","last_change":"2024-04-26T20:07:38.815636+0000","last_active":"2024-04-26T20:10:00.715645+0000","last_peered":"2024-04-26T20:10:00.715645+0000","last_clean":"2024-04-26T20:10:00.715645+0000","last_became_active":"2024-04-26T20:07:38.815278+0000","last_became_peered":"2024-04-26T20:07:38.815278+0000","last_unstale":"2024-04-26T20:10:00.715645+0000","last_undegraded":"2024-04-26T20:10:00.715645+0000","last_fullsized":"2024-04-26T20:10:00.715645+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:27.522132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'36","reported_seq":598,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:07.627554+0000","last_change":"2024-04-26T20:07:31.711393+0000","last_active":"2024-04-26T20:13:07.627554+0000","last_peered":"2024-04-26T20:13:07.627554+0000","last_clean":"2024-04-26T20:13:07.627554+0000","last_became_active":"2024-04-26T20:07:31.710998+0000","last_became_peered":"2024-04-26T20:07:31.710998+0000","last_unstale":"2024-04-26T20:13:07.627554+0000","last_undegraded":"2024-04-26T20:13:07.627554+0000","last_fullsized":"2024-04-26T20:13:07.627554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:00.674950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60739,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":521,"num_read_kb":320,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.676036+0000","last_change":"2024-04-26T20:07:34.752774+0000","last_active":"2024-04-26T20:09:58.676036+0000","last_peered":"2024-04-26T20:09:58.676036+0000","last_clean":"2024-04-26T20:09:58.676036+0000","last_became_active":"2024-04-26T20:07:34.752471+0000","last_became_peered":"2024-04-26T20:07:34.752471+0000","last_unstale":"2024-04-26T20:09:58.676036+0000","last_undegraded":"2024-04-26T20:09:58.676036+0000","last_fullsized":"2024-04-26T20:09:58.676036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.646231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645658+0000","last_change":"2024-04-26T20:07:38.830322+0000","last_active":"2024-04-26T20:10:07.645658+0000","last_peered":"2024-04-26T20:10:07.645658+0000","last_clean":"2024-04-26T20:10:07.645658+0000","last_became_active":"2024-04-26T20:07:38.829910+0000","last_became_peered":"2024-04-26T20:07:38.829910+0000","last_unstale":"2024-04-26T20:10:07.645658+0000","last_undegraded":"2024-04-26T20:10:07.645658+0000","last_fullsized":"2024-04-26T20:10:07.645658+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:01.979793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"95'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.610124+0000","last_change":"2024-04-26T20:07:31.712054+0000","last_active":"2024-04-26T20:09:52.610124+0000","last_peered":"2024-04-26T20:09:52.610124+0000","last_clean":"2024-04-26T20:09:52.610124+0000","last_became_active":"2024-04-26T20:07:31.711618+0000","last_became_peered":"2024-04-26T20:07:31.711618+0000","last_unstale":"2024-04-26T20:09:52.610124+0000","last_undegraded":"2024-04-26T20:09:52.610124+0000","last_fullsized":"2024-04-26T20:09:52.610124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:46.465777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.646073+0000","last_change":"2024-04-26T20:07:34.772993+0000","last_active":"2024-04-26T20:09:57.646073+0000","last_peered":"2024-04-26T20:09:57.646073+0000","last_clean":"2024-04-26T20:09:57.646073+0000","last_became_active":"2024-04-26T20:07:34.772587+0000","last_became_peered":"2024-04-26T20:07:34.772587+0000","last_unstale":"2024-04-26T20:09:57.646073+0000","last_undegraded":"2024-04-26T20:09:57.646073+0000","last_fullsized":"2024-04-26T20:09:57.646073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:38.459869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.829340+0000","last_change":"2024-04-26T20:08:15.588307+0000","last_active":"2024-04-26T20:09:59.829340+0000","last_peered":"2024-04-26T20:09:59.829340+0000","last_clean":"2024-04-26T20:09:59.829340+0000","last_became_active":"2024-04-26T20:08:15.587978+0000","last_became_peered":"2024-04-26T20:08:15.587978+0000","last_unstale":"2024-04-26T20:09:59.829340+0000","last_undegraded":"2024-04-26T20:09:59.829340+0000","last_fullsized":"2024-04-26T20:09:59.829340+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:36.491892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643976+0000","last_change":"2024-04-26T20:07:31.720938+0000","last_active":"2024-04-26T20:09:52.643976+0000","last_peered":"2024-04-26T20:09:52.643976+0000","last_clean":"2024-04-26T20:09:52.643976+0000","last_became_active":"2024-04-26T20:07:31.720579+0000","last_became_peered":"2024-04-26T20:07:31.720579+0000","last_unstale":"2024-04-26T20:09:52.643976+0000","last_undegraded":"2024-04-26T20:09:52.643976+0000","last_fullsized":"2024-04-26T20:09:52.643976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:55.737976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268260+0000","last_change":"2024-04-26T20:07:34.764802+0000","last_active":"2024-04-26T20:09:58.268260+0000","last_peered":"2024-04-26T20:09:58.268260+0000","last_clean":"2024-04-26T20:09:58.268260+0000","last_became_active":"2024-04-26T20:07:34.763468+0000","last_became_peered":"2024-04-26T20:07:34.763468+0000","last_unstale":"2024-04-26T20:09:58.268260+0000","last_undegraded":"2024-04-26T20:09:58.268260+0000","last_fullsized":"2024-04-26T20:09:58.268260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:23.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.716233+0000","last_change":"2024-04-26T20:07:38.830392+0000","last_active":"2024-04-26T20:10:05.716233+0000","last_peered":"2024-04-26T20:10:05.716233+0000","last_clean":"2024-04-26T20:10:05.716233+0000","last_became_active":"2024-04-26T20:07:38.822964+0000","last_became_peered":"2024-04-26T20:07:38.822964+0000","last_unstale":"2024-04-26T20:10:05.716233+0000","last_undegraded":"2024-04-26T20:10:05.716233+0000","last_fullsized":"2024-04-26T20:10:05.716233+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:17.992368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'3","reported_seq":300,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:05.825283+0000","last_change":"2024-04-26T20:07:31.710567+0000","last_active":"2024-04-26T20:13:05.825283+0000","last_peered":"2024-04-26T20:13:05.825283+0000","last_clean":"2024-04-26T20:13:05.825283+0000","last_became_active":"2024-04-26T20:07:31.710197+0000","last_became_peered":"2024-04-26T20:07:31.710197+0000","last_unstale":"2024-04-26T20:13:05.825283+0000","last_undegraded":"2024-04-26T20:13:05.825283+0000","last_fullsized":"2024-04-26T20:13:05.825283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:10.648320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":257,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268229+0000","last_change":"2024-04-26T20:07:34.743237+0000","last_active":"2024-04-26T20:09:58.268229+0000","last_peered":"2024-04-26T20:09:58.268229+0000","last_clean":"2024-04-26T20:09:58.268229+0000","last_became_active":"2024-04-26T20:07:34.742146+0000","last_became_peered":"2024-04-26T20:07:34.742146+0000","last_unstale":"2024-04-26T20:09:58.268229+0000","last_undegraded":"2024-04-26T20:09:58.268229+0000","last_fullsized":"2024-04-26T20:09:58.268229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:15.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.647738+0000","last_change":"2024-04-26T20:08:15.591942+0000","last_active":"2024-04-26T20:10:07.647738+0000","last_peered":"2024-04-26T20:10:07.647738+0000","last_clean":"2024-04-26T20:10:07.647738+0000","last_became_active":"2024-04-26T20:08:15.590842+0000","last_became_peered":"2024-04-26T20:08:15.590842+0000","last_unstale":"2024-04-26T20:10:07.647738+0000","last_undegraded":"2024-04-26T20:10:07.647738+0000","last_fullsized":"2024-04-26T20:10:07.647738+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:44.344152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"95'15","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.456016+0000","last_change":"2024-04-26T20:07:31.716877+0000","last_active":"2024-04-26T20:09:55.456016+0000","last_peered":"2024-04-26T20:09:55.456016+0000","last_clean":"2024-04-26T20:09:55.456016+0000","last_became_active":"2024-04-26T20:07:31.716560+0000","last_became_peered":"2024-04-26T20:07:31.716560+0000","last_unstale":"2024-04-26T20:09:55.456016+0000","last_undegraded":"2024-04-26T20:09:55.456016+0000","last_fullsized":"2024-04-26T20:09:55.456016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:08:47.680991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.837358+0000","last_change":"2024-04-26T20:07:34.749893+0000","last_active":"2024-04-26T20:09:55.837358+0000","last_peered":"2024-04-26T20:09:55.837358+0000","last_clean":"2024-04-26T20:09:55.837358+0000","last_became_active":"2024-04-26T20:07:34.748535+0000","last_became_peered":"2024-04-26T20:07:34.748535+0000","last_unstale":"2024-04-26T20:09:55.837358+0000","last_undegraded":"2024-04-26T20:09:55.837358+0000","last_fullsized":"2024-04-26T20:09:55.837358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:32.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643963+0000","last_change":"2024-04-26T20:07:38.824046+0000","last_active":"2024-04-26T20:09:52.643963+0000","last_peered":"2024-04-26T20:09:52.643963+0000","last_clean":"2024-04-26T20:09:52.643963+0000","last_became_active":"2024-04-26T20:07:38.823549+0000","last_became_peered":"2024-04-26T20:07:38.823549+0000","last_unstale":"2024-04-26T20:09:52.643963+0000","last_undegraded":"2024-04-26T20:09:52.643963+0000","last_fullsized":"2024-04-26T20:09:52.643963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:26.218201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:50.455410+0000","last_change":"2024-04-26T20:07:31.711968+0000","last_active":"2024-04-26T20:09:50.455410+0000","last_peered":"2024-04-26T20:09:50.455410+0000","last_clean":"2024-04-26T20:09:50.455410+0000","last_became_active":"2024-04-26T20:07:31.710982+0000","last_became_peered":"2024-04-26T20:07:31.710982+0000","last_unstale":"2024-04-26T20:09:50.455410+0000","last_undegraded":"2024-04-26T20:09:50.455410+0000","last_fullsized":"2024-04-26T20:09:50.455410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:29.526536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.828290+0000","last_change":"2024-04-26T20:07:34.758101+0000","last_active":"2024-04-26T20:09:49.828290+0000","last_peered":"2024-04-26T20:09:49.828290+0000","last_clean":"2024-04-26T20:09:49.828290+0000","last_became_active":"2024-04-26T20:07:34.757747+0000","last_became_peered":"2024-04-26T20:07:34.757747+0000","last_unstale":"2024-04-26T20:09:49.828290+0000","last_undegraded":"2024-04-26T20:09:49.828290+0000","last_fullsized":"2024-04-26T20:09:49.828290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.337040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:45.454881+0000","last_change":"2024-04-26T20:07:39.312976+0000","last_active":"2024-04-26T20:09:45.454881+0000","last_peered":"2024-04-26T20:09:45.454881+0000","last_clean":"2024-04-26T20:09:45.454881+0000","last_became_active":"2024-04-26T20:07:39.312443+0000","last_became_peered":"2024-04-26T20:07:39.312443+0000","last_unstale":"2024-04-26T20:09:45.454881+0000","last_undegraded":"2024-04-26T20:09:45.454881+0000","last_fullsized":"2024-04-26T20:09:45.454881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:23.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:35.712065+0000","last_change":"2024-04-26T20:08:14.587592+0000","last_active":"2024-04-26T20:09:35.712065+0000","last_peered":"2024-04-26T20:09:35.712065+0000","last_clean":"2024-04-26T20:09:35.712065+0000","last_became_active":"2024-04-26T20:08:14.587157+0000","last_became_peered":"2024-04-26T20:08:14.587157+0000","last_unstale":"2024-04-26T20:09:35.712065+0000","last_undegraded":"2024-04-26T20:09:35.712065+0000","last_fullsized":"2024-04-26T20:09:35.712065+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:22.184416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.969101+0000","last_change":"2024-04-26T20:07:34.764376+0000","last_active":"2024-04-26T20:09:42.969101+0000","last_peered":"2024-04-26T20:09:42.969101+0000","last_clean":"2024-04-26T20:09:42.969101+0000","last_became_active":"2024-04-26T20:07:34.764040+0000","last_became_peered":"2024-04-26T20:07:34.764040+0000","last_unstale":"2024-04-26T20:09:42.969101+0000","last_undegraded":"2024-04-26T20:09:42.969101+0000","last_fullsized":"2024-04-26T20:09:42.969101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:43.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644412+0000","last_change":"2024-04-26T20:07:38.837075+0000","last_active":"2024-04-26T20:09:47.644412+0000","last_peered":"2024-04-26T20:09:47.644412+0000","last_clean":"2024-04-26T20:09:47.644412+0000","last_became_active":"2024-04-26T20:07:38.835912+0000","last_became_peered":"2024-04-26T20:07:38.835912+0000","last_unstale":"2024-04-26T20:09:47.644412+0000","last_undegraded":"2024-04-26T20:09:47.644412+0000","last_fullsized":"2024-04-26T20:09:47.644412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.853262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.266109+0000","last_change":"2024-04-26T20:07:31.728087+0000","last_active":"2024-04-26T20:09:38.266109+0000","last_peered":"2024-04-26T20:09:38.266109+0000","last_clean":"2024-04-26T20:09:38.266109+0000","last_became_active":"2024-04-26T20:07:31.727688+0000","last_became_peered":"2024-04-26T20:07:31.727688+0000","last_unstale":"2024-04-26T20:09:38.266109+0000","last_undegraded":"2024-04-26T20:09:38.266109+0000","last_fullsized":"2024-04-26T20:09:38.266109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:16.220049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.969049+0000","last_change":"2024-04-26T20:07:34.765161+0000","last_active":"2024-04-26T20:09:42.969049+0000","last_peered":"2024-04-26T20:09:42.969049+0000","last_clean":"2024-04-26T20:09:42.969049+0000","last_became_active":"2024-04-26T20:07:34.764789+0000","last_became_peered":"2024-04-26T20:07:34.764789+0000","last_unstale":"2024-04-26T20:09:42.969049+0000","last_undegraded":"2024-04-26T20:09:42.969049+0000","last_fullsized":"2024-04-26T20:09:42.969049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:52.417157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.864999+0000","last_change":"2024-04-26T20:07:39.313159+0000","last_active":"2024-04-26T20:09:49.864999+0000","last_peered":"2024-04-26T20:09:49.864999+0000","last_clean":"2024-04-26T20:09:49.864999+0000","last_became_active":"2024-04-26T20:07:39.312637+0000","last_became_peered":"2024-04-26T20:07:39.312637+0000","last_unstale":"2024-04-26T20:09:49.864999+0000","last_undegraded":"2024-04-26T20:09:49.864999+0000","last_fullsized":"2024-04-26T20:09:49.864999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:10.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.642222+0000","last_change":"2024-04-26T20:08:14.586352+0000","last_active":"2024-04-26T20:09:37.642222+0000","last_peered":"2024-04-26T20:09:37.642222+0000","last_clean":"2024-04-26T20:09:37.642222+0000","last_became_active":"2024-04-26T20:08:14.585975+0000","last_became_peered":"2024-04-26T20:08:14.585975+0000","last_unstale":"2024-04-26T20:09:37.642222+0000","last_undegraded":"2024-04-26T20:09:37.642222+0000","last_fullsized":"2024-04-26T20:09:37.642222+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:27.676683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'4","reported_seq":298,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:05.824458+0000","last_change":"2024-04-26T20:07:31.726272+0000","last_active":"2024-04-26T20:13:05.824458+0000","last_peered":"2024-04-26T20:13:05.824458+0000","last_clean":"2024-04-26T20:13:05.824458+0000","last_became_active":"2024-04-26T20:07:31.725889+0000","last_became_peered":"2024-04-26T20:07:31.725889+0000","last_unstale":"2024-04-26T20:13:05.824458+0000","last_undegraded":"2024-04-26T20:13:05.824458+0000","last_fullsized":"2024-04-26T20:13:05.824458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:03.091138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.266670+0000","last_change":"2024-04-26T20:07:38.815171+0000","last_active":"2024-04-26T20:09:43.266670+0000","last_peered":"2024-04-26T20:09:43.266670+0000","last_clean":"2024-04-26T20:09:43.266670+0000","last_became_active":"2024-04-26T20:07:38.814769+0000","last_became_peered":"2024-04-26T20:07:38.814769+0000","last_unstale":"2024-04-26T20:09:43.266670+0000","last_undegraded":"2024-04-26T20:09:43.266670+0000","last_fullsized":"2024-04-26T20:09:43.266670+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.506176+0000","last_change":"2024-04-26T20:07:34.766538+0000","last_active":"2024-04-26T20:09:38.506176+0000","last_peered":"2024-04-26T20:09:38.506176+0000","last_clean":"2024-04-26T20:09:38.506176+0000","last_became_active":"2024-04-26T20:07:34.761651+0000","last_became_peered":"2024-04-26T20:07:34.761651+0000","last_unstale":"2024-04-26T20:09:38.506176+0000","last_undegraded":"2024-04-26T20:09:38.506176+0000","last_fullsized":"2024-04-26T20:09:38.506176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:11.213046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'3","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.012132+0000","last_change":"2024-04-26T20:07:31.720351+0000","last_active":"2024-04-26T20:09:37.012132+0000","last_peered":"2024-04-26T20:09:37.012132+0000","last_clean":"2024-04-26T20:09:37.012132+0000","last_became_active":"2024-04-26T20:07:31.719966+0000","last_became_peered":"2024-04-26T20:07:31.719966+0000","last_unstale":"2024-04-26T20:09:37.012132+0000","last_undegraded":"2024-04-26T20:09:37.012132+0000","last_fullsized":"2024-04-26T20:09:37.012132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:17.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:28.271807+0000","last_change":"2024-04-26T20:05:56.337783+0000","last_active":"2024-04-26T20:10:28.271807+0000","last_peered":"2024-04-26T20:10:28.271807+0000","last_clean":"2024-04-26T20:10:28.271807+0000","last_became_active":"2024-04-26T20:05:56.337301+0000","last_became_peered":"2024-04-26T20:05:56.337301+0000","last_unstale":"2024-04-26T20:10:28.271807+0000","last_undegraded":"2024-04-26T20:10:28.271807+0000","last_fullsized":"2024-04-26T20:10:28.271807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_clean_scrub_stamp":"2024-04-26T20:01:58.332403+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:45.808606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643997+0000","last_change":"2024-04-26T20:07:39.313210+0000","last_active":"2024-04-26T20:09:52.643997+0000","last_peered":"2024-04-26T20:09:52.643997+0000","last_clean":"2024-04-26T20:09:52.643997+0000","last_became_active":"2024-04-26T20:07:39.312575+0000","last_became_peered":"2024-04-26T20:07:39.312575+0000","last_unstale":"2024-04-26T20:09:52.643997+0000","last_undegraded":"2024-04-26T20:09:52.643997+0000","last_fullsized":"2024-04-26T20:09:52.643997+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:02.877951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.643273+0000","last_change":"2024-04-26T20:07:34.759474+0000","last_active":"2024-04-26T20:09:42.643273+0000","last_peered":"2024-04-26T20:09:42.643273+0000","last_clean":"2024-04-26T20:09:42.643273+0000","last_became_active":"2024-04-26T20:07:34.759127+0000","last_became_peered":"2024-04-26T20:07:34.759127+0000","last_unstale":"2024-04-26T20:09:42.643273+0000","last_undegraded":"2024-04-26T20:09:42.643273+0000","last_fullsized":"2024-04-26T20:09:42.643273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.874866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:33.265636+0000","last_change":"2024-04-26T20:07:31.710633+0000","last_active":"2024-04-26T20:09:33.265636+0000","last_peered":"2024-04-26T20:09:33.265636+0000","last_clean":"2024-04-26T20:09:33.265636+0000","last_became_active":"2024-04-26T20:07:31.710290+0000","last_became_peered":"2024-04-26T20:07:31.710290+0000","last_unstale":"2024-04-26T20:09:33.265636+0000","last_undegraded":"2024-04-26T20:09:33.265636+0000","last_fullsized":"2024-04-26T20:09:33.265636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:18.114770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827720+0000","last_change":"2024-04-26T20:07:38.829768+0000","last_active":"2024-04-26T20:09:44.827720+0000","last_peered":"2024-04-26T20:09:44.827720+0000","last_clean":"2024-04-26T20:09:44.827720+0000","last_became_active":"2024-04-26T20:07:38.829364+0000","last_became_peered":"2024-04-26T20:07:38.829364+0000","last_unstale":"2024-04-26T20:09:44.827720+0000","last_undegraded":"2024-04-26T20:09:44.827720+0000","last_fullsized":"2024-04-26T20:09:44.827720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:56.887062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:40.454431+0000","last_change":"2024-04-26T20:07:31.720296+0000","last_active":"2024-04-26T20:09:40.454431+0000","last_peered":"2024-04-26T20:09:40.454431+0000","last_clean":"2024-04-26T20:09:40.454431+0000","last_became_active":"2024-04-26T20:07:31.719918+0000","last_became_peered":"2024-04-26T20:07:31.719918+0000","last_unstale":"2024-04-26T20:09:40.454431+0000","last_undegraded":"2024-04-26T20:09:40.454431+0000","last_fullsized":"2024-04-26T20:09:40.454431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:43.783104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.643331+0000","last_change":"2024-04-26T20:07:34.770634+0000","last_active":"2024-04-26T20:09:42.643331+0000","last_peered":"2024-04-26T20:09:42.643331+0000","last_clean":"2024-04-26T20:09:42.643331+0000","last_became_active":"2024-04-26T20:07:34.770170+0000","last_became_peered":"2024-04-26T20:07:34.770170+0000","last_unstale":"2024-04-26T20:09:42.643331+0000","last_undegraded":"2024-04-26T20:09:42.643331+0000","last_fullsized":"2024-04-26T20:09:42.643331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:21.528735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.576313+0000","last_change":"2024-04-26T20:07:39.317345+0000","last_active":"2024-04-26T20:09:39.576313+0000","last_peered":"2024-04-26T20:09:39.576313+0000","last_clean":"2024-04-26T20:09:39.576313+0000","last_became_active":"2024-04-26T20:07:39.315998+0000","last_became_peered":"2024-04-26T20:07:39.315998+0000","last_unstale":"2024-04-26T20:09:39.576313+0000","last_undegraded":"2024-04-26T20:09:39.576313+0000","last_fullsized":"2024-04-26T20:09:39.576313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:18.715213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'10","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:14.756534+0000","last_change":"2024-04-26T20:07:31.729195+0000","last_active":"2024-04-26T20:10:14.756534+0000","last_peered":"2024-04-26T20:10:14.756534+0000","last_clean":"2024-04-26T20:10:14.756534+0000","last_became_active":"2024-04-26T20:07:31.728746+0000","last_became_peered":"2024-04-26T20:07:31.728746+0000","last_unstale":"2024-04-26T20:10:14.756534+0000","last_undegraded":"2024-04-26T20:10:14.756534+0000","last_fullsized":"2024-04-26T20:10:14.756534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:34.443091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577252+0000","last_change":"2024-04-26T20:07:34.765013+0000","last_active":"2024-04-26T20:09:44.577252+0000","last_peered":"2024-04-26T20:09:44.577252+0000","last_clean":"2024-04-26T20:09:44.577252+0000","last_became_active":"2024-04-26T20:07:34.764597+0000","last_became_peered":"2024-04-26T20:07:34.764597+0000","last_unstale":"2024-04-26T20:09:44.577252+0000","last_undegraded":"2024-04-26T20:09:44.577252+0000","last_fullsized":"2024-04-26T20:09:44.577252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:51.513026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609504+0000","last_change":"2024-04-26T20:07:38.805910+0000","last_active":"2024-04-26T20:09:47.609504+0000","last_peered":"2024-04-26T20:09:47.609504+0000","last_clean":"2024-04-26T20:09:47.609504+0000","last_became_active":"2024-04-26T20:07:38.804580+0000","last_became_peered":"2024-04-26T20:07:38.804580+0000","last_unstale":"2024-04-26T20:09:47.609504+0000","last_undegraded":"2024-04-26T20:09:47.609504+0000","last_fullsized":"2024-04-26T20:09:47.609504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:41.321084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.012912+0000","last_change":"2024-04-26T20:07:31.915771+0000","last_active":"2024-04-26T20:09:42.012912+0000","last_peered":"2024-04-26T20:09:42.012912+0000","last_clean":"2024-04-26T20:09:42.012912+0000","last_became_active":"2024-04-26T20:07:31.915422+0000","last_became_peered":"2024-04-26T20:07:31.915422+0000","last_unstale":"2024-04-26T20:09:42.012912+0000","last_undegraded":"2024-04-26T20:09:42.012912+0000","last_fullsized":"2024-04-26T20:09:42.012912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:33.309270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.045699+0000","last_change":"2024-04-26T20:09:16.089155+0000","last_active":"2024-04-26T20:09:48.045699+0000","last_peered":"2024-04-26T20:09:48.045699+0000","last_clean":"2024-04-26T20:09:48.045699+0000","last_became_active":"2024-04-26T20:09:16.088425+0000","last_became_peered":"2024-04-26T20:09:16.088425+0000","last_unstale":"2024-04-26T20:09:48.045699+0000","last_undegraded":"2024-04-26T20:09:48.045699+0000","last_fullsized":"2024-04-26T20:09:48.045699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:00.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432128+0000","last_change":"2024-04-26T20:07:39.313079+0000","last_active":"2024-04-26T20:09:56.432128+0000","last_peered":"2024-04-26T20:09:56.432128+0000","last_clean":"2024-04-26T20:09:56.432128+0000","last_became_active":"2024-04-26T20:07:39.312603+0000","last_became_peered":"2024-04-26T20:07:39.312603+0000","last_unstale":"2024-04-26T20:09:56.432128+0000","last_undegraded":"2024-04-26T20:09:56.432128+0000","last_fullsized":"2024-04-26T20:09:56.432128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:45.717375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.184665+0000","last_change":"2024-04-26T20:07:31.715373+0000","last_active":"2024-04-26T20:09:43.184665+0000","last_peered":"2024-04-26T20:09:43.184665+0000","last_clean":"2024-04-26T20:09:43.184665+0000","last_became_active":"2024-04-26T20:07:31.714973+0000","last_became_peered":"2024-04-26T20:07:31.714973+0000","last_unstale":"2024-04-26T20:09:43.184665+0000","last_undegraded":"2024-04-26T20:09:43.184665+0000","last_fullsized":"2024-04-26T20:09:43.184665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.772671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:45.454845+0000","last_change":"2024-04-26T20:07:34.753619+0000","last_active":"2024-04-26T20:09:45.454845+0000","last_peered":"2024-04-26T20:09:45.454845+0000","last_clean":"2024-04-26T20:09:45.454845+0000","last_became_active":"2024-04-26T20:07:34.753263+0000","last_became_peered":"2024-04-26T20:07:34.753263+0000","last_unstale":"2024-04-26T20:09:45.454845+0000","last_undegraded":"2024-04-26T20:09:45.454845+0000","last_fullsized":"2024-04-26T20:09:45.454845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.358153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.715184+0000","last_change":"2024-04-26T20:08:15.840243+0000","last_active":"2024-04-26T20:09:55.715184+0000","last_peered":"2024-04-26T20:09:55.715184+0000","last_clean":"2024-04-26T20:09:55.715184+0000","last_became_active":"2024-04-26T20:08:15.838891+0000","last_became_peered":"2024-04-26T20:08:15.838891+0000","last_unstale":"2024-04-26T20:09:55.715184+0000","last_undegraded":"2024-04-26T20:09:55.715184+0000","last_fullsized":"2024-04-26T20:09:55.715184+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:15.018199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577186+0000","last_change":"2024-04-26T20:07:31.722516+0000","last_active":"2024-04-26T20:09:44.577186+0000","last_peered":"2024-04-26T20:09:44.577186+0000","last_clean":"2024-04-26T20:09:44.577186+0000","last_became_active":"2024-04-26T20:07:31.722096+0000","last_became_peered":"2024-04-26T20:07:31.722096+0000","last_unstale":"2024-04-26T20:09:44.577186+0000","last_undegraded":"2024-04-26T20:09:44.577186+0000","last_fullsized":"2024-04-26T20:09:44.577186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:57.693947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827697+0000","last_change":"2024-04-26T20:07:34.768754+0000","last_active":"2024-04-26T20:09:44.827697+0000","last_peered":"2024-04-26T20:09:44.827697+0000","last_clean":"2024-04-26T20:09:44.827697+0000","last_became_active":"2024-04-26T20:07:34.768405+0000","last_became_peered":"2024-04-26T20:07:34.768405+0000","last_unstale":"2024-04-26T20:09:44.827697+0000","last_undegraded":"2024-04-26T20:09:44.827697+0000","last_fullsized":"2024-04-26T20:09:44.827697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:08.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432095+0000","last_change":"2024-04-26T20:07:38.828804+0000","last_active":"2024-04-26T20:09:56.432095+0000","last_peered":"2024-04-26T20:09:56.432095+0000","last_clean":"2024-04-26T20:09:56.432095+0000","last_became_active":"2024-04-26T20:07:38.828452+0000","last_became_peered":"2024-04-26T20:07:38.828452+0000","last_unstale":"2024-04-26T20:09:56.432095+0000","last_undegraded":"2024-04-26T20:09:56.432095+0000","last_fullsized":"2024-04-26T20:09:56.432095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:29.905102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:11:52.622917+0000","last_change":"2024-04-26T20:07:31.710696+0000","last_active":"2024-04-26T20:11:52.622917+0000","last_peered":"2024-04-26T20:11:52.622917+0000","last_clean":"2024-04-26T20:11:52.622917+0000","last_became_active":"2024-04-26T20:07:31.710039+0000","last_became_peered":"2024-04-26T20:07:31.710039+0000","last_unstale":"2024-04-26T20:11:52.622917+0000","last_undegraded":"2024-04-26T20:11:52.622917+0000","last_fullsized":"2024-04-26T20:11:52.622917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:50.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.802095+0000","last_change":"2024-04-26T20:09:15.798031+0000","last_active":"2024-04-26T20:09:15.802095+0000","last_peered":"2024-04-26T20:09:15.802095+0000","last_clean":"2024-04-26T20:09:15.802095+0000","last_became_active":"2024-04-26T20:09:15.797535+0000","last_became_peered":"2024-04-26T20:09:15.797535+0000","last_unstale":"2024-04-26T20:09:15.802095+0000","last_undegraded":"2024-04-26T20:09:15.802095+0000","last_fullsized":"2024-04-26T20:09:15.802095+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:03.059618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432117+0000","last_change":"2024-04-26T20:07:39.312800+0000","last_active":"2024-04-26T20:09:56.432117+0000","last_peered":"2024-04-26T20:09:56.432117+0000","last_clean":"2024-04-26T20:09:56.432117+0000","last_became_active":"2024-04-26T20:07:39.312309+0000","last_became_peered":"2024-04-26T20:07:39.312309+0000","last_unstale":"2024-04-26T20:09:56.432117+0000","last_undegraded":"2024-04-26T20:09:56.432117+0000","last_fullsized":"2024-04-26T20:09:56.432117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:02.013468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:50.455381+0000","last_change":"2024-04-26T20:07:31.715509+0000","last_active":"2024-04-26T20:09:50.455381+0000","last_peered":"2024-04-26T20:09:50.455381+0000","last_clean":"2024-04-26T20:09:50.455381+0000","last_became_active":"2024-04-26T20:07:31.714606+0000","last_became_peered":"2024-04-26T20:07:31.714606+0000","last_unstale":"2024-04-26T20:09:50.455381+0000","last_undegraded":"2024-04-26T20:09:50.455381+0000","last_fullsized":"2024-04-26T20:09:50.455381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:03.979681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.267172+0000","last_change":"2024-04-26T20:07:34.763771+0000","last_active":"2024-04-26T20:09:48.267172+0000","last_peered":"2024-04-26T20:09:48.267172+0000","last_clean":"2024-04-26T20:09:48.267172+0000","last_became_active":"2024-04-26T20:07:34.763422+0000","last_became_peered":"2024-04-26T20:07:34.763422+0000","last_unstale":"2024-04-26T20:09:48.267172+0000","last_undegraded":"2024-04-26T20:09:48.267172+0000","last_fullsized":"2024-04-26T20:09:48.267172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:27.224024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.185094+0000","last_change":"2024-04-26T20:07:38.830156+0000","last_active":"2024-04-26T20:09:48.185094+0000","last_peered":"2024-04-26T20:09:48.185094+0000","last_clean":"2024-04-26T20:09:48.185094+0000","last_became_active":"2024-04-26T20:07:38.829530+0000","last_became_peered":"2024-04-26T20:07:38.829530+0000","last_unstale":"2024-04-26T20:09:48.185094+0000","last_undegraded":"2024-04-26T20:09:48.185094+0000","last_fullsized":"2024-04-26T20:09:48.185094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:21.720974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609555+0000","last_change":"2024-04-26T20:07:31.915638+0000","last_active":"2024-04-26T20:09:47.609555+0000","last_peered":"2024-04-26T20:09:47.609555+0000","last_clean":"2024-04-26T20:09:47.609555+0000","last_became_active":"2024-04-26T20:07:31.915266+0000","last_became_peered":"2024-04-26T20:07:31.915266+0000","last_unstale":"2024-04-26T20:09:47.609555+0000","last_undegraded":"2024-04-26T20:09:47.609555+0000","last_fullsized":"2024-04-26T20:09:47.609555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:51.452828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267679+0000","last_change":"2024-04-26T20:07:34.754476+0000","last_active":"2024-04-26T20:09:53.267679+0000","last_peered":"2024-04-26T20:09:53.267679+0000","last_clean":"2024-04-26T20:09:53.267679+0000","last_became_active":"2024-04-26T20:07:34.754054+0000","last_became_peered":"2024-04-26T20:07:34.754054+0000","last_unstale":"2024-04-26T20:09:53.267679+0000","last_undegraded":"2024-04-26T20:09:53.267679+0000","last_fullsized":"2024-04-26T20:09:53.267679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.497083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:51.675511+0000","last_change":"2024-04-26T20:07:38.835265+0000","last_active":"2024-04-26T20:09:51.675511+0000","last_peered":"2024-04-26T20:09:51.675511+0000","last_clean":"2024-04-26T20:09:51.675511+0000","last_became_active":"2024-04-26T20:07:38.833873+0000","last_became_peered":"2024-04-26T20:07:38.833873+0000","last_unstale":"2024-04-26T20:09:51.675511+0000","last_undegraded":"2024-04-26T20:09:51.675511+0000","last_fullsized":"2024-04-26T20:09:51.675511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:17.850236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.643274+0000","last_change":"2024-04-26T20:07:31.725576+0000","last_active":"2024-04-26T20:09:47.643274+0000","last_peered":"2024-04-26T20:09:47.643274+0000","last_clean":"2024-04-26T20:09:47.643274+0000","last_became_active":"2024-04-26T20:07:31.725166+0000","last_became_peered":"2024-04-26T20:07:31.725166+0000","last_unstale":"2024-04-26T20:09:47.643274+0000","last_undegraded":"2024-04-26T20:09:47.643274+0000","last_fullsized":"2024-04-26T20:09:47.643274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:04.539215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267647+0000","last_change":"2024-04-26T20:07:34.748263+0000","last_active":"2024-04-26T20:09:53.267647+0000","last_peered":"2024-04-26T20:09:53.267647+0000","last_clean":"2024-04-26T20:09:53.267647+0000","last_became_active":"2024-04-26T20:07:34.747889+0000","last_became_peered":"2024-04-26T20:07:34.747889+0000","last_unstale":"2024-04-26T20:09:53.267647+0000","last_undegraded":"2024-04-26T20:09:53.267647+0000","last_fullsized":"2024-04-26T20:09:53.267647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.257405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643935+0000","last_change":"2024-04-26T20:07:38.803999+0000","last_active":"2024-04-26T20:09:52.643935+0000","last_peered":"2024-04-26T20:09:52.643935+0000","last_clean":"2024-04-26T20:09:52.643935+0000","last_became_active":"2024-04-26T20:07:38.803585+0000","last_became_peered":"2024-04-26T20:07:38.803585+0000","last_unstale":"2024-04-26T20:09:52.643935+0000","last_undegraded":"2024-04-26T20:09:52.643935+0000","last_fullsized":"2024-04-26T20:09:52.643935+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:58.863511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644544+0000","last_change":"2024-04-26T20:07:31.730445+0000","last_active":"2024-04-26T20:09:47.644544+0000","last_peered":"2024-04-26T20:09:47.644544+0000","last_clean":"2024-04-26T20:09:47.644544+0000","last_became_active":"2024-04-26T20:07:31.729944+0000","last_became_peered":"2024-04-26T20:07:31.729944+0000","last_unstale":"2024-04-26T20:09:47.644544+0000","last_undegraded":"2024-04-26T20:09:47.644544+0000","last_fullsized":"2024-04-26T20:09:47.644544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:06.274097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267728+0000","last_change":"2024-04-26T20:07:34.755449+0000","last_active":"2024-04-26T20:09:53.267728+0000","last_peered":"2024-04-26T20:09:53.267728+0000","last_clean":"2024-04-26T20:09:53.267728+0000","last_became_active":"2024-04-26T20:07:34.755100+0000","last_became_peered":"2024-04-26T20:07:34.755100+0000","last_unstale":"2024-04-26T20:09:53.267728+0000","last_undegraded":"2024-04-26T20:09:53.267728+0000","last_fullsized":"2024-04-26T20:09:53.267728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.774751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.827010+0000","last_change":"2024-04-26T20:09:16.177597+0000","last_active":"2024-04-26T20:09:39.827010+0000","last_peered":"2024-04-26T20:09:39.827010+0000","last_clean":"2024-04-26T20:09:39.827010+0000","last_became_active":"2024-04-26T20:09:16.177150+0000","last_became_peered":"2024-04-26T20:09:16.177150+0000","last_unstale":"2024-04-26T20:09:39.827010+0000","last_undegraded":"2024-04-26T20:09:39.827010+0000","last_fullsized":"2024-04-26T20:09:39.827010+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:50.910457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,5],"acting":[11,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.608818+0000","last_change":"2024-04-26T20:07:34.747723+0000","last_active":"2024-04-26T20:09:42.608818+0000","last_peered":"2024-04-26T20:09:42.608818+0000","last_clean":"2024-04-26T20:09:42.608818+0000","last_became_active":"2024-04-26T20:07:34.747411+0000","last_became_peered":"2024-04-26T20:07:34.747411+0000","last_unstale":"2024-04-26T20:09:42.608818+0000","last_undegraded":"2024-04-26T20:09:42.608818+0000","last_fullsized":"2024-04-26T20:09:42.608818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.608845+0000","last_change":"2024-04-26T20:07:34.773012+0000","last_active":"2024-04-26T20:09:42.608845+0000","last_peered":"2024-04-26T20:09:42.608845+0000","last_clean":"2024-04-26T20:09:42.608845+0000","last_became_active":"2024-04-26T20:07:34.772735+0000","last_became_peered":"2024-04-26T20:07:34.772735+0000","last_unstale":"2024-04-26T20:09:42.608845+0000","last_undegraded":"2024-04-26T20:09:42.608845+0000","last_fullsized":"2024-04-26T20:09:42.608845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:08.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044906+0000","last_change":"2024-04-26T20:07:34.762977+0000","last_active":"2024-04-26T20:09:43.044906+0000","last_peered":"2024-04-26T20:09:43.044906+0000","last_clean":"2024-04-26T20:09:43.044906+0000","last_became_active":"2024-04-26T20:07:34.762530+0000","last_became_peered":"2024-04-26T20:07:34.762530+0000","last_unstale":"2024-04-26T20:09:43.044906+0000","last_undegraded":"2024-04-26T20:09:43.044906+0000","last_fullsized":"2024-04-26T20:09:43.044906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:24.811825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644495+0000","last_change":"2024-04-26T20:07:34.769986+0000","last_active":"2024-04-26T20:09:47.644495+0000","last_peered":"2024-04-26T20:09:47.644495+0000","last_clean":"2024-04-26T20:09:47.644495+0000","last_became_active":"2024-04-26T20:07:34.769591+0000","last_became_peered":"2024-04-26T20:07:34.769591+0000","last_unstale":"2024-04-26T20:09:47.644495+0000","last_undegraded":"2024-04-26T20:09:47.644495+0000","last_fullsized":"2024-04-26T20:09:47.644495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:27.466675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827735+0000","last_change":"2024-04-26T20:07:34.754733+0000","last_active":"2024-04-26T20:09:44.827735+0000","last_peered":"2024-04-26T20:09:44.827735+0000","last_clean":"2024-04-26T20:09:44.827735+0000","last_became_active":"2024-04-26T20:07:34.754374+0000","last_became_peered":"2024-04-26T20:07:34.754374+0000","last_unstale":"2024-04-26T20:09:44.827735+0000","last_undegraded":"2024-04-26T20:09:44.827735+0000","last_fullsized":"2024-04-26T20:09:44.827735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:02.732025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609576+0000","last_change":"2024-04-26T20:07:34.750329+0000","last_active":"2024-04-26T20:09:47.609576+0000","last_peered":"2024-04-26T20:09:47.609576+0000","last_clean":"2024-04-26T20:09:47.609576+0000","last_became_active":"2024-04-26T20:07:34.749970+0000","last_became_peered":"2024-04-26T20:07:34.749970+0000","last_unstale":"2024-04-26T20:09:47.609576+0000","last_undegraded":"2024-04-26T20:09:47.609576+0000","last_fullsized":"2024-04-26T20:09:47.609576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:50.375128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577278+0000","last_change":"2024-04-26T20:07:34.750623+0000","last_active":"2024-04-26T20:09:44.577278+0000","last_peered":"2024-04-26T20:09:44.577278+0000","last_clean":"2024-04-26T20:09:44.577278+0000","last_became_active":"2024-04-26T20:07:34.750314+0000","last_became_peered":"2024-04-26T20:07:34.750314+0000","last_unstale":"2024-04-26T20:09:44.577278+0000","last_undegraded":"2024-04-26T20:09:44.577278+0000","last_fullsized":"2024-04-26T20:09:44.577278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:16.773986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.643225+0000","last_change":"2024-04-26T20:07:34.755474+0000","last_active":"2024-04-26T20:09:47.643225+0000","last_peered":"2024-04-26T20:09:47.643225+0000","last_clean":"2024-04-26T20:09:47.643225+0000","last_became_active":"2024-04-26T20:07:34.755133+0000","last_became_peered":"2024-04-26T20:07:34.755133+0000","last_unstale":"2024-04-26T20:09:47.643225+0000","last_undegraded":"2024-04-26T20:09:47.643225+0000","last_fullsized":"2024-04-26T20:09:47.643225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:02.861643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.578236+0000","last_change":"2024-04-26T20:07:34.752727+0000","last_active":"2024-04-26T20:09:49.578236+0000","last_peered":"2024-04-26T20:09:49.578236+0000","last_clean":"2024-04-26T20:09:49.578236+0000","last_became_active":"2024-04-26T20:07:34.752420+0000","last_became_peered":"2024-04-26T20:07:34.752420+0000","last_unstale":"2024-04-26T20:09:49.578236+0000","last_undegraded":"2024-04-26T20:09:49.578236+0000","last_fullsized":"2024-04-26T20:09:49.578236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:45.076837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.185630+0000","last_change":"2024-04-26T20:07:34.775116+0000","last_active":"2024-04-26T20:09:53.185630+0000","last_peered":"2024-04-26T20:09:53.185630+0000","last_clean":"2024-04-26T20:09:53.185630+0000","last_became_active":"2024-04-26T20:07:34.774710+0000","last_became_peered":"2024-04-26T20:07:34.774710+0000","last_unstale":"2024-04-26T20:09:53.185630+0000","last_undegraded":"2024-04-26T20:09:53.185630+0000","last_fullsized":"2024-04-26T20:09:53.185630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:53.203918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.645416+0000","last_change":"2024-04-26T20:07:34.773245+0000","last_active":"2024-04-26T20:09:52.645416+0000","last_peered":"2024-04-26T20:09:52.645416+0000","last_clean":"2024-04-26T20:09:52.645416+0000","last_became_active":"2024-04-26T20:07:34.772879+0000","last_became_peered":"2024-04-26T20:07:34.772879+0000","last_unstale":"2024-04-26T20:09:52.645416+0000","last_undegraded":"2024-04-26T20:09:52.645416+0000","last_fullsized":"2024-04-26T20:09:52.645416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:54.450454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.828260+0000","last_change":"2024-04-26T20:07:34.753716+0000","last_active":"2024-04-26T20:09:49.828260+0000","last_peered":"2024-04-26T20:09:49.828260+0000","last_clean":"2024-04-26T20:09:49.828260+0000","last_became_active":"2024-04-26T20:07:34.753378+0000","last_became_peered":"2024-04-26T20:07:34.753378+0000","last_unstale":"2024-04-26T20:09:49.828260+0000","last_undegraded":"2024-04-26T20:09:49.828260+0000","last_fullsized":"2024-04-26T20:09:49.828260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:47.165305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65050,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1559,"num_read_kb":849,"num_write":174,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383765,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":554234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645401,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1802240,"data_stored":1202241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808929,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":558566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070567,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1130496,"data_stored":554390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332203,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1150609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495729,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":549646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":610295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":551034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411465,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40668,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704484,"statfs":{"total":95995035648,"available":95953391616,"internally_reserved":0,"allocated":1200128,"data_stored":612346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":549840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920893,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":554296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757367,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1146197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:13:15.151 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-26T20:13:15.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:13:15.234 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:13 smithi131 ceph-mon[27919]: pgmap v506: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:15.294 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:13 smithi131 ceph-mon[27919]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:15.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.608+0000 7f4999e00640 1 -- 172.21.15.131:0/1754441055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994104b70 msgr2=0x7f4994104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.608+0000 7f4999e00640 1 --2- 172.21.15.131:0/1754441055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994104b70 0x7f4994104f90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4984009900 tx=0x7f498402a400 comp rx=0 tx=0).stop 2024-04-26T20:13:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.609+0000 7f4999e00640 1 -- 172.21.15.131:0/1754441055 shutdown_connections 2024-04-26T20:13:15.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.609+0000 7f4999e00640 1 --2- 172.21.15.131:0/1754441055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994106720 0x7f499410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:15.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.609+0000 7f4999e00640 1 --2- 172.21.15.131:0/1754441055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4994105d60 0x7f49941061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:15.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.609+0000 7f4999e00640 1 --2- 172.21.15.131:0/1754441055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994104b70 0x7f4994104f90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:15.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.609+0000 7f4999e00640 1 -- 172.21.15.131:0/1754441055 >> 172.21.15.131:0/1754441055 conn(0x7f4994100120 msgr2=0x7f4994102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:15.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.609+0000 7f4999e00640 1 -- 172.21.15.131:0/1754441055 shutdown_connections 2024-04-26T20:13:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.609+0000 7f4999e00640 1 -- 172.21.15.131:0/1754441055 wait complete. 2024-04-26T20:13:15.999 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:15 smithi157 ceph-mon[32703]: pgmap v507: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:16.084 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:15 smithi148 ceph-mon[32281]: pgmap v507: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:16.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 Processor -- start 2024-04-26T20:13:16.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 -- start start 2024-04-26T20:13:16.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994104b70 0x7f499419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:16.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4994105d60 0x7f499419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:16.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994106720 0x7f49941a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:16.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49941106f0 con 0x7f4994106720 2024-04-26T20:13:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4994110570 con 0x7f4994105d60 2024-04-26T20:13:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4999e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4994110870 con 0x7f4994104b70 2024-04-26T20:13:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4993400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994106720 0x7f49941a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:16.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4992a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994104b70 0x7f499419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4992a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994104b70 0x7f499419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51824/0 (socket says 172.21.15.131:51824) 2024-04-26T20:13:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4992a00640 1 -- 172.21.15.131:0/2891010687 learned_addr learned my addr 172.21.15.131:0/2891010687 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:16.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4993400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994106720 0x7f49941a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47412/0 (socket says 172.21.15.131:47412) 2024-04-26T20:13:16.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.610+0000 7f4992000640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4994105d60 0x7f499419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:16.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4992a00640 1 -- 172.21.15.131:0/2891010687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4994105d60 msgr2=0x7f499419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:16.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4992a00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4994105d60 0x7f499419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:16.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4992a00640 1 -- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994106720 msgr2=0x7f49941a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:17.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4992a00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994106720 0x7f49941a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:17.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4992a00640 1 -- 172.21.15.131:0/2891010687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941a2990 con 0x7f4994104b70 2024-04-26T20:13:17.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4993400640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994106720 0x7f49941a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:13:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4992a00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994104b70 0x7f499419adc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4984031500 tx=0x7f4984003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:17.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4998a00640 1 -- 172.21.15.131:0/2891010687 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498403f070 con 0x7f4994104b70 2024-04-26T20:13:17.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4998a00640 1 -- 172.21.15.131:0/2891010687 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4984031c50 con 0x7f4994104b70 2024-04-26T20:13:17.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49941a2c20 con 0x7f4994104b70 2024-04-26T20:13:17.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.611+0000 7f4998a00640 1 -- 172.21.15.131:0/2891010687 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984035030 con 0x7f4994104b70 2024-04-26T20:13:17.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.612+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49941a3110 con 0x7f4994104b70 2024-04-26T20:13:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.613+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499410be90 con 0x7f4994104b70 2024-04-26T20:13:17.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.615+0000 7f4998a00640 1 -- 172.21.15.131:0/2891010687 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4984042070 con 0x7f4994104b70 2024-04-26T20:13:17.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.616+0000 7f4998a00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49780a3c40 0x7f49780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:17.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.616+0000 7f4998a00640 1 -- 172.21.15.131:0/2891010687 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f49840f1790 con 0x7f4994104b70 2024-04-26T20:13:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.617+0000 7f4992000640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49780a3c40 0x7f49780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:17.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.617+0000 7f4992000640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49780a3c40 0x7f49780a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f499419c280 tx=0x7f4980005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:17.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.619+0000 7f4998a00640 1 -- 172.21.15.131:0/2891010687 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f49840bd570 con 0x7f4994104b70 2024-04-26T20:13:18.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:15 smithi131 ceph-mon[27919]: pgmap v507: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:18.100 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:17 smithi157 ceph-mon[32703]: pgmap v508: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:18.269 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:17 smithi157 ceph-mon[32703]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:18.383 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:17 smithi148 ceph-mon[32281]: pgmap v508: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:18.456 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:17 smithi148 ceph-mon[32281]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.825+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 --> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4994062d70 con 0x7f49780a3c40 2024-04-26T20:13:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.854+0000 7f4998a00640 1 -- 172.21.15.131:0/2891010687 <== mgr.14152 v2:172.21.15.131:6800/2537375548 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340369 (secure 0 0 0) 0x7f4994062d70 con 0x7f49780a3c40 2024-04-26T20:13:18.592 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:13:18.661 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-26T20:13:18.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49780a3c40 msgr2=0x7f49780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:18.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49780a3c40 0x7f49780a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f499419c280 tx=0x7f4980005f90 comp rx=0 tx=0).stop 2024-04-26T20:13:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994104b70 msgr2=0x7f499419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:18.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994104b70 0x7f499419adc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4984031500 tx=0x7f4984003c10 comp rx=0 tx=0).stop 2024-04-26T20:13:18.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 shutdown_connections 2024-04-26T20:13:18.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4994106720 0x7f49941a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4994105d60 0x7f499419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:18.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f49780a3c40 0x7f49780a6100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:18.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 --2- 172.21.15.131:0/2891010687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994104b70 0x7f499419adc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:19.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 >> 172.21.15.131:0/2891010687 conn(0x7f4994100120 msgr2=0x7f4994109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 shutdown_connections 2024-04-26T20:13:19.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:15.861+0000 7f4999e00640 1 -- 172.21.15.131:0/2891010687 wait complete. 2024-04-26T20:13:19.288 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-04-26T20:13:15.750568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655652,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1715,"num_read_kb":973,"num_write":357,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368480,"kb_used_data":15140,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573344,"statfs":{"total":1151940427776,"available":1151563104256,"internally_reserved":0,"allocated":15503360,"data_stored":8593694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006521"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.827058+0000","last_change":"2024-04-26T20:07:34.771977+0000","last_active":"2024-04-26T20:09:39.827058+0000","last_peered":"2024-04-26T20:09:39.827058+0000","last_clean":"2024-04-26T20:09:39.827058+0000","last_became_active":"2024-04-26T20:07:34.771658+0000","last_became_peered":"2024-04-26T20:07:34.771658+0000","last_unstale":"2024-04-26T20:09:39.827058+0000","last_undegraded":"2024-04-26T20:09:39.827058+0000","last_fullsized":"2024-04-26T20:09:39.827058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:38.419057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044979+0000","last_change":"2024-04-26T20:07:34.768774+0000","last_active":"2024-04-26T20:09:43.044979+0000","last_peered":"2024-04-26T20:09:43.044979+0000","last_clean":"2024-04-26T20:09:43.044979+0000","last_became_active":"2024-04-26T20:07:34.768383+0000","last_became_peered":"2024-04-26T20:07:34.768383+0000","last_unstale":"2024-04-26T20:09:43.044979+0000","last_undegraded":"2024-04-26T20:09:43.044979+0000","last_fullsized":"2024-04-26T20:09:43.044979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:43.813261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044955+0000","last_change":"2024-04-26T20:07:34.761796+0000","last_active":"2024-04-26T20:09:43.044955+0000","last_peered":"2024-04-26T20:09:43.044955+0000","last_clean":"2024-04-26T20:09:43.044955+0000","last_became_active":"2024-04-26T20:07:34.761403+0000","last_became_peered":"2024-04-26T20:07:34.761403+0000","last_unstale":"2024-04-26T20:09:43.044955+0000","last_undegraded":"2024-04-26T20:09:43.044955+0000","last_fullsized":"2024-04-26T20:09:43.044955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:06.832304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.012166+0000","last_change":"2024-04-26T20:07:34.756728+0000","last_active":"2024-04-26T20:09:37.012166+0000","last_peered":"2024-04-26T20:09:37.012166+0000","last_clean":"2024-04-26T20:09:37.012166+0000","last_became_active":"2024-04-26T20:07:34.756406+0000","last_became_peered":"2024-04-26T20:07:34.756406+0000","last_unstale":"2024-04-26T20:09:37.012166+0000","last_undegraded":"2024-04-26T20:09:37.012166+0000","last_fullsized":"2024-04-26T20:09:37.012166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:27.753674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457110+0000","last_change":"2024-04-26T20:07:34.758243+0000","last_active":"2024-04-26T20:10:05.457110+0000","last_peered":"2024-04-26T20:10:05.457110+0000","last_clean":"2024-04-26T20:10:05.457110+0000","last_became_active":"2024-04-26T20:07:34.757868+0000","last_became_peered":"2024-04-26T20:07:34.757868+0000","last_unstale":"2024-04-26T20:10:05.457110+0000","last_undegraded":"2024-04-26T20:10:05.457110+0000","last_fullsized":"2024-04-26T20:10:05.457110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:53.361835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.971598+0000","last_change":"2024-04-26T20:08:15.585311+0000","last_active":"2024-04-26T20:10:07.971598+0000","last_peered":"2024-04-26T20:10:07.971598+0000","last_clean":"2024-04-26T20:10:07.971598+0000","last_became_active":"2024-04-26T20:08:15.584982+0000","last_became_peered":"2024-04-26T20:08:15.584982+0000","last_unstale":"2024-04-26T20:10:07.971598+0000","last_undegraded":"2024-04-26T20:10:07.971598+0000","last_fullsized":"2024-04-26T20:10:07.971598+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:56.828218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:08.269506+0000","last_change":"2024-04-26T20:07:34.757529+0000","last_active":"2024-04-26T20:10:08.269506+0000","last_peered":"2024-04-26T20:10:08.269506+0000","last_clean":"2024-04-26T20:10:08.269506+0000","last_became_active":"2024-04-26T20:07:34.757166+0000","last_became_peered":"2024-04-26T20:07:34.757166+0000","last_unstale":"2024-04-26T20:10:08.269506+0000","last_undegraded":"2024-04-26T20:10:08.269506+0000","last_fullsized":"2024-04-26T20:10:08.269506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:01.309698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:04.581136+0000","last_change":"2024-04-26T20:07:34.765352+0000","last_active":"2024-04-26T20:10:04.581136+0000","last_peered":"2024-04-26T20:10:04.581136+0000","last_clean":"2024-04-26T20:10:04.581136+0000","last_became_active":"2024-04-26T20:07:34.758400+0000","last_became_peered":"2024-04-26T20:07:34.758400+0000","last_unstale":"2024-04-26T20:10:04.581136+0000","last_undegraded":"2024-04-26T20:10:04.581136+0000","last_fullsized":"2024-04-26T20:10:04.581136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:43.940746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.186777+0000","last_change":"2024-04-26T20:07:34.754413+0000","last_active":"2024-04-26T20:10:03.186777+0000","last_peered":"2024-04-26T20:10:03.186777+0000","last_clean":"2024-04-26T20:10:03.186777+0000","last_became_active":"2024-04-26T20:07:34.754073+0000","last_became_peered":"2024-04-26T20:07:34.754073+0000","last_unstale":"2024-04-26T20:10:03.186777+0000","last_undegraded":"2024-04-26T20:10:03.186777+0000","last_fullsized":"2024-04-26T20:10:03.186777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:25.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.971090+0000","last_change":"2024-04-26T20:07:34.748739+0000","last_active":"2024-04-26T20:10:02.971090+0000","last_peered":"2024-04-26T20:10:02.971090+0000","last_clean":"2024-04-26T20:10:02.971090+0000","last_became_active":"2024-04-26T20:07:34.748378+0000","last_became_peered":"2024-04-26T20:07:34.748378+0000","last_unstale":"2024-04-26T20:10:02.971090+0000","last_undegraded":"2024-04-26T20:10:02.971090+0000","last_fullsized":"2024-04-26T20:10:02.971090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:13.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.268839+0000","last_change":"2024-04-26T20:07:34.747402+0000","last_active":"2024-04-26T20:10:03.268839+0000","last_peered":"2024-04-26T20:10:03.268839+0000","last_clean":"2024-04-26T20:10:03.268839+0000","last_became_active":"2024-04-26T20:07:34.746965+0000","last_became_peered":"2024-04-26T20:07:34.746965+0000","last_unstale":"2024-04-26T20:10:03.268839+0000","last_undegraded":"2024-04-26T20:10:03.268839+0000","last_fullsized":"2024-04-26T20:10:03.268839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:32.066286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:16.367770+0000","last_change":"2024-04-26T20:09:16.366157+0000","last_active":"2024-04-26T20:09:16.367770+0000","last_peered":"2024-04-26T20:09:16.367770+0000","last_clean":"2024-04-26T20:09:16.367770+0000","last_became_active":"2024-04-26T20:09:16.365132+0000","last_became_peered":"2024-04-26T20:09:16.365132+0000","last_unstale":"2024-04-26T20:09:16.367770+0000","last_undegraded":"2024-04-26T20:09:16.367770+0000","last_fullsized":"2024-04-26T20:09:16.367770+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.559981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.646861+0000","last_change":"2024-04-26T20:07:34.766542+0000","last_active":"2024-04-26T20:10:02.646861+0000","last_peered":"2024-04-26T20:10:02.646861+0000","last_clean":"2024-04-26T20:10:02.646861+0000","last_became_active":"2024-04-26T20:07:34.766185+0000","last_became_peered":"2024-04-26T20:07:34.766185+0000","last_unstale":"2024-04-26T20:10:02.646861+0000","last_undegraded":"2024-04-26T20:10:02.646861+0000","last_fullsized":"2024-04-26T20:10:02.646861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:00:34.499145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.047069+0000","last_change":"2024-04-26T20:07:34.762610+0000","last_active":"2024-04-26T20:09:58.047069+0000","last_peered":"2024-04-26T20:09:58.047069+0000","last_clean":"2024-04-26T20:09:58.047069+0000","last_became_active":"2024-04-26T20:07:34.762100+0000","last_became_peered":"2024-04-26T20:07:34.762100+0000","last_unstale":"2024-04-26T20:09:58.047069+0000","last_undegraded":"2024-04-26T20:09:58.047069+0000","last_fullsized":"2024-04-26T20:09:58.047069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:18.737170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.047033+0000","last_change":"2024-04-26T20:07:34.766754+0000","last_active":"2024-04-26T20:09:58.047033+0000","last_peered":"2024-04-26T20:09:58.047033+0000","last_clean":"2024-04-26T20:09:58.047033+0000","last_became_active":"2024-04-26T20:07:34.762204+0000","last_became_peered":"2024-04-26T20:07:34.762204+0000","last_unstale":"2024-04-26T20:09:58.047033+0000","last_undegraded":"2024-04-26T20:09:58.047033+0000","last_fullsized":"2024-04-26T20:09:58.047033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:25.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.186093+0000","last_change":"2024-04-26T20:07:34.769739+0000","last_active":"2024-04-26T20:09:58.186093+0000","last_peered":"2024-04-26T20:09:58.186093+0000","last_clean":"2024-04-26T20:09:58.186093+0000","last_became_active":"2024-04-26T20:07:34.769383+0000","last_became_peered":"2024-04-26T20:07:34.769383+0000","last_unstale":"2024-04-26T20:09:58.186093+0000","last_undegraded":"2024-04-26T20:09:58.186093+0000","last_fullsized":"2024-04-26T20:09:58.186093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:08.771590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:54.828723+0000","last_change":"2024-04-26T20:07:34.772010+0000","last_active":"2024-04-26T20:09:54.828723+0000","last_peered":"2024-04-26T20:09:54.828723+0000","last_clean":"2024-04-26T20:09:54.828723+0000","last_became_active":"2024-04-26T20:07:34.771682+0000","last_became_peered":"2024-04-26T20:07:34.771682+0000","last_unstale":"2024-04-26T20:09:54.828723+0000","last_undegraded":"2024-04-26T20:09:54.828723+0000","last_fullsized":"2024-04-26T20:09:54.828723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:57.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.211446+0000","last_change":"2024-04-26T20:07:34.757547+0000","last_active":"2024-04-26T20:09:55.211446+0000","last_peered":"2024-04-26T20:09:55.211446+0000","last_clean":"2024-04-26T20:09:55.211446+0000","last_became_active":"2024-04-26T20:07:34.757180+0000","last_became_peered":"2024-04-26T20:07:34.757180+0000","last_unstale":"2024-04-26T20:09:55.211446+0000","last_undegraded":"2024-04-26T20:09:55.211446+0000","last_fullsized":"2024-04-26T20:09:55.211446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:32.365944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.610166+0000","last_change":"2024-04-26T20:07:34.767363+0000","last_active":"2024-04-26T20:09:52.610166+0000","last_peered":"2024-04-26T20:09:52.610166+0000","last_clean":"2024-04-26T20:09:52.610166+0000","last_became_active":"2024-04-26T20:07:34.766909+0000","last_became_peered":"2024-04-26T20:07:34.766909+0000","last_unstale":"2024-04-26T20:09:52.610166+0000","last_undegraded":"2024-04-26T20:09:52.610166+0000","last_fullsized":"2024-04-26T20:09:52.610166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:32:31.500141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.612056+0000","last_change":"2024-04-26T20:07:38.823959+0000","last_active":"2024-04-26T20:10:07.612056+0000","last_peered":"2024-04-26T20:10:07.612056+0000","last_clean":"2024-04-26T20:10:07.612056+0000","last_became_active":"2024-04-26T20:07:38.823524+0000","last_became_peered":"2024-04-26T20:07:38.823524+0000","last_unstale":"2024-04-26T20:10:07.612056+0000","last_undegraded":"2024-04-26T20:10:07.612056+0000","last_fullsized":"2024-04-26T20:10:07.612056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:23.348376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'9","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:35.712160+0000","last_change":"2024-04-26T20:07:31.721324+0000","last_active":"2024-04-26T20:09:35.712160+0000","last_peered":"2024-04-26T20:09:35.712160+0000","last_clean":"2024-04-26T20:09:35.712160+0000","last_became_active":"2024-04-26T20:07:31.720140+0000","last_became_peered":"2024-04-26T20:07:31.720140+0000","last_unstale":"2024-04-26T20:09:35.712160+0000","last_undegraded":"2024-04-26T20:09:35.712160+0000","last_fullsized":"2024-04-26T20:09:35.712160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:27.636660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:34.575396+0000","last_change":"2024-04-26T20:07:34.760387+0000","last_active":"2024-04-26T20:09:34.575396+0000","last_peered":"2024-04-26T20:09:34.575396+0000","last_clean":"2024-04-26T20:09:34.575396+0000","last_became_active":"2024-04-26T20:07:34.760042+0000","last_became_peered":"2024-04-26T20:07:34.760042+0000","last_unstale":"2024-04-26T20:09:34.575396+0000","last_undegraded":"2024-04-26T20:09:34.575396+0000","last_fullsized":"2024-04-26T20:09:34.575396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:39.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.015378+0000","last_change":"2024-04-26T20:08:15.587181+0000","last_active":"2024-04-26T20:10:07.015378+0000","last_peered":"2024-04-26T20:10:07.015378+0000","last_clean":"2024-04-26T20:10:07.015378+0000","last_became_active":"2024-04-26T20:08:15.586847+0000","last_became_peered":"2024-04-26T20:08:15.586847+0000","last_unstale":"2024-04-26T20:10:07.015378+0000","last_undegraded":"2024-04-26T20:10:07.015378+0000","last_fullsized":"2024-04-26T20:10:07.015378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:12.495523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:34.575467+0000","last_change":"2024-04-26T20:07:31.711623+0000","last_active":"2024-04-26T20:09:34.575467+0000","last_peered":"2024-04-26T20:09:34.575467+0000","last_clean":"2024-04-26T20:09:34.575467+0000","last_became_active":"2024-04-26T20:07:31.710488+0000","last_became_peered":"2024-04-26T20:07:31.710488+0000","last_unstale":"2024-04-26T20:09:34.575467+0000","last_undegraded":"2024-04-26T20:09:34.575467+0000","last_fullsized":"2024-04-26T20:09:34.575467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:04.747170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.045052+0000","last_change":"2024-04-26T20:09:15.801039+0000","last_active":"2024-04-26T20:09:43.045052+0000","last_peered":"2024-04-26T20:09:43.045052+0000","last_clean":"2024-04-26T20:09:43.045052+0000","last_became_active":"2024-04-26T20:09:15.800049+0000","last_became_peered":"2024-04-26T20:09:15.800049+0000","last_unstale":"2024-04-26T20:09:43.045052+0000","last_undegraded":"2024-04-26T20:09:43.045052+0000","last_fullsized":"2024-04-26T20:09:43.045052+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:49.168625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,9],"acting":[0,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.016057+0000","last_change":"2024-04-26T20:07:38.822443+0000","last_active":"2024-04-26T20:10:12.016057+0000","last_peered":"2024-04-26T20:10:12.016057+0000","last_clean":"2024-04-26T20:10:12.016057+0000","last_became_active":"2024-04-26T20:07:38.822046+0000","last_became_peered":"2024-04-26T20:07:38.822046+0000","last_unstale":"2024-04-26T20:10:12.016057+0000","last_undegraded":"2024-04-26T20:10:12.016057+0000","last_fullsized":"2024-04-26T20:10:12.016057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:26.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457125+0000","last_change":"2024-04-26T20:07:31.715319+0000","last_active":"2024-04-26T20:10:05.457125+0000","last_peered":"2024-04-26T20:10:05.457125+0000","last_clean":"2024-04-26T20:10:05.457125+0000","last_became_active":"2024-04-26T20:07:31.714915+0000","last_became_peered":"2024-04-26T20:07:31.714915+0000","last_unstale":"2024-04-26T20:10:05.457125+0000","last_undegraded":"2024-04-26T20:10:05.457125+0000","last_fullsized":"2024-04-26T20:10:05.457125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:27.898652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:08.187267+0000","last_change":"2024-04-26T20:07:34.753422+0000","last_active":"2024-04-26T20:10:08.187267+0000","last_peered":"2024-04-26T20:10:08.187267+0000","last_clean":"2024-04-26T20:10:08.187267+0000","last_became_active":"2024-04-26T20:07:34.753009+0000","last_became_peered":"2024-04-26T20:07:34.753009+0000","last_unstale":"2024-04-26T20:10:08.187267+0000","last_undegraded":"2024-04-26T20:10:08.187267+0000","last_fullsized":"2024-04-26T20:10:08.187267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:37.369715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.016037+0000","last_change":"2024-04-26T20:07:38.830239+0000","last_active":"2024-04-26T20:10:12.016037+0000","last_peered":"2024-04-26T20:10:12.016037+0000","last_clean":"2024-04-26T20:10:12.016037+0000","last_became_active":"2024-04-26T20:07:38.829509+0000","last_became_peered":"2024-04-26T20:07:38.829509+0000","last_unstale":"2024-04-26T20:10:12.016037+0000","last_undegraded":"2024-04-26T20:10:12.016037+0000","last_fullsized":"2024-04-26T20:10:12.016037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:53.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:44.834478+0000","last_change":"2024-04-26T20:08:14.576129+0000","last_active":"2024-04-26T20:10:44.834478+0000","last_peered":"2024-04-26T20:10:44.834478+0000","last_clean":"2024-04-26T20:10:44.834478+0000","last_became_active":"2024-04-26T20:08:14.573168+0000","last_became_peered":"2024-04-26T20:08:14.573168+0000","last_unstale":"2024-04-26T20:10:44.834478+0000","last_undegraded":"2024-04-26T20:10:44.834478+0000","last_fullsized":"2024-04-26T20:10:44.834478+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:38.234154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645639+0000","last_change":"2024-04-26T20:07:34.749270+0000","last_active":"2024-04-26T20:10:07.645639+0000","last_peered":"2024-04-26T20:10:07.645639+0000","last_clean":"2024-04-26T20:10:07.645639+0000","last_became_active":"2024-04-26T20:07:34.748585+0000","last_became_peered":"2024-04-26T20:07:34.748585+0000","last_unstale":"2024-04-26T20:10:07.645639+0000","last_undegraded":"2024-04-26T20:10:07.645639+0000","last_fullsized":"2024-04-26T20:10:07.645639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:03.091720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.612582+0000","last_change":"2024-04-26T20:07:38.806247+0000","last_active":"2024-04-26T20:10:12.612582+0000","last_peered":"2024-04-26T20:10:12.612582+0000","last_clean":"2024-04-26T20:10:12.612582+0000","last_became_active":"2024-04-26T20:07:38.805748+0000","last_became_peered":"2024-04-26T20:07:38.805748+0000","last_unstale":"2024-04-26T20:10:12.612582+0000","last_undegraded":"2024-04-26T20:10:12.612582+0000","last_fullsized":"2024-04-26T20:10:12.612582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:16.962494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":313,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:11.634233+0000","last_change":"2024-04-26T20:07:31.713618+0000","last_active":"2024-04-26T20:13:11.634233+0000","last_peered":"2024-04-26T20:13:11.634233+0000","last_clean":"2024-04-26T20:13:11.634233+0000","last_became_active":"2024-04-26T20:07:31.713259+0000","last_became_peered":"2024-04-26T20:07:31.713259+0000","last_unstale":"2024-04-26T20:13:11.634233+0000","last_undegraded":"2024-04-26T20:13:11.634233+0000","last_fullsized":"2024-04-26T20:13:11.634233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:25.215087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":264,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.457088+0000","last_change":"2024-04-26T20:08:14.585688+0000","last_active":"2024-04-26T20:10:05.457088+0000","last_peered":"2024-04-26T20:10:05.457088+0000","last_clean":"2024-04-26T20:10:05.457088+0000","last_became_active":"2024-04-26T20:08:14.585337+0000","last_became_peered":"2024-04-26T20:08:14.585337+0000","last_unstale":"2024-04-26T20:10:05.457088+0000","last_undegraded":"2024-04-26T20:10:05.457088+0000","last_fullsized":"2024-04-26T20:10:05.457088+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:03.515457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:12.612549+0000","last_change":"2024-04-26T20:07:38.829772+0000","last_active":"2024-04-26T20:10:12.612549+0000","last_peered":"2024-04-26T20:10:12.612549+0000","last_clean":"2024-04-26T20:10:12.612549+0000","last_became_active":"2024-04-26T20:07:38.829327+0000","last_became_peered":"2024-04-26T20:07:38.829327+0000","last_unstale":"2024-04-26T20:10:12.612549+0000","last_undegraded":"2024-04-26T20:10:12.612549+0000","last_fullsized":"2024-04-26T20:10:12.612549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:37.252863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.971049+0000","last_change":"2024-04-26T20:07:31.728654+0000","last_active":"2024-04-26T20:10:02.971049+0000","last_peered":"2024-04-26T20:10:02.971049+0000","last_clean":"2024-04-26T20:10:02.971049+0000","last_became_active":"2024-04-26T20:07:31.728278+0000","last_became_peered":"2024-04-26T20:07:31.728278+0000","last_unstale":"2024-04-26T20:10:02.971049+0000","last_undegraded":"2024-04-26T20:10:02.971049+0000","last_fullsized":"2024-04-26T20:10:02.971049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:24.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645617+0000","last_change":"2024-04-26T20:07:34.739575+0000","last_active":"2024-04-26T20:10:07.645617+0000","last_peered":"2024-04-26T20:10:07.645617+0000","last_clean":"2024-04-26T20:10:07.645617+0000","last_became_active":"2024-04-26T20:07:34.739264+0000","last_became_peered":"2024-04-26T20:07:34.739264+0000","last_unstale":"2024-04-26T20:10:07.645617+0000","last_undegraded":"2024-04-26T20:10:07.645617+0000","last_fullsized":"2024-04-26T20:10:07.645617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:25.130064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827749+0000","last_change":"2024-04-26T20:07:38.822669+0000","last_active":"2024-04-26T20:09:44.827749+0000","last_peered":"2024-04-26T20:09:44.827749+0000","last_clean":"2024-04-26T20:09:44.827749+0000","last_became_active":"2024-04-26T20:07:38.822197+0000","last_became_peered":"2024-04-26T20:07:38.822197+0000","last_unstale":"2024-04-26T20:09:44.827749+0000","last_undegraded":"2024-04-26T20:09:44.827749+0000","last_fullsized":"2024-04-26T20:09:44.827749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:56.782794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"95'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.646035+0000","last_change":"2024-04-26T20:07:31.723527+0000","last_active":"2024-04-26T20:09:57.646035+0000","last_peered":"2024-04-26T20:09:57.646035+0000","last_clean":"2024-04-26T20:09:57.646035+0000","last_became_active":"2024-04-26T20:07:31.722574+0000","last_became_peered":"2024-04-26T20:07:31.722574+0000","last_unstale":"2024-04-26T20:09:57.646035+0000","last_undegraded":"2024-04-26T20:09:57.646035+0000","last_fullsized":"2024-04-26T20:09:57.646035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.521613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.580024+0000","last_change":"2024-04-26T20:07:34.771459+0000","last_active":"2024-04-26T20:09:59.580024+0000","last_peered":"2024-04-26T20:09:59.580024+0000","last_clean":"2024-04-26T20:09:59.580024+0000","last_became_active":"2024-04-26T20:07:34.771124+0000","last_became_peered":"2024-04-26T20:07:34.771124+0000","last_unstale":"2024-04-26T20:09:59.580024+0000","last_undegraded":"2024-04-26T20:09:59.580024+0000","last_fullsized":"2024-04-26T20:09:59.580024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:27.560440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.043976+0000","last_change":"2024-04-26T20:07:38.838858+0000","last_active":"2024-04-26T20:09:38.043976+0000","last_peered":"2024-04-26T20:09:38.043976+0000","last_clean":"2024-04-26T20:09:38.043976+0000","last_became_active":"2024-04-26T20:07:38.838125+0000","last_became_peered":"2024-04-26T20:07:38.838125+0000","last_unstale":"2024-04-26T20:09:38.043976+0000","last_undegraded":"2024-04-26T20:09:38.043976+0000","last_fullsized":"2024-04-26T20:09:38.043976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:01.856633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.644435+0000","last_change":"2024-04-26T20:07:31.710690+0000","last_active":"2024-04-26T20:09:57.644435+0000","last_peered":"2024-04-26T20:09:57.644435+0000","last_clean":"2024-04-26T20:09:57.644435+0000","last_became_active":"2024-04-26T20:07:31.710353+0000","last_became_peered":"2024-04-26T20:07:31.710353+0000","last_unstale":"2024-04-26T20:09:57.644435+0000","last_undegraded":"2024-04-26T20:09:57.644435+0000","last_fullsized":"2024-04-26T20:09:57.644435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:47.442673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715632+0000","last_change":"2024-04-26T20:07:34.776023+0000","last_active":"2024-04-26T20:10:00.715632+0000","last_peered":"2024-04-26T20:10:00.715632+0000","last_clean":"2024-04-26T20:10:00.715632+0000","last_became_active":"2024-04-26T20:07:34.775571+0000","last_became_peered":"2024-04-26T20:07:34.775571+0000","last_unstale":"2024-04-26T20:10:00.715632+0000","last_undegraded":"2024-04-26T20:10:00.715632+0000","last_fullsized":"2024-04-26T20:10:00.715632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:40.573046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.580074+0000","last_change":"2024-04-26T20:07:38.820636+0000","last_active":"2024-04-26T20:09:59.580074+0000","last_peered":"2024-04-26T20:09:59.580074+0000","last_clean":"2024-04-26T20:09:59.580074+0000","last_became_active":"2024-04-26T20:07:38.819390+0000","last_became_peered":"2024-04-26T20:07:38.819390+0000","last_unstale":"2024-04-26T20:09:59.580074+0000","last_undegraded":"2024-04-26T20:09:59.580074+0000","last_fullsized":"2024-04-26T20:09:59.580074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:58.793479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'4","reported_seq":305,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:11.633576+0000","last_change":"2024-04-26T20:07:31.724192+0000","last_active":"2024-04-26T20:13:11.633576+0000","last_peered":"2024-04-26T20:13:11.633576+0000","last_clean":"2024-04-26T20:13:11.633576+0000","last_became_active":"2024-04-26T20:07:31.723816+0000","last_became_peered":"2024-04-26T20:07:31.723816+0000","last_unstale":"2024-04-26T20:13:11.633576+0000","last_undegraded":"2024-04-26T20:13:11.633576+0000","last_fullsized":"2024-04-26T20:13:11.633576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:31.707282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":260,"num_read_kb":260,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.186757+0000","last_change":"2024-04-26T20:07:34.759027+0000","last_active":"2024-04-26T20:10:03.186757+0000","last_peered":"2024-04-26T20:10:03.186757+0000","last_clean":"2024-04-26T20:10:03.186757+0000","last_became_active":"2024-04-26T20:07:34.758689+0000","last_became_peered":"2024-04-26T20:07:34.758689+0000","last_unstale":"2024-04-26T20:10:03.186757+0000","last_undegraded":"2024-04-26T20:10:03.186757+0000","last_fullsized":"2024-04-26T20:10:03.186757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:29.110222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715598+0000","last_change":"2024-04-26T20:07:38.819000+0000","last_active":"2024-04-26T20:10:00.715598+0000","last_peered":"2024-04-26T20:10:00.715598+0000","last_clean":"2024-04-26T20:10:00.715598+0000","last_became_active":"2024-04-26T20:07:38.818526+0000","last_became_peered":"2024-04-26T20:07:38.818526+0000","last_unstale":"2024-04-26T20:10:00.715598+0000","last_undegraded":"2024-04-26T20:10:00.715598+0000","last_fullsized":"2024-04-26T20:10:00.715598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:28.830096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268278+0000","last_change":"2024-04-26T20:07:31.712952+0000","last_active":"2024-04-26T20:09:58.268278+0000","last_peered":"2024-04-26T20:09:58.268278+0000","last_clean":"2024-04-26T20:09:58.268278+0000","last_became_active":"2024-04-26T20:07:31.712081+0000","last_became_peered":"2024-04-26T20:07:31.712081+0000","last_unstale":"2024-04-26T20:09:58.268278+0000","last_undegraded":"2024-04-26T20:09:58.268278+0000","last_fullsized":"2024-04-26T20:09:58.268278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:46.923536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:03.047921+0000","last_change":"2024-04-26T20:09:16.367690+0000","last_active":"2024-04-26T20:10:03.047921+0000","last_peered":"2024-04-26T20:10:03.047921+0000","last_clean":"2024-04-26T20:10:03.047921+0000","last_became_active":"2024-04-26T20:09:16.366496+0000","last_became_peered":"2024-04-26T20:09:16.366496+0000","last_unstale":"2024-04-26T20:10:03.047921+0000","last_undegraded":"2024-04-26T20:10:03.047921+0000","last_fullsized":"2024-04-26T20:10:03.047921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:36.154062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:02.014903+0000","last_change":"2024-04-26T20:07:38.822520+0000","last_active":"2024-04-26T20:10:02.014903+0000","last_peered":"2024-04-26T20:10:02.014903+0000","last_clean":"2024-04-26T20:10:02.014903+0000","last_became_active":"2024-04-26T20:07:38.822056+0000","last_became_peered":"2024-04-26T20:07:38.822056+0000","last_unstale":"2024-04-26T20:10:02.014903+0000","last_undegraded":"2024-04-26T20:10:02.014903+0000","last_fullsized":"2024-04-26T20:10:02.014903+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.324996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"95'4","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.837386+0000","last_change":"2024-04-26T20:07:31.724015+0000","last_active":"2024-04-26T20:09:55.837386+0000","last_peered":"2024-04-26T20:09:55.837386+0000","last_clean":"2024-04-26T20:09:55.837386+0000","last_became_active":"2024-04-26T20:07:31.723686+0000","last_became_peered":"2024-04-26T20:07:31.723686+0000","last_unstale":"2024-04-26T20:09:55.837386+0000","last_undegraded":"2024-04-26T20:09:55.837386+0000","last_fullsized":"2024-04-26T20:09:55.837386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:55.925280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.186136+0000","last_change":"2024-04-26T20:07:34.750276+0000","last_active":"2024-04-26T20:09:58.186136+0000","last_peered":"2024-04-26T20:09:58.186136+0000","last_clean":"2024-04-26T20:09:58.186136+0000","last_became_active":"2024-04-26T20:07:34.749716+0000","last_became_peered":"2024-04-26T20:07:34.749716+0000","last_unstale":"2024-04-26T20:09:58.186136+0000","last_undegraded":"2024-04-26T20:09:58.186136+0000","last_fullsized":"2024-04-26T20:09:58.186136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:22.429780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:00.715645+0000","last_change":"2024-04-26T20:07:38.815636+0000","last_active":"2024-04-26T20:10:00.715645+0000","last_peered":"2024-04-26T20:10:00.715645+0000","last_clean":"2024-04-26T20:10:00.715645+0000","last_became_active":"2024-04-26T20:07:38.815278+0000","last_became_peered":"2024-04-26T20:07:38.815278+0000","last_unstale":"2024-04-26T20:10:00.715645+0000","last_undegraded":"2024-04-26T20:10:00.715645+0000","last_fullsized":"2024-04-26T20:10:00.715645+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:27.522132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'36","reported_seq":608,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:12.635046+0000","last_change":"2024-04-26T20:07:31.711393+0000","last_active":"2024-04-26T20:13:12.635046+0000","last_peered":"2024-04-26T20:13:12.635046+0000","last_clean":"2024-04-26T20:13:12.635046+0000","last_became_active":"2024-04-26T20:07:31.710998+0000","last_became_peered":"2024-04-26T20:07:31.710998+0000","last_unstale":"2024-04-26T20:13:12.635046+0000","last_undegraded":"2024-04-26T20:13:12.635046+0000","last_fullsized":"2024-04-26T20:13:12.635046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:00.674950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60739,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":531,"num_read_kb":325,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.676036+0000","last_change":"2024-04-26T20:07:34.752774+0000","last_active":"2024-04-26T20:09:58.676036+0000","last_peered":"2024-04-26T20:09:58.676036+0000","last_clean":"2024-04-26T20:09:58.676036+0000","last_became_active":"2024-04-26T20:07:34.752471+0000","last_became_peered":"2024-04-26T20:07:34.752471+0000","last_unstale":"2024-04-26T20:09:58.676036+0000","last_undegraded":"2024-04-26T20:09:58.676036+0000","last_fullsized":"2024-04-26T20:09:58.676036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.646231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.645658+0000","last_change":"2024-04-26T20:07:38.830322+0000","last_active":"2024-04-26T20:10:07.645658+0000","last_peered":"2024-04-26T20:10:07.645658+0000","last_clean":"2024-04-26T20:10:07.645658+0000","last_became_active":"2024-04-26T20:07:38.829910+0000","last_became_peered":"2024-04-26T20:07:38.829910+0000","last_unstale":"2024-04-26T20:10:07.645658+0000","last_undegraded":"2024-04-26T20:10:07.645658+0000","last_fullsized":"2024-04-26T20:10:07.645658+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:01.979793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"95'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.610124+0000","last_change":"2024-04-26T20:07:31.712054+0000","last_active":"2024-04-26T20:09:52.610124+0000","last_peered":"2024-04-26T20:09:52.610124+0000","last_clean":"2024-04-26T20:09:52.610124+0000","last_became_active":"2024-04-26T20:07:31.711618+0000","last_became_peered":"2024-04-26T20:07:31.711618+0000","last_unstale":"2024-04-26T20:09:52.610124+0000","last_undegraded":"2024-04-26T20:09:52.610124+0000","last_fullsized":"2024-04-26T20:09:52.610124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:46.465777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:57.646073+0000","last_change":"2024-04-26T20:07:34.772993+0000","last_active":"2024-04-26T20:09:57.646073+0000","last_peered":"2024-04-26T20:09:57.646073+0000","last_clean":"2024-04-26T20:09:57.646073+0000","last_became_active":"2024-04-26T20:07:34.772587+0000","last_became_peered":"2024-04-26T20:07:34.772587+0000","last_unstale":"2024-04-26T20:09:57.646073+0000","last_undegraded":"2024-04-26T20:09:57.646073+0000","last_fullsized":"2024-04-26T20:09:57.646073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:38.459869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:59.829340+0000","last_change":"2024-04-26T20:08:15.588307+0000","last_active":"2024-04-26T20:09:59.829340+0000","last_peered":"2024-04-26T20:09:59.829340+0000","last_clean":"2024-04-26T20:09:59.829340+0000","last_became_active":"2024-04-26T20:08:15.587978+0000","last_became_peered":"2024-04-26T20:08:15.587978+0000","last_unstale":"2024-04-26T20:09:59.829340+0000","last_undegraded":"2024-04-26T20:09:59.829340+0000","last_fullsized":"2024-04-26T20:09:59.829340+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:46:36.491892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643976+0000","last_change":"2024-04-26T20:07:31.720938+0000","last_active":"2024-04-26T20:09:52.643976+0000","last_peered":"2024-04-26T20:09:52.643976+0000","last_clean":"2024-04-26T20:09:52.643976+0000","last_became_active":"2024-04-26T20:07:31.720579+0000","last_became_peered":"2024-04-26T20:07:31.720579+0000","last_unstale":"2024-04-26T20:09:52.643976+0000","last_undegraded":"2024-04-26T20:09:52.643976+0000","last_fullsized":"2024-04-26T20:09:52.643976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:55.737976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268260+0000","last_change":"2024-04-26T20:07:34.764802+0000","last_active":"2024-04-26T20:09:58.268260+0000","last_peered":"2024-04-26T20:09:58.268260+0000","last_clean":"2024-04-26T20:09:58.268260+0000","last_became_active":"2024-04-26T20:07:34.763468+0000","last_became_peered":"2024-04-26T20:07:34.763468+0000","last_unstale":"2024-04-26T20:09:58.268260+0000","last_undegraded":"2024-04-26T20:09:58.268260+0000","last_fullsized":"2024-04-26T20:09:58.268260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:23.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:05.716233+0000","last_change":"2024-04-26T20:07:38.830392+0000","last_active":"2024-04-26T20:10:05.716233+0000","last_peered":"2024-04-26T20:10:05.716233+0000","last_clean":"2024-04-26T20:10:05.716233+0000","last_became_active":"2024-04-26T20:07:38.822964+0000","last_became_peered":"2024-04-26T20:07:38.822964+0000","last_unstale":"2024-04-26T20:10:05.716233+0000","last_undegraded":"2024-04-26T20:10:05.716233+0000","last_fullsized":"2024-04-26T20:10:05.716233+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:17.992368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'3","reported_seq":305,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:10.837066+0000","last_change":"2024-04-26T20:07:31.710567+0000","last_active":"2024-04-26T20:13:10.837066+0000","last_peered":"2024-04-26T20:13:10.837066+0000","last_clean":"2024-04-26T20:13:10.837066+0000","last_became_active":"2024-04-26T20:07:31.710197+0000","last_became_peered":"2024-04-26T20:07:31.710197+0000","last_unstale":"2024-04-26T20:13:10.837066+0000","last_undegraded":"2024-04-26T20:13:10.837066+0000","last_fullsized":"2024-04-26T20:13:10.837066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:10.648320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":262,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:58.268229+0000","last_change":"2024-04-26T20:07:34.743237+0000","last_active":"2024-04-26T20:09:58.268229+0000","last_peered":"2024-04-26T20:09:58.268229+0000","last_clean":"2024-04-26T20:09:58.268229+0000","last_became_active":"2024-04-26T20:07:34.742146+0000","last_became_peered":"2024-04-26T20:07:34.742146+0000","last_unstale":"2024-04-26T20:09:58.268229+0000","last_undegraded":"2024-04-26T20:09:58.268229+0000","last_fullsized":"2024-04-26T20:09:58.268229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:15.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:07.647738+0000","last_change":"2024-04-26T20:08:15.591942+0000","last_active":"2024-04-26T20:10:07.647738+0000","last_peered":"2024-04-26T20:10:07.647738+0000","last_clean":"2024-04-26T20:10:07.647738+0000","last_became_active":"2024-04-26T20:08:15.590842+0000","last_became_peered":"2024-04-26T20:08:15.590842+0000","last_unstale":"2024-04-26T20:10:07.647738+0000","last_undegraded":"2024-04-26T20:10:07.647738+0000","last_fullsized":"2024-04-26T20:10:07.647738+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:26:44.344152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"95'15","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.456016+0000","last_change":"2024-04-26T20:07:31.716877+0000","last_active":"2024-04-26T20:09:55.456016+0000","last_peered":"2024-04-26T20:09:55.456016+0000","last_clean":"2024-04-26T20:09:55.456016+0000","last_became_active":"2024-04-26T20:07:31.716560+0000","last_became_peered":"2024-04-26T20:07:31.716560+0000","last_unstale":"2024-04-26T20:09:55.456016+0000","last_undegraded":"2024-04-26T20:09:55.456016+0000","last_fullsized":"2024-04-26T20:09:55.456016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:08:47.680991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.837358+0000","last_change":"2024-04-26T20:07:34.749893+0000","last_active":"2024-04-26T20:09:55.837358+0000","last_peered":"2024-04-26T20:09:55.837358+0000","last_clean":"2024-04-26T20:09:55.837358+0000","last_became_active":"2024-04-26T20:07:34.748535+0000","last_became_peered":"2024-04-26T20:07:34.748535+0000","last_unstale":"2024-04-26T20:09:55.837358+0000","last_undegraded":"2024-04-26T20:09:55.837358+0000","last_fullsized":"2024-04-26T20:09:55.837358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:32.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643963+0000","last_change":"2024-04-26T20:07:38.824046+0000","last_active":"2024-04-26T20:09:52.643963+0000","last_peered":"2024-04-26T20:09:52.643963+0000","last_clean":"2024-04-26T20:09:52.643963+0000","last_became_active":"2024-04-26T20:07:38.823549+0000","last_became_peered":"2024-04-26T20:07:38.823549+0000","last_unstale":"2024-04-26T20:09:52.643963+0000","last_undegraded":"2024-04-26T20:09:52.643963+0000","last_fullsized":"2024-04-26T20:09:52.643963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:26.218201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:50.455410+0000","last_change":"2024-04-26T20:07:31.711968+0000","last_active":"2024-04-26T20:09:50.455410+0000","last_peered":"2024-04-26T20:09:50.455410+0000","last_clean":"2024-04-26T20:09:50.455410+0000","last_became_active":"2024-04-26T20:07:31.710982+0000","last_became_peered":"2024-04-26T20:07:31.710982+0000","last_unstale":"2024-04-26T20:09:50.455410+0000","last_undegraded":"2024-04-26T20:09:50.455410+0000","last_fullsized":"2024-04-26T20:09:50.455410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:29.526536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.828290+0000","last_change":"2024-04-26T20:07:34.758101+0000","last_active":"2024-04-26T20:09:49.828290+0000","last_peered":"2024-04-26T20:09:49.828290+0000","last_clean":"2024-04-26T20:09:49.828290+0000","last_became_active":"2024-04-26T20:07:34.757747+0000","last_became_peered":"2024-04-26T20:07:34.757747+0000","last_unstale":"2024-04-26T20:09:49.828290+0000","last_undegraded":"2024-04-26T20:09:49.828290+0000","last_fullsized":"2024-04-26T20:09:49.828290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.337040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:45.454881+0000","last_change":"2024-04-26T20:07:39.312976+0000","last_active":"2024-04-26T20:09:45.454881+0000","last_peered":"2024-04-26T20:09:45.454881+0000","last_clean":"2024-04-26T20:09:45.454881+0000","last_became_active":"2024-04-26T20:07:39.312443+0000","last_became_peered":"2024-04-26T20:07:39.312443+0000","last_unstale":"2024-04-26T20:09:45.454881+0000","last_undegraded":"2024-04-26T20:09:45.454881+0000","last_fullsized":"2024-04-26T20:09:45.454881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:23.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:35.712065+0000","last_change":"2024-04-26T20:08:14.587592+0000","last_active":"2024-04-26T20:09:35.712065+0000","last_peered":"2024-04-26T20:09:35.712065+0000","last_clean":"2024-04-26T20:09:35.712065+0000","last_became_active":"2024-04-26T20:08:14.587157+0000","last_became_peered":"2024-04-26T20:08:14.587157+0000","last_unstale":"2024-04-26T20:09:35.712065+0000","last_undegraded":"2024-04-26T20:09:35.712065+0000","last_fullsized":"2024-04-26T20:09:35.712065+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:22.184416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.969101+0000","last_change":"2024-04-26T20:07:34.764376+0000","last_active":"2024-04-26T20:09:42.969101+0000","last_peered":"2024-04-26T20:09:42.969101+0000","last_clean":"2024-04-26T20:09:42.969101+0000","last_became_active":"2024-04-26T20:07:34.764040+0000","last_became_peered":"2024-04-26T20:07:34.764040+0000","last_unstale":"2024-04-26T20:09:42.969101+0000","last_undegraded":"2024-04-26T20:09:42.969101+0000","last_fullsized":"2024-04-26T20:09:42.969101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:43.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644412+0000","last_change":"2024-04-26T20:07:38.837075+0000","last_active":"2024-04-26T20:09:47.644412+0000","last_peered":"2024-04-26T20:09:47.644412+0000","last_clean":"2024-04-26T20:09:47.644412+0000","last_became_active":"2024-04-26T20:07:38.835912+0000","last_became_peered":"2024-04-26T20:07:38.835912+0000","last_unstale":"2024-04-26T20:09:47.644412+0000","last_undegraded":"2024-04-26T20:09:47.644412+0000","last_fullsized":"2024-04-26T20:09:47.644412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.853262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.266109+0000","last_change":"2024-04-26T20:07:31.728087+0000","last_active":"2024-04-26T20:09:38.266109+0000","last_peered":"2024-04-26T20:09:38.266109+0000","last_clean":"2024-04-26T20:09:38.266109+0000","last_became_active":"2024-04-26T20:07:31.727688+0000","last_became_peered":"2024-04-26T20:07:31.727688+0000","last_unstale":"2024-04-26T20:09:38.266109+0000","last_undegraded":"2024-04-26T20:09:38.266109+0000","last_fullsized":"2024-04-26T20:09:38.266109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:16.220049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.969049+0000","last_change":"2024-04-26T20:07:34.765161+0000","last_active":"2024-04-26T20:09:42.969049+0000","last_peered":"2024-04-26T20:09:42.969049+0000","last_clean":"2024-04-26T20:09:42.969049+0000","last_became_active":"2024-04-26T20:07:34.764789+0000","last_became_peered":"2024-04-26T20:07:34.764789+0000","last_unstale":"2024-04-26T20:09:42.969049+0000","last_undegraded":"2024-04-26T20:09:42.969049+0000","last_fullsized":"2024-04-26T20:09:42.969049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:52.417157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.864999+0000","last_change":"2024-04-26T20:07:39.313159+0000","last_active":"2024-04-26T20:09:49.864999+0000","last_peered":"2024-04-26T20:09:49.864999+0000","last_clean":"2024-04-26T20:09:49.864999+0000","last_became_active":"2024-04-26T20:07:39.312637+0000","last_became_peered":"2024-04-26T20:07:39.312637+0000","last_unstale":"2024-04-26T20:09:49.864999+0000","last_undegraded":"2024-04-26T20:09:49.864999+0000","last_fullsized":"2024-04-26T20:09:49.864999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:10.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.642222+0000","last_change":"2024-04-26T20:08:14.586352+0000","last_active":"2024-04-26T20:09:37.642222+0000","last_peered":"2024-04-26T20:09:37.642222+0000","last_clean":"2024-04-26T20:09:37.642222+0000","last_became_active":"2024-04-26T20:08:14.585975+0000","last_became_peered":"2024-04-26T20:08:14.585975+0000","last_unstale":"2024-04-26T20:09:37.642222+0000","last_undegraded":"2024-04-26T20:09:37.642222+0000","last_fullsized":"2024-04-26T20:09:37.642222+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:27.676683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'4","reported_seq":303,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:13:10.836193+0000","last_change":"2024-04-26T20:07:31.726272+0000","last_active":"2024-04-26T20:13:10.836193+0000","last_peered":"2024-04-26T20:13:10.836193+0000","last_clean":"2024-04-26T20:13:10.836193+0000","last_became_active":"2024-04-26T20:07:31.725889+0000","last_became_peered":"2024-04-26T20:07:31.725889+0000","last_unstale":"2024-04-26T20:13:10.836193+0000","last_undegraded":"2024-04-26T20:13:10.836193+0000","last_fullsized":"2024-04-26T20:13:10.836193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:03.091138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":259,"num_read_kb":259,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.266670+0000","last_change":"2024-04-26T20:07:38.815171+0000","last_active":"2024-04-26T20:09:43.266670+0000","last_peered":"2024-04-26T20:09:43.266670+0000","last_clean":"2024-04-26T20:09:43.266670+0000","last_became_active":"2024-04-26T20:07:38.814769+0000","last_became_peered":"2024-04-26T20:07:38.814769+0000","last_unstale":"2024-04-26T20:09:43.266670+0000","last_undegraded":"2024-04-26T20:09:43.266670+0000","last_fullsized":"2024-04-26T20:09:43.266670+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:38.506176+0000","last_change":"2024-04-26T20:07:34.766538+0000","last_active":"2024-04-26T20:09:38.506176+0000","last_peered":"2024-04-26T20:09:38.506176+0000","last_clean":"2024-04-26T20:09:38.506176+0000","last_became_active":"2024-04-26T20:07:34.761651+0000","last_became_peered":"2024-04-26T20:07:34.761651+0000","last_unstale":"2024-04-26T20:09:38.506176+0000","last_undegraded":"2024-04-26T20:09:38.506176+0000","last_fullsized":"2024-04-26T20:09:38.506176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:11.213046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'3","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:37.012132+0000","last_change":"2024-04-26T20:07:31.720351+0000","last_active":"2024-04-26T20:09:37.012132+0000","last_peered":"2024-04-26T20:09:37.012132+0000","last_clean":"2024-04-26T20:09:37.012132+0000","last_became_active":"2024-04-26T20:07:31.719966+0000","last_became_peered":"2024-04-26T20:07:31.719966+0000","last_unstale":"2024-04-26T20:09:37.012132+0000","last_undegraded":"2024-04-26T20:09:37.012132+0000","last_fullsized":"2024-04-26T20:09:37.012132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:17.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:28.271807+0000","last_change":"2024-04-26T20:05:56.337783+0000","last_active":"2024-04-26T20:10:28.271807+0000","last_peered":"2024-04-26T20:10:28.271807+0000","last_clean":"2024-04-26T20:10:28.271807+0000","last_became_active":"2024-04-26T20:05:56.337301+0000","last_became_peered":"2024-04-26T20:05:56.337301+0000","last_unstale":"2024-04-26T20:10:28.271807+0000","last_undegraded":"2024-04-26T20:10:28.271807+0000","last_fullsized":"2024-04-26T20:10:28.271807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:58.332403+0000","last_clean_scrub_stamp":"2024-04-26T20:01:58.332403+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:45.808606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643997+0000","last_change":"2024-04-26T20:07:39.313210+0000","last_active":"2024-04-26T20:09:52.643997+0000","last_peered":"2024-04-26T20:09:52.643997+0000","last_clean":"2024-04-26T20:09:52.643997+0000","last_became_active":"2024-04-26T20:07:39.312575+0000","last_became_peered":"2024-04-26T20:07:39.312575+0000","last_unstale":"2024-04-26T20:09:52.643997+0000","last_undegraded":"2024-04-26T20:09:52.643997+0000","last_fullsized":"2024-04-26T20:09:52.643997+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:02.877951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.643273+0000","last_change":"2024-04-26T20:07:34.759474+0000","last_active":"2024-04-26T20:09:42.643273+0000","last_peered":"2024-04-26T20:09:42.643273+0000","last_clean":"2024-04-26T20:09:42.643273+0000","last_became_active":"2024-04-26T20:07:34.759127+0000","last_became_peered":"2024-04-26T20:07:34.759127+0000","last_unstale":"2024-04-26T20:09:42.643273+0000","last_undegraded":"2024-04-26T20:09:42.643273+0000","last_fullsized":"2024-04-26T20:09:42.643273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.874866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:33.265636+0000","last_change":"2024-04-26T20:07:31.710633+0000","last_active":"2024-04-26T20:09:33.265636+0000","last_peered":"2024-04-26T20:09:33.265636+0000","last_clean":"2024-04-26T20:09:33.265636+0000","last_became_active":"2024-04-26T20:07:31.710290+0000","last_became_peered":"2024-04-26T20:07:31.710290+0000","last_unstale":"2024-04-26T20:09:33.265636+0000","last_undegraded":"2024-04-26T20:09:33.265636+0000","last_fullsized":"2024-04-26T20:09:33.265636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:18.114770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827720+0000","last_change":"2024-04-26T20:07:38.829768+0000","last_active":"2024-04-26T20:09:44.827720+0000","last_peered":"2024-04-26T20:09:44.827720+0000","last_clean":"2024-04-26T20:09:44.827720+0000","last_became_active":"2024-04-26T20:07:38.829364+0000","last_became_peered":"2024-04-26T20:07:38.829364+0000","last_unstale":"2024-04-26T20:09:44.827720+0000","last_undegraded":"2024-04-26T20:09:44.827720+0000","last_fullsized":"2024-04-26T20:09:44.827720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:39:56.887062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"95'1","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:40.454431+0000","last_change":"2024-04-26T20:07:31.720296+0000","last_active":"2024-04-26T20:09:40.454431+0000","last_peered":"2024-04-26T20:09:40.454431+0000","last_clean":"2024-04-26T20:09:40.454431+0000","last_became_active":"2024-04-26T20:07:31.719918+0000","last_became_peered":"2024-04-26T20:07:31.719918+0000","last_unstale":"2024-04-26T20:09:40.454431+0000","last_undegraded":"2024-04-26T20:09:40.454431+0000","last_fullsized":"2024-04-26T20:09:40.454431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:43.783104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.643331+0000","last_change":"2024-04-26T20:07:34.770634+0000","last_active":"2024-04-26T20:09:42.643331+0000","last_peered":"2024-04-26T20:09:42.643331+0000","last_clean":"2024-04-26T20:09:42.643331+0000","last_became_active":"2024-04-26T20:07:34.770170+0000","last_became_peered":"2024-04-26T20:07:34.770170+0000","last_unstale":"2024-04-26T20:09:42.643331+0000","last_undegraded":"2024-04-26T20:09:42.643331+0000","last_fullsized":"2024-04-26T20:09:42.643331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:21.528735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.576313+0000","last_change":"2024-04-26T20:07:39.317345+0000","last_active":"2024-04-26T20:09:39.576313+0000","last_peered":"2024-04-26T20:09:39.576313+0000","last_clean":"2024-04-26T20:09:39.576313+0000","last_became_active":"2024-04-26T20:07:39.315998+0000","last_became_peered":"2024-04-26T20:07:39.315998+0000","last_unstale":"2024-04-26T20:09:39.576313+0000","last_undegraded":"2024-04-26T20:09:39.576313+0000","last_fullsized":"2024-04-26T20:09:39.576313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:18.715213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'10","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:10:14.756534+0000","last_change":"2024-04-26T20:07:31.729195+0000","last_active":"2024-04-26T20:10:14.756534+0000","last_peered":"2024-04-26T20:10:14.756534+0000","last_clean":"2024-04-26T20:10:14.756534+0000","last_became_active":"2024-04-26T20:07:31.728746+0000","last_became_peered":"2024-04-26T20:07:31.728746+0000","last_unstale":"2024-04-26T20:10:14.756534+0000","last_undegraded":"2024-04-26T20:10:14.756534+0000","last_fullsized":"2024-04-26T20:10:14.756534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:34.443091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577252+0000","last_change":"2024-04-26T20:07:34.765013+0000","last_active":"2024-04-26T20:09:44.577252+0000","last_peered":"2024-04-26T20:09:44.577252+0000","last_clean":"2024-04-26T20:09:44.577252+0000","last_became_active":"2024-04-26T20:07:34.764597+0000","last_became_peered":"2024-04-26T20:07:34.764597+0000","last_unstale":"2024-04-26T20:09:44.577252+0000","last_undegraded":"2024-04-26T20:09:44.577252+0000","last_fullsized":"2024-04-26T20:09:44.577252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:51.513026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609504+0000","last_change":"2024-04-26T20:07:38.805910+0000","last_active":"2024-04-26T20:09:47.609504+0000","last_peered":"2024-04-26T20:09:47.609504+0000","last_clean":"2024-04-26T20:09:47.609504+0000","last_became_active":"2024-04-26T20:07:38.804580+0000","last_became_peered":"2024-04-26T20:07:38.804580+0000","last_unstale":"2024-04-26T20:09:47.609504+0000","last_undegraded":"2024-04-26T20:09:47.609504+0000","last_fullsized":"2024-04-26T20:09:47.609504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:41.321084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"95'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.012912+0000","last_change":"2024-04-26T20:07:31.915771+0000","last_active":"2024-04-26T20:09:42.012912+0000","last_peered":"2024-04-26T20:09:42.012912+0000","last_clean":"2024-04-26T20:09:42.012912+0000","last_became_active":"2024-04-26T20:07:31.915422+0000","last_became_peered":"2024-04-26T20:07:31.915422+0000","last_unstale":"2024-04-26T20:09:42.012912+0000","last_undegraded":"2024-04-26T20:09:42.012912+0000","last_fullsized":"2024-04-26T20:09:42.012912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:33.309270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.045699+0000","last_change":"2024-04-26T20:09:16.089155+0000","last_active":"2024-04-26T20:09:48.045699+0000","last_peered":"2024-04-26T20:09:48.045699+0000","last_clean":"2024-04-26T20:09:48.045699+0000","last_became_active":"2024-04-26T20:09:16.088425+0000","last_became_peered":"2024-04-26T20:09:16.088425+0000","last_unstale":"2024-04-26T20:09:48.045699+0000","last_undegraded":"2024-04-26T20:09:48.045699+0000","last_fullsized":"2024-04-26T20:09:48.045699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:00.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432128+0000","last_change":"2024-04-26T20:07:39.313079+0000","last_active":"2024-04-26T20:09:56.432128+0000","last_peered":"2024-04-26T20:09:56.432128+0000","last_clean":"2024-04-26T20:09:56.432128+0000","last_became_active":"2024-04-26T20:07:39.312603+0000","last_became_peered":"2024-04-26T20:07:39.312603+0000","last_unstale":"2024-04-26T20:09:56.432128+0000","last_undegraded":"2024-04-26T20:09:56.432128+0000","last_fullsized":"2024-04-26T20:09:56.432128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:45.717375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.184665+0000","last_change":"2024-04-26T20:07:31.715373+0000","last_active":"2024-04-26T20:09:43.184665+0000","last_peered":"2024-04-26T20:09:43.184665+0000","last_clean":"2024-04-26T20:09:43.184665+0000","last_became_active":"2024-04-26T20:07:31.714973+0000","last_became_peered":"2024-04-26T20:07:31.714973+0000","last_unstale":"2024-04-26T20:09:43.184665+0000","last_undegraded":"2024-04-26T20:09:43.184665+0000","last_fullsized":"2024-04-26T20:09:43.184665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.772671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:45.454845+0000","last_change":"2024-04-26T20:07:34.753619+0000","last_active":"2024-04-26T20:09:45.454845+0000","last_peered":"2024-04-26T20:09:45.454845+0000","last_clean":"2024-04-26T20:09:45.454845+0000","last_became_active":"2024-04-26T20:07:34.753263+0000","last_became_peered":"2024-04-26T20:07:34.753263+0000","last_unstale":"2024-04-26T20:09:45.454845+0000","last_undegraded":"2024-04-26T20:09:45.454845+0000","last_fullsized":"2024-04-26T20:09:45.454845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.358153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:55.715184+0000","last_change":"2024-04-26T20:08:15.840243+0000","last_active":"2024-04-26T20:09:55.715184+0000","last_peered":"2024-04-26T20:09:55.715184+0000","last_clean":"2024-04-26T20:09:55.715184+0000","last_became_active":"2024-04-26T20:08:15.838891+0000","last_became_peered":"2024-04-26T20:08:15.838891+0000","last_unstale":"2024-04-26T20:09:55.715184+0000","last_undegraded":"2024-04-26T20:09:55.715184+0000","last_fullsized":"2024-04-26T20:09:55.715184+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:15.018199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"95'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577186+0000","last_change":"2024-04-26T20:07:31.722516+0000","last_active":"2024-04-26T20:09:44.577186+0000","last_peered":"2024-04-26T20:09:44.577186+0000","last_clean":"2024-04-26T20:09:44.577186+0000","last_became_active":"2024-04-26T20:07:31.722096+0000","last_became_peered":"2024-04-26T20:07:31.722096+0000","last_unstale":"2024-04-26T20:09:44.577186+0000","last_undegraded":"2024-04-26T20:09:44.577186+0000","last_fullsized":"2024-04-26T20:09:44.577186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:57.693947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827697+0000","last_change":"2024-04-26T20:07:34.768754+0000","last_active":"2024-04-26T20:09:44.827697+0000","last_peered":"2024-04-26T20:09:44.827697+0000","last_clean":"2024-04-26T20:09:44.827697+0000","last_became_active":"2024-04-26T20:07:34.768405+0000","last_became_peered":"2024-04-26T20:07:34.768405+0000","last_unstale":"2024-04-26T20:09:44.827697+0000","last_undegraded":"2024-04-26T20:09:44.827697+0000","last_fullsized":"2024-04-26T20:09:44.827697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:08.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432095+0000","last_change":"2024-04-26T20:07:38.828804+0000","last_active":"2024-04-26T20:09:56.432095+0000","last_peered":"2024-04-26T20:09:56.432095+0000","last_clean":"2024-04-26T20:09:56.432095+0000","last_became_active":"2024-04-26T20:07:38.828452+0000","last_became_peered":"2024-04-26T20:07:38.828452+0000","last_unstale":"2024-04-26T20:09:56.432095+0000","last_undegraded":"2024-04-26T20:09:56.432095+0000","last_fullsized":"2024-04-26T20:09:56.432095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:29.905102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:11:52.622917+0000","last_change":"2024-04-26T20:07:31.710696+0000","last_active":"2024-04-26T20:11:52.622917+0000","last_peered":"2024-04-26T20:11:52.622917+0000","last_clean":"2024-04-26T20:11:52.622917+0000","last_became_active":"2024-04-26T20:07:31.710039+0000","last_became_peered":"2024-04-26T20:07:31.710039+0000","last_unstale":"2024-04-26T20:11:52.622917+0000","last_undegraded":"2024-04-26T20:11:52.622917+0000","last_fullsized":"2024-04-26T20:11:52.622917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:50.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:15.802095+0000","last_change":"2024-04-26T20:09:15.798031+0000","last_active":"2024-04-26T20:09:15.802095+0000","last_peered":"2024-04-26T20:09:15.802095+0000","last_clean":"2024-04-26T20:09:15.802095+0000","last_became_active":"2024-04-26T20:09:15.797535+0000","last_became_peered":"2024-04-26T20:09:15.797535+0000","last_unstale":"2024-04-26T20:09:15.802095+0000","last_undegraded":"2024-04-26T20:09:15.802095+0000","last_fullsized":"2024-04-26T20:09:15.802095+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:03.059618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:56.432117+0000","last_change":"2024-04-26T20:07:39.312800+0000","last_active":"2024-04-26T20:09:56.432117+0000","last_peered":"2024-04-26T20:09:56.432117+0000","last_clean":"2024-04-26T20:09:56.432117+0000","last_became_active":"2024-04-26T20:07:39.312309+0000","last_became_peered":"2024-04-26T20:07:39.312309+0000","last_unstale":"2024-04-26T20:09:56.432117+0000","last_undegraded":"2024-04-26T20:09:56.432117+0000","last_fullsized":"2024-04-26T20:09:56.432117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:02.013468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:50.455381+0000","last_change":"2024-04-26T20:07:31.715509+0000","last_active":"2024-04-26T20:09:50.455381+0000","last_peered":"2024-04-26T20:09:50.455381+0000","last_clean":"2024-04-26T20:09:50.455381+0000","last_became_active":"2024-04-26T20:07:31.714606+0000","last_became_peered":"2024-04-26T20:07:31.714606+0000","last_unstale":"2024-04-26T20:09:50.455381+0000","last_undegraded":"2024-04-26T20:09:50.455381+0000","last_fullsized":"2024-04-26T20:09:50.455381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:03.979681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.267172+0000","last_change":"2024-04-26T20:07:34.763771+0000","last_active":"2024-04-26T20:09:48.267172+0000","last_peered":"2024-04-26T20:09:48.267172+0000","last_clean":"2024-04-26T20:09:48.267172+0000","last_became_active":"2024-04-26T20:07:34.763422+0000","last_became_peered":"2024-04-26T20:07:34.763422+0000","last_unstale":"2024-04-26T20:09:48.267172+0000","last_undegraded":"2024-04-26T20:09:48.267172+0000","last_fullsized":"2024-04-26T20:09:48.267172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:27.224024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:48.185094+0000","last_change":"2024-04-26T20:07:38.830156+0000","last_active":"2024-04-26T20:09:48.185094+0000","last_peered":"2024-04-26T20:09:48.185094+0000","last_clean":"2024-04-26T20:09:48.185094+0000","last_became_active":"2024-04-26T20:07:38.829530+0000","last_became_peered":"2024-04-26T20:07:38.829530+0000","last_unstale":"2024-04-26T20:09:48.185094+0000","last_undegraded":"2024-04-26T20:09:48.185094+0000","last_fullsized":"2024-04-26T20:09:48.185094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:21.720974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609555+0000","last_change":"2024-04-26T20:07:31.915638+0000","last_active":"2024-04-26T20:09:47.609555+0000","last_peered":"2024-04-26T20:09:47.609555+0000","last_clean":"2024-04-26T20:09:47.609555+0000","last_became_active":"2024-04-26T20:07:31.915266+0000","last_became_peered":"2024-04-26T20:07:31.915266+0000","last_unstale":"2024-04-26T20:09:47.609555+0000","last_undegraded":"2024-04-26T20:09:47.609555+0000","last_fullsized":"2024-04-26T20:09:47.609555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:51.452828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267679+0000","last_change":"2024-04-26T20:07:34.754476+0000","last_active":"2024-04-26T20:09:53.267679+0000","last_peered":"2024-04-26T20:09:53.267679+0000","last_clean":"2024-04-26T20:09:53.267679+0000","last_became_active":"2024-04-26T20:07:34.754054+0000","last_became_peered":"2024-04-26T20:07:34.754054+0000","last_unstale":"2024-04-26T20:09:53.267679+0000","last_undegraded":"2024-04-26T20:09:53.267679+0000","last_fullsized":"2024-04-26T20:09:53.267679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.497083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:51.675511+0000","last_change":"2024-04-26T20:07:38.835265+0000","last_active":"2024-04-26T20:09:51.675511+0000","last_peered":"2024-04-26T20:09:51.675511+0000","last_clean":"2024-04-26T20:09:51.675511+0000","last_became_active":"2024-04-26T20:07:38.833873+0000","last_became_peered":"2024-04-26T20:07:38.833873+0000","last_unstale":"2024-04-26T20:09:51.675511+0000","last_undegraded":"2024-04-26T20:09:51.675511+0000","last_fullsized":"2024-04-26T20:09:51.675511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:17.850236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.643274+0000","last_change":"2024-04-26T20:07:31.725576+0000","last_active":"2024-04-26T20:09:47.643274+0000","last_peered":"2024-04-26T20:09:47.643274+0000","last_clean":"2024-04-26T20:09:47.643274+0000","last_became_active":"2024-04-26T20:07:31.725166+0000","last_became_peered":"2024-04-26T20:07:31.725166+0000","last_unstale":"2024-04-26T20:09:47.643274+0000","last_undegraded":"2024-04-26T20:09:47.643274+0000","last_fullsized":"2024-04-26T20:09:47.643274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:04.539215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267647+0000","last_change":"2024-04-26T20:07:34.748263+0000","last_active":"2024-04-26T20:09:53.267647+0000","last_peered":"2024-04-26T20:09:53.267647+0000","last_clean":"2024-04-26T20:09:53.267647+0000","last_became_active":"2024-04-26T20:07:34.747889+0000","last_became_peered":"2024-04-26T20:07:34.747889+0000","last_unstale":"2024-04-26T20:09:53.267647+0000","last_undegraded":"2024-04-26T20:09:53.267647+0000","last_fullsized":"2024-04-26T20:09:53.267647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.257405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.643935+0000","last_change":"2024-04-26T20:07:38.803999+0000","last_active":"2024-04-26T20:09:52.643935+0000","last_peered":"2024-04-26T20:09:52.643935+0000","last_clean":"2024-04-26T20:09:52.643935+0000","last_became_active":"2024-04-26T20:07:38.803585+0000","last_became_peered":"2024-04-26T20:07:38.803585+0000","last_unstale":"2024-04-26T20:09:52.643935+0000","last_undegraded":"2024-04-26T20:09:52.643935+0000","last_fullsized":"2024-04-26T20:09:52.643935+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:37.772732+0000","last_clean_scrub_stamp":"2024-04-26T20:07:37.772732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:58.863511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644544+0000","last_change":"2024-04-26T20:07:31.730445+0000","last_active":"2024-04-26T20:09:47.644544+0000","last_peered":"2024-04-26T20:09:47.644544+0000","last_clean":"2024-04-26T20:09:47.644544+0000","last_became_active":"2024-04-26T20:07:31.729944+0000","last_became_peered":"2024-04-26T20:07:31.729944+0000","last_unstale":"2024-04-26T20:09:47.644544+0000","last_undegraded":"2024-04-26T20:09:47.644544+0000","last_fullsized":"2024-04-26T20:09:47.644544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:30.663376+0000","last_clean_scrub_stamp":"2024-04-26T20:07:30.663376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:06.274097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.267728+0000","last_change":"2024-04-26T20:07:34.755449+0000","last_active":"2024-04-26T20:09:53.267728+0000","last_peered":"2024-04-26T20:09:53.267728+0000","last_clean":"2024-04-26T20:09:53.267728+0000","last_became_active":"2024-04-26T20:07:34.755100+0000","last_became_peered":"2024-04-26T20:07:34.755100+0000","last_unstale":"2024-04-26T20:09:53.267728+0000","last_undegraded":"2024-04-26T20:09:53.267728+0000","last_fullsized":"2024-04-26T20:09:53.267728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:19.774751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:39.827010+0000","last_change":"2024-04-26T20:09:16.177597+0000","last_active":"2024-04-26T20:09:39.827010+0000","last_peered":"2024-04-26T20:09:39.827010+0000","last_clean":"2024-04-26T20:09:39.827010+0000","last_became_active":"2024-04-26T20:09:16.177150+0000","last_became_peered":"2024-04-26T20:09:16.177150+0000","last_unstale":"2024-04-26T20:09:39.827010+0000","last_undegraded":"2024-04-26T20:09:39.827010+0000","last_fullsized":"2024-04-26T20:09:39.827010+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:50.910457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,5],"acting":[11,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.608818+0000","last_change":"2024-04-26T20:07:34.747723+0000","last_active":"2024-04-26T20:09:42.608818+0000","last_peered":"2024-04-26T20:09:42.608818+0000","last_clean":"2024-04-26T20:09:42.608818+0000","last_became_active":"2024-04-26T20:07:34.747411+0000","last_became_peered":"2024-04-26T20:07:34.747411+0000","last_unstale":"2024-04-26T20:09:42.608818+0000","last_undegraded":"2024-04-26T20:09:42.608818+0000","last_fullsized":"2024-04-26T20:09:42.608818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:42.608845+0000","last_change":"2024-04-26T20:07:34.773012+0000","last_active":"2024-04-26T20:09:42.608845+0000","last_peered":"2024-04-26T20:09:42.608845+0000","last_clean":"2024-04-26T20:09:42.608845+0000","last_became_active":"2024-04-26T20:07:34.772735+0000","last_became_peered":"2024-04-26T20:07:34.772735+0000","last_unstale":"2024-04-26T20:09:42.608845+0000","last_undegraded":"2024-04-26T20:09:42.608845+0000","last_fullsized":"2024-04-26T20:09:42.608845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:08.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:43.044906+0000","last_change":"2024-04-26T20:07:34.762977+0000","last_active":"2024-04-26T20:09:43.044906+0000","last_peered":"2024-04-26T20:09:43.044906+0000","last_clean":"2024-04-26T20:09:43.044906+0000","last_became_active":"2024-04-26T20:07:34.762530+0000","last_became_peered":"2024-04-26T20:07:34.762530+0000","last_unstale":"2024-04-26T20:09:43.044906+0000","last_undegraded":"2024-04-26T20:09:43.044906+0000","last_fullsized":"2024-04-26T20:09:43.044906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:24.811825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.644495+0000","last_change":"2024-04-26T20:07:34.769986+0000","last_active":"2024-04-26T20:09:47.644495+0000","last_peered":"2024-04-26T20:09:47.644495+0000","last_clean":"2024-04-26T20:09:47.644495+0000","last_became_active":"2024-04-26T20:07:34.769591+0000","last_became_peered":"2024-04-26T20:07:34.769591+0000","last_unstale":"2024-04-26T20:09:47.644495+0000","last_undegraded":"2024-04-26T20:09:47.644495+0000","last_fullsized":"2024-04-26T20:09:47.644495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:27.466675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.827735+0000","last_change":"2024-04-26T20:07:34.754733+0000","last_active":"2024-04-26T20:09:44.827735+0000","last_peered":"2024-04-26T20:09:44.827735+0000","last_clean":"2024-04-26T20:09:44.827735+0000","last_became_active":"2024-04-26T20:07:34.754374+0000","last_became_peered":"2024-04-26T20:07:34.754374+0000","last_unstale":"2024-04-26T20:09:44.827735+0000","last_undegraded":"2024-04-26T20:09:44.827735+0000","last_fullsized":"2024-04-26T20:09:44.827735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:02.732025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.609576+0000","last_change":"2024-04-26T20:07:34.750329+0000","last_active":"2024-04-26T20:09:47.609576+0000","last_peered":"2024-04-26T20:09:47.609576+0000","last_clean":"2024-04-26T20:09:47.609576+0000","last_became_active":"2024-04-26T20:07:34.749970+0000","last_became_peered":"2024-04-26T20:07:34.749970+0000","last_unstale":"2024-04-26T20:09:47.609576+0000","last_undegraded":"2024-04-26T20:09:47.609576+0000","last_fullsized":"2024-04-26T20:09:47.609576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:50.375128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:44.577278+0000","last_change":"2024-04-26T20:07:34.750623+0000","last_active":"2024-04-26T20:09:44.577278+0000","last_peered":"2024-04-26T20:09:44.577278+0000","last_clean":"2024-04-26T20:09:44.577278+0000","last_became_active":"2024-04-26T20:07:34.750314+0000","last_became_peered":"2024-04-26T20:07:34.750314+0000","last_unstale":"2024-04-26T20:09:44.577278+0000","last_undegraded":"2024-04-26T20:09:44.577278+0000","last_fullsized":"2024-04-26T20:09:44.577278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:16.773986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:47.643225+0000","last_change":"2024-04-26T20:07:34.755474+0000","last_active":"2024-04-26T20:09:47.643225+0000","last_peered":"2024-04-26T20:09:47.643225+0000","last_clean":"2024-04-26T20:09:47.643225+0000","last_became_active":"2024-04-26T20:07:34.755133+0000","last_became_peered":"2024-04-26T20:07:34.755133+0000","last_unstale":"2024-04-26T20:09:47.643225+0000","last_undegraded":"2024-04-26T20:09:47.643225+0000","last_fullsized":"2024-04-26T20:09:47.643225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:02.861643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.578236+0000","last_change":"2024-04-26T20:07:34.752727+0000","last_active":"2024-04-26T20:09:49.578236+0000","last_peered":"2024-04-26T20:09:49.578236+0000","last_clean":"2024-04-26T20:09:49.578236+0000","last_became_active":"2024-04-26T20:07:34.752420+0000","last_became_peered":"2024-04-26T20:07:34.752420+0000","last_unstale":"2024-04-26T20:09:49.578236+0000","last_undegraded":"2024-04-26T20:09:49.578236+0000","last_fullsized":"2024-04-26T20:09:49.578236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:45.076837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:53.185630+0000","last_change":"2024-04-26T20:07:34.775116+0000","last_active":"2024-04-26T20:09:53.185630+0000","last_peered":"2024-04-26T20:09:53.185630+0000","last_clean":"2024-04-26T20:09:53.185630+0000","last_became_active":"2024-04-26T20:07:34.774710+0000","last_became_peered":"2024-04-26T20:07:34.774710+0000","last_unstale":"2024-04-26T20:09:53.185630+0000","last_undegraded":"2024-04-26T20:09:53.185630+0000","last_fullsized":"2024-04-26T20:09:53.185630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:53.203918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:52.645416+0000","last_change":"2024-04-26T20:07:34.773245+0000","last_active":"2024-04-26T20:09:52.645416+0000","last_peered":"2024-04-26T20:09:52.645416+0000","last_clean":"2024-04-26T20:09:52.645416+0000","last_became_active":"2024-04-26T20:07:34.772879+0000","last_became_peered":"2024-04-26T20:07:34.772879+0000","last_unstale":"2024-04-26T20:09:52.645416+0000","last_undegraded":"2024-04-26T20:09:52.645416+0000","last_fullsized":"2024-04-26T20:09:52.645416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:54.450454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T20:09:49.828260+0000","last_change":"2024-04-26T20:07:34.753716+0000","last_active":"2024-04-26T20:09:49.828260+0000","last_peered":"2024-04-26T20:09:49.828260+0000","last_clean":"2024-04-26T20:09:49.828260+0000","last_became_active":"2024-04-26T20:07:34.753378+0000","last_became_peered":"2024-04-26T20:07:34.753378+0000","last_unstale":"2024-04-26T20:09:49.828260+0000","last_undegraded":"2024-04-26T20:09:49.828260+0000","last_fullsized":"2024-04-26T20:09:49.828260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:07:33.703879+0000","last_clean_scrub_stamp":"2024-04-26T20:07:33.703879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:47.165305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65050,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1589,"num_read_kb":864,"num_write":174,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383766,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":554234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645402,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1802240,"data_stored":1202241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808930,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":558566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070568,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1130496,"data_stored":554390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332204,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1150609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495730,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":549646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":610295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738516,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":551034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411466,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40668,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704484,"statfs":{"total":95995035648,"available":95953391616,"internally_reserved":0,"allocated":1200128,"data_stored":612346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182532,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":549840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920894,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":554296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1146197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:13:19.326 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:13:19.376 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:17 smithi131 ceph-mon[27919]: pgmap v508: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:19.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:17 smithi131 ceph-mon[27919]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T20:13:19.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 -- 172.21.15.131:0/2616389574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 msgr2=0x7fda30103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 --2- 172.21.15.131:0/2616389574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda30103de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009900 tx=0x7fda1c02a410 comp rx=0 tx=0).stop 2024-04-26T20:13:19.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 -- 172.21.15.131:0/2616389574 shutdown_connections 2024-04-26T20:13:19.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 --2- 172.21.15.131:0/2616389574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30104320 0x7fda3010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:19.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 --2- 172.21.15.131:0/2616389574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda30103de0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:19.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 --2- 172.21.15.131:0/2616389574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda30102770 0x7fda30102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:19.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 -- 172.21.15.131:0/2616389574 >> 172.21.15.131:0/2616389574 conn(0x7fda300fdce0 msgr2=0x7fda30100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:19.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 -- 172.21.15.131:0/2616389574 shutdown_connections 2024-04-26T20:13:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.601+0000 7fda36200640 1 -- 172.21.15.131:0/2616389574 wait complete. 2024-04-26T20:13:19.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:19 smithi148 ceph-mon[32281]: pgmap v509: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.602+0000 7fda36200640 1 Processor -- start 2024-04-26T20:13:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda36200640 1 -- start start 2024-04-26T20:13:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda36200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30102770 0x7fda3019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda36200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda3019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:20.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda36200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda30104320 0x7fda301a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda36200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda3010e2d0 con 0x7fda30102770 2024-04-26T20:13:20.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda36200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda3010e150 con 0x7fda30103960 2024-04-26T20:13:20.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda36200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda3010e450 con 0x7fda30104320 2024-04-26T20:13:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda34e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30102770 0x7fda3019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:20.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda2fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda3019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:20.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda34e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30102770 0x7fda3019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47438/0 (socket says 172.21.15.131:47438) 2024-04-26T20:13:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda34e00640 1 -- 172.21.15.131:0/3733050683 learned_addr learned my addr 172.21.15.131:0/3733050683 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:20.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.603+0000 7fda2fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda3019b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:51238/0 (socket says 172.21.15.131:51238) 2024-04-26T20:13:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda35800640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda30104320 0x7fda301a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:20.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda34e00640 1 -- 172.21.15.131:0/3733050683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda30104320 msgr2=0x7fda301a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda34e00640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda30104320 0x7fda301a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:20.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda34e00640 1 -- 172.21.15.131:0/3733050683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 msgr2=0x7fda3019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:20.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda34e00640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda3019b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda34e00640 1 -- 172.21.15.131:0/3733050683 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda301a27a0 con 0x7fda30102770 2024-04-26T20:13:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda2fe00640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda3019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:13:21.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda35800640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda30104320 0x7fda301a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:13:21.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda34e00640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30102770 0x7fda3019ab80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9c0 tx=0x7fda2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda2d600640 1 -- 172.21.15.131:0/3733050683 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2400cd80 con 0x7fda30102770 2024-04-26T20:13:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda2d600640 1 -- 172.21.15.131:0/3733050683 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda240035f0 con 0x7fda30102770 2024-04-26T20:13:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda2d600640 1 -- 172.21.15.131:0/3733050683 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24010690 con 0x7fda30102770 2024-04-26T20:13:21.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.604+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda301a2a90 con 0x7fda30102770 2024-04-26T20:13:21.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.605+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301a2fb0 con 0x7fda30102770 2024-04-26T20:13:21.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.606+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda301a3240 con 0x7fda30102770 2024-04-26T20:13:21.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.608+0000 7fda2d600640 1 -- 172.21.15.131:0/3733050683 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fda240037a0 con 0x7fda30102770 2024-04-26T20:13:21.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.609+0000 7fda2d600640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda180a3c40 0x7fda180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:21.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.609+0000 7fda2d600640 1 -- 172.21.15.131:0/3733050683 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fda240d0d90 con 0x7fda30102770 2024-04-26T20:13:21.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.610+0000 7fda2fe00640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda180a3c40 0x7fda180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:21.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.610+0000 7fda2fe00640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda180a3c40 0x7fda180a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda1c031500 tx=0x7fda1c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.614+0000 7fda2d600640 1 -- 172.21.15.131:0/3733050683 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fda2409d220 con 0x7fda30102770 2024-04-26T20:13:21.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:19 smithi131 ceph-mon[27919]: pgmap v509: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:21.976 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:19 smithi157 ceph-mon[32703]: pgmap v509: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:22.135 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:20 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3733050683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:13:22.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.818+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda301a34b0 con 0x7fda30102770 2024-04-26T20:13:22.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.821+0000 7fda2d600640 1 -- 172.21.15.131:0/3733050683 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7fda2409cb20 con 0x7fda30102770 2024-04-26T20:13:22.162 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:13:22.171 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:13:22.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda180a3c40 msgr2=0x7fda180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda180a3c40 0x7fda180a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda1c031500 tx=0x7fda1c005a20 comp rx=0 tx=0).stop 2024-04-26T20:13:22.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30102770 msgr2=0x7fda3019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:22.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30102770 0x7fda3019ab80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9c0 tx=0x7fda2400ee80 comp rx=0 tx=0).stop 2024-04-26T20:13:22.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 shutdown_connections 2024-04-26T20:13:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fda180a3c40 0x7fda180a6100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda30104320 0x7fda301a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:22.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda30103960 0x7fda3019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 --2- 172.21.15.131:0/3733050683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda30102770 0x7fda3019ab80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:22.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.824+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 >> 172.21.15.131:0/3733050683 conn(0x7fda300fdce0 msgr2=0x7fda30074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:22.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.825+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 shutdown_connections 2024-04-26T20:13:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:19.825+0000 7fda36200640 1 -- 172.21.15.131:0/3733050683 wait complete. 2024-04-26T20:13:22.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:13:22.907 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:13:22.918 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:20 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3733050683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:13:22.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:21 smithi131 ceph-mon[27919]: pgmap v510: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:22.975 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:21 smithi157 ceph-mon[32703]: pgmap v510: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:22.986 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:20 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3733050683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:13:23.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:21 smithi148 ceph-mon[32281]: pgmap v510: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:23.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 -- 172.21.15.131:0/3420150694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83200772f0 msgr2=0x7f8320077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:23.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 --2- 172.21.15.131:0/3420150694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83200772f0 0x7f8320077710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8314009860 tx=0x7f831402a340 comp rx=0 tx=0).stop 2024-04-26T20:13:23.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 -- 172.21.15.131:0/3420150694 shutdown_connections 2024-04-26T20:13:23.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 --2- 172.21.15.131:0/3420150694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 0x7f8320079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:23.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 --2- 172.21.15.131:0/3420150694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8320078530 0x7f8320078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:23.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 --2- 172.21.15.131:0/3420150694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83200772f0 0x7f8320077710 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 -- 172.21.15.131:0/3420150694 >> 172.21.15.131:0/3420150694 conn(0x7f83201000e0 msgr2=0x7f8320102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 -- 172.21.15.131:0/3420150694 shutdown_connections 2024-04-26T20:13:23.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.183+0000 7f8328200640 1 -- 172.21.15.131:0/3420150694 wait complete. 2024-04-26T20:13:23.821 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:23 smithi148 ceph-mon[32281]: pgmap v511: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:23.906 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:23 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/12350910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:13:23.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:23 smithi157 ceph-mon[32703]: pgmap v511: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:24.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:23 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/12350910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:13:24.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.184+0000 7f8328200640 1 Processor -- start 2024-04-26T20:13:24.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.184+0000 7f8328200640 1 -- start start 2024-04-26T20:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8328200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83200772f0 0x7f832019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:24.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8328200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8320078530 0x7f832019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:24.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8328200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 0x7f83201a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:24.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8328200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8320114ad0 con 0x7f83200772f0 2024-04-26T20:13:24.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8328200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8320114950 con 0x7f8320078530 2024-04-26T20:13:24.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8328200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8320114c50 con 0x7f8320078ed0 2024-04-26T20:13:24.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8326200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 0x7f83201a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8325800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83200772f0 0x7f832019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8326200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 0x7f83201a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51866/0 (socket says 172.21.15.131:51866) 2024-04-26T20:13:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8326200640 1 -- 172.21.15.131:0/12350910 learned_addr learned my addr 172.21.15.131:0/12350910 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8324e00640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8320078530 0x7f832019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8326200640 1 -- 172.21.15.131:0/12350910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8320078530 msgr2=0x7f832019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.185+0000 7f8326200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8320078530 0x7f832019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f8326200640 1 -- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83200772f0 msgr2=0x7f832019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:25.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f8326200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83200772f0 0x7f832019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f8326200640 1 -- 172.21.15.131:0/12350910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83201a6dd0 con 0x7f8320078ed0 2024-04-26T20:13:25.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f8326200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 0x7f83201a6760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f831800eb10 tx=0x7f831800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f831e000640 1 -- 172.21.15.131:0/12350910 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318004b20 con 0x7f8320078ed0 2024-04-26T20:13:25.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f831e000640 1 -- 172.21.15.131:0/12350910 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8318004cc0 con 0x7f8320078ed0 2024-04-26T20:13:25.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83201a70c0 con 0x7f8320078ed0 2024-04-26T20:13:25.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f831e000640 1 -- 172.21.15.131:0/12350910 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318010670 con 0x7f8320078ed0 2024-04-26T20:13:25.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.186+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201a7610 con 0x7f8320078ed0 2024-04-26T20:13:25.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.187+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83201116d0 con 0x7f8320078ed0 2024-04-26T20:13:25.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.190+0000 7f831e000640 1 -- 172.21.15.131:0/12350910 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8318005020 con 0x7f8320078ed0 2024-04-26T20:13:25.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.191+0000 7f831e000640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83080a3c40 0x7f83080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.191+0000 7f831e000640 1 -- 172.21.15.131:0/12350910 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f83180d0f00 con 0x7f8320078ed0 2024-04-26T20:13:26.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.191+0000 7f8325800640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83080a3c40 0x7f83080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:26.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.192+0000 7f8325800640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83080a3c40 0x7f83080a6100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f8314030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:26.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.195+0000 7f831e000640 1 -- 172.21.15.131:0/12350910 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f831809d330 con 0x7f8320078ed0 2024-04-26T20:13:26.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.402+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8320063530 con 0x7f8320078ed0 2024-04-26T20:13:26.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.403+0000 7f831e000640 1 -- 172.21.15.131:0/12350910 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f831809cc30 con 0x7f8320078ed0 2024-04-26T20:13:26.231 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:13:26.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83080a3c40 msgr2=0x7f83080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:26.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83080a3c40 0x7f83080a6100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f8314030470 comp rx=0 tx=0).stop 2024-04-26T20:13:26.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 msgr2=0x7f83201a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:26.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 0x7f83201a6760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f831800eb10 tx=0x7f831800c2d0 comp rx=0 tx=0).stop 2024-04-26T20:13:26.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 shutdown_connections 2024-04-26T20:13:26.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f83080a3c40 0x7f83080a6100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:26.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8320078ed0 0x7f83201a6760 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:26.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8320078530 0x7f832019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:26.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 --2- 172.21.15.131:0/12350910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83200772f0 0x7f832019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 >> 172.21.15.131:0/12350910 conn(0x7f83201000e0 msgr2=0x7f83201018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:26.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 shutdown_connections 2024-04-26T20:13:26.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:23.406+0000 7f8328200640 1 -- 172.21.15.131:0/12350910 wait complete. 2024-04-26T20:13:27.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:13:27.029 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:23 smithi131 ceph-mon[27919]: pgmap v511: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:27.151 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:23 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/12350910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:13:27.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:25 smithi131 ceph-mon[27919]: pgmap v512: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:27.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:25 smithi148 ceph-mon[32281]: pgmap v512: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:27.223 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:25 smithi157 ceph-mon[32703]: pgmap v512: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:27.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.427+0000 7f4ed0a00640 1 -- 172.21.15.131:0/3814627945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc104a70 msgr2=0x7f4ecc104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:27.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.427+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/3814627945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc104a70 0x7f4ecc104e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009ca0 tx=0x7f4ebc02a8c0 comp rx=0 tx=0).stop 2024-04-26T20:13:27.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.427+0000 7f4ed0a00640 1 -- 172.21.15.131:0/3814627945 shutdown_connections 2024-04-26T20:13:27.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.427+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/3814627945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc106620 0x7f4ecc10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:27.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.427+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/3814627945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ecc105c60 0x7f4ecc1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:27.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.427+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/3814627945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc104a70 0x7f4ecc104e90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:27.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.428+0000 7f4ed0a00640 1 -- 172.21.15.131:0/3814627945 >> 172.21.15.131:0/3814627945 conn(0x7f4ecc0fffe0 msgr2=0x7f4ecc102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:27.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.428+0000 7f4ed0a00640 1 -- 172.21.15.131:0/3814627945 shutdown_connections 2024-04-26T20:13:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.428+0000 7f4ed0a00640 1 -- 172.21.15.131:0/3814627945 wait complete. 2024-04-26T20:13:27.842 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:27 smithi148 ceph-mon[32281]: pgmap v513: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:27.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 Processor -- start 2024-04-26T20:13:27.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 -- start start 2024-04-26T20:13:27.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc104a70 0x7f4ecc19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:28.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ecc105c60 0x7f4ecc19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:28.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc106620 0x7f4ecc1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:28.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc1105e0 con 0x7f4ecc104a70 2024-04-26T20:13:28.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc110460 con 0x7f4ecc106620 2024-04-26T20:13:28.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.429+0000 7f4ed0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc110760 con 0x7f4ecc105c60 2024-04-26T20:13:28.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4eca800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc104a70 0x7f4ecc19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:28.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4ec1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ecc105c60 0x7f4ecc19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:28.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4eca800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc104a70 0x7f4ecc19acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45088/0 (socket says 172.21.15.131:45088) 2024-04-26T20:13:28.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4eca800640 1 -- 172.21.15.131:0/2366146146 learned_addr learned my addr 172.21.15.131:0/2366146146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4ecb200640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc106620 0x7f4ecc1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:28.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4ecb200640 1 -- 172.21.15.131:0/2366146146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ecc105c60 msgr2=0x7f4ecc19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:28.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4ecb200640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ecc105c60 0x7f4ecc19b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:29.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4ecb200640 1 -- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc104a70 msgr2=0x7f4ecc19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:29.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4ecb200640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc104a70 0x7f4ecc19acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:29.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.430+0000 7f4ecb200640 1 -- 172.21.15.131:0/2366146146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc1a28f0 con 0x7f4ecc106620 2024-04-26T20:13:29.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.431+0000 7f4ecb200640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc106620 0x7f4ecc1a2250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e970 tx=0x7f4ec400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.431+0000 7f4ec3e00640 1 -- 172.21.15.131:0/2366146146 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400ccc0 con 0x7f4ecc106620 2024-04-26T20:13:29.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.431+0000 7f4ec3e00640 1 -- 172.21.15.131:0/2366146146 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec4005020 con 0x7f4ecc106620 2024-04-26T20:13:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.431+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc1a2be0 con 0x7f4ecc106620 2024-04-26T20:13:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.431+0000 7f4ec3e00640 1 -- 172.21.15.131:0/2366146146 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4004a00 con 0x7f4ecc106620 2024-04-26T20:13:29.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.431+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc1a30e0 con 0x7f4ecc106620 2024-04-26T20:13:29.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.432+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc10be10 con 0x7f4ecc106620 2024-04-26T20:13:29.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.434+0000 7f4ec3e00640 1 -- 172.21.15.131:0/2366146146 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4ec4004ba0 con 0x7f4ecc106620 2024-04-26T20:13:29.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.435+0000 7f4ec3e00640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4eb00a3c40 0x7f4eb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.436+0000 7f4ec3e00640 1 -- 172.21.15.131:0/2366146146 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4ec40d0a80 con 0x7f4ecc106620 2024-04-26T20:13:30.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.436+0000 7f4eca800640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4eb00a3c40 0x7f4eb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:30.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.437+0000 7f4eca800640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4eb00a3c40 0x7f4eb00a6100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc031b50 tx=0x7f4ebc009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:30.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.438+0000 7f4ec3e00640 1 -- 172.21.15.131:0/2366146146 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4ec409ceb0 con 0x7f4ecc106620 2024-04-26T20:13:30.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.643+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4ecc02dd20 con 0x7f4ecc106620 2024-04-26T20:13:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.644+0000 7f4ec3e00640 1 -- 172.21.15.131:0/2366146146 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f4ec409c7b0 con 0x7f4ecc106620 2024-04-26T20:13:30.721 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:13:30.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4eb00a3c40 msgr2=0x7f4eb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:30.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4eb00a3c40 0x7f4eb00a6100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc031b50 tx=0x7f4ebc009880 comp rx=0 tx=0).stop 2024-04-26T20:13:30.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc106620 msgr2=0x7f4ecc1a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:30.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc106620 0x7f4ecc1a2250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e970 tx=0x7f4ec400ee30 comp rx=0 tx=0).stop 2024-04-26T20:13:31.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 shutdown_connections 2024-04-26T20:13:31.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc106620 0x7f4ecc1a2250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:31.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4eb00a3c40 0x7f4eb00a6100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ecc105c60 0x7f4ecc19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:31.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 --2- 172.21.15.131:0/2366146146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc104a70 0x7f4ecc19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:31.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 >> 172.21.15.131:0/2366146146 conn(0x7f4ecc0fffe0 msgr2=0x7f4ecc108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 shutdown_connections 2024-04-26T20:13:31.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:27.647+0000 7f4ed0a00640 1 -- 172.21.15.131:0/2366146146 wait complete. 2024-04-26T20:13:31.631 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:13:31.737 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:27 smithi131 ceph-mon[27919]: pgmap v513: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:31.849 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:28 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2366146146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:13:31.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:29 smithi131 ceph-mon[27919]: pgmap v514: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:32.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:31 smithi131 ceph-mon[27919]: pgmap v515: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:32.299 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:28 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2366146146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:13:32.306 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:29 smithi148 ceph-mon[32281]: pgmap v514: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:32.408 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:31 smithi148 ceph-mon[32281]: pgmap v515: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:32.514 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:27 smithi157 ceph-mon[32703]: pgmap v513: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:32.520 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:28 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2366146146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:13:32.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:29 smithi157 ceph-mon[32703]: pgmap v514: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:32.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:31 smithi157 ceph-mon[32703]: pgmap v515: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:32.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 -- 172.21.15.131:0/3037590779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240105d30 msgr2=0x7f5240108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 --2- 172.21.15.131:0/3037590779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240105d30 0x7f5240108120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5228009860 tx=0x7f522802a340 comp rx=0 tx=0).stop 2024-04-26T20:13:32.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 -- 172.21.15.131:0/3037590779 shutdown_connections 2024-04-26T20:13:32.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 --2- 172.21.15.131:0/3037590779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 0x7f524010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 --2- 172.21.15.131:0/3037590779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240105d30 0x7f5240108120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 --2- 172.21.15.131:0/3037590779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240103400 0x7f52401057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 -- 172.21.15.131:0/3037590779 >> 172.21.15.131:0/3037590779 conn(0x7f52400fce10 msgr2=0x7f52400ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 -- 172.21.15.131:0/3037590779 shutdown_connections 2024-04-26T20:13:33.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.374+0000 7f5246c00640 1 -- 172.21.15.131:0/3037590779 wait complete. 2024-04-26T20:13:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 Processor -- start 2024-04-26T20:13:33.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 -- start start 2024-04-26T20:13:33.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240103400 0x7f524019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:33.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240105d30 0x7f524019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:33.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 0x7f52401a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:33.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f524010e4a0 con 0x7f5240108660 2024-04-26T20:13:33.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f524010e320 con 0x7f5240103400 2024-04-26T20:13:33.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.376+0000 7f5246c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f524010e620 con 0x7f5240105d30 2024-04-26T20:13:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f5244c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 0x7f52401a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:34.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240103400 0x7f524019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:34.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f5237400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240105d30 0x7f524019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:34.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240103400 0x7f524019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:43130/0 (socket says 172.21.15.131:43130) 2024-04-26T20:13:34.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f5244c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 0x7f52401a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45100/0 (socket says 172.21.15.131:45100) 2024-04-26T20:13:34.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 -- 172.21.15.131:0/1994748465 learned_addr learned my addr 172.21.15.131:0/1994748465 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 -- 172.21.15.131:0/1994748465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240105d30 msgr2=0x7f524019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:34.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240105d30 0x7f524019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 -- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 msgr2=0x7f52401a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:34.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 0x7f52401a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:34.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f523fe00640 1 -- 172.21.15.131:0/1994748465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52401a29d0 con 0x7f5240103400 2024-04-26T20:13:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.377+0000 7f5244c00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 0x7f52401a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:13:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.378+0000 7f523fe00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240103400 0x7f524019adc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9c0 tx=0x7f523000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.378+0000 7f523d600640 1 -- 172.21.15.131:0/1994748465 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523000cd80 con 0x7f5240103400 2024-04-26T20:13:35.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.378+0000 7f523d600640 1 -- 172.21.15.131:0/1994748465 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52300035f0 con 0x7f5240103400 2024-04-26T20:13:35.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.378+0000 7f523d600640 1 -- 172.21.15.131:0/1994748465 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230010690 con 0x7f5240103400 2024-04-26T20:13:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.378+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52401a2cc0 con 0x7f5240103400 2024-04-26T20:13:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.378+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52401a31c0 con 0x7f5240103400 2024-04-26T20:13:35.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.379+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524010a1b0 con 0x7f5240103400 2024-04-26T20:13:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.382+0000 7f523d600640 1 -- 172.21.15.131:0/1994748465 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5230004000 con 0x7f5240103400 2024-04-26T20:13:35.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.383+0000 7f523d600640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52240a3c40 0x7f52240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:35.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.383+0000 7f523d600640 1 -- 172.21.15.131:0/1994748465 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f52300d0e50 con 0x7f5240103400 2024-04-26T20:13:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.383+0000 7f5237400640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52240a3c40 0x7f52240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:35.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.384+0000 7f5237400640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52240a3c40 0x7f52240a6100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f522802a930 tx=0x7f5228030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.385+0000 7f523d600640 1 -- 172.21.15.131:0/1994748465 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f523009d280 con 0x7f5240103400 2024-04-26T20:13:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.587+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f524002dc80 con 0x7f5240103400 2024-04-26T20:13:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.591+0000 7f523d600640 1 -- 172.21.15.131:0/1994748465 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f523009cb80 con 0x7f5240103400 2024-04-26T20:13:36.017 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:13:36.024 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:33 smithi148 ceph-mon[32281]: pgmap v516: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:36.030 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:33 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1994748465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:13:36.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.593+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52240a3c40 msgr2=0x7f52240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:36.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.593+0000 7f5246c00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52240a3c40 0x7f52240a6100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f522802a930 tx=0x7f5228030470 comp rx=0 tx=0).stop 2024-04-26T20:13:36.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.593+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240103400 msgr2=0x7f524019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.593+0000 7f5246c00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240103400 0x7f524019adc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9c0 tx=0x7f523000edc0 comp rx=0 tx=0).stop 2024-04-26T20:13:36.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 shutdown_connections 2024-04-26T20:13:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5240108660 0x7f52401a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52240a3c40 0x7f52240a6100 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:36.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240105d30 0x7f524019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:36.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 --2- 172.21.15.131:0/1994748465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5240103400 0x7f524019adc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 >> 172.21.15.131:0/1994748465 conn(0x7f52400fce10 msgr2=0x7f5240102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 shutdown_connections 2024-04-26T20:13:36.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:32.594+0000 7f5246c00640 1 -- 172.21.15.131:0/1994748465 wait complete. 2024-04-26T20:13:36.743 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:33 smithi157 ceph-mon[32703]: pgmap v516: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:36.749 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:33 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1994748465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:13:36.874 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:35 smithi157 ceph-mon[32703]: pgmap v517: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:37.010 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:13:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:33 smithi131 ceph-mon[27919]: pgmap v516: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:37.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:33 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1994748465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:13:37.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:35 smithi131 ceph-mon[27919]: pgmap v517: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:37.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:36 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:13:37.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:36 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:13:37.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:35 smithi148 ceph-mon[32281]: pgmap v517: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:37.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:36 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:13:37.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 -- 172.21.15.131:0/1416030500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8109fc0 msgr2=0x7f40b810a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 --2- 172.21.15.131:0/1416030500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8109fc0 0x7f40b810a440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009900 tx=0x7f40a402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:13:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 -- 172.21.15.131:0/1416030500 shutdown_connections 2024-04-26T20:13:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 --2- 172.21.15.131:0/1416030500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b810a980 0x7f40b81112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 --2- 172.21.15.131:0/1416030500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8109fc0 0x7f40b810a440 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 --2- 172.21.15.131:0/1416030500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b8108e70 0x7f40b8109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:38.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 -- 172.21.15.131:0/1416030500 >> 172.21.15.131:0/1416030500 conn(0x7f40b8077e00 msgr2=0x7f40b807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 -- 172.21.15.131:0/1416030500 shutdown_connections 2024-04-26T20:13:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.538+0000 7f40b7e00640 1 -- 172.21.15.131:0/1416030500 wait complete. 2024-04-26T20:13:38.251 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:37 smithi157 ceph-mon[32703]: pgmap v518: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:38.399 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:37 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:13:38.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:37 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:13:38.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:37 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:13:38.589 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:37 smithi148 ceph-mon[32281]: pgmap v518: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:38.596 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:37 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:13:38.703 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:37 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:13:38.850 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:37 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:13:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 Processor -- start 2024-04-26T20:13:39.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 -- start start 2024-04-26T20:13:39.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8108e70 0x7f40b819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:39.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b8109fc0 0x7f40b819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b810a980 0x7f40b81a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:39.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81149f0 con 0x7f40b810a980 2024-04-26T20:13:39.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8114870 con 0x7f40b8109fc0 2024-04-26T20:13:39.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.539+0000 7f40b7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8114b70 con 0x7f40b8108e70 2024-04-26T20:13:39.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b810a980 0x7f40b81a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:39.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b810a980 0x7f40b81a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46478/0 (socket says 172.21.15.131:46478) 2024-04-26T20:13:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 -- 172.21.15.131:0/3081399974 learned_addr learned my addr 172.21.15.131:0/3081399974 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:39.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b6a00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8108e70 0x7f40b819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:39.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b6000640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b8109fc0 0x7f40b819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:39.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 -- 172.21.15.131:0/3081399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8108e70 msgr2=0x7f40b819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:39.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8108e70 0x7f40b819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:39.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 -- 172.21.15.131:0/3081399974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b8109fc0 msgr2=0x7f40b819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b8109fc0 0x7f40b819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:39.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.540+0000 7f40b7400640 1 -- 172.21.15.131:0/3081399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b81a6cd0 con 0x7f40b810a980 2024-04-26T20:13:39.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.541+0000 7f40b7400640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b810a980 0x7f40b81a6660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f40b000eaa0 tx=0x7f40b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:39.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.541+0000 7f40af400640 1 -- 172.21.15.131:0/3081399974 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b000cdb0 con 0x7f40b810a980 2024-04-26T20:13:40.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.541+0000 7f40af400640 1 -- 172.21.15.131:0/3081399974 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0005020 con 0x7f40b810a980 2024-04-26T20:13:40.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.541+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b81a6fc0 con 0x7f40b810a980 2024-04-26T20:13:40.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.541+0000 7f40af400640 1 -- 172.21.15.131:0/3081399974 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b00116d0 con 0x7f40b810a980 2024-04-26T20:13:40.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.541+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a7480 con 0x7f40b810a980 2024-04-26T20:13:40.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.542+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b8110180 con 0x7f40b810a980 2024-04-26T20:13:40.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.544+0000 7f40af400640 1 -- 172.21.15.131:0/3081399974 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f40b00051d0 con 0x7f40b810a980 2024-04-26T20:13:40.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.545+0000 7f40af400640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f409c0a3cc0 0x7f409c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:40.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.546+0000 7f40af400640 1 -- 172.21.15.131:0/3081399974 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f40b00d0aa0 con 0x7f40b810a980 2024-04-26T20:13:40.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.546+0000 7f40b6a00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f409c0a3cc0 0x7f409c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.546+0000 7f40b6a00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f409c0a3cc0 0x7f409c0a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009bd0 tx=0x7f40a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.550+0000 7f40af400640 1 -- 172.21.15.131:0/3081399974 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f40b0016030 con 0x7f40b810a980 2024-04-26T20:13:40.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.753+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f40b802dcd0 con 0x7f40b810a980 2024-04-26T20:13:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.753+0000 7f40af400640 1 -- 172.21.15.131:0/3081399974 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f40b009c880 con 0x7f40b810a980 2024-04-26T20:13:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:13:41.033 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:37 smithi131 ceph-mon[27919]: pgmap v518: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:41.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:37 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:13:41.123 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:37 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:13:41.231 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:37 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:13:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f409c0a3cc0 msgr2=0x7f409c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:41.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f409c0a3cc0 0x7f409c0a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009bd0 tx=0x7f40a8005f90 comp rx=0 tx=0).stop 2024-04-26T20:13:41.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b810a980 msgr2=0x7f40b81a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:41.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b810a980 0x7f40b81a6660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f40b000eaa0 tx=0x7f40b000ef60 comp rx=0 tx=0).stop 2024-04-26T20:13:41.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 shutdown_connections 2024-04-26T20:13:41.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40b810a980 0x7f40b81a6660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:41.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b8109fc0 0x7f40b819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:41.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f409c0a3cc0 0x7f409c0a6180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:41.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 --2- 172.21.15.131:0/3081399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b8108e70 0x7f40b819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 >> 172.21.15.131:0/3081399974 conn(0x7f40b8077e00 msgr2=0x7f40b810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 shutdown_connections 2024-04-26T20:13:41.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:37.756+0000 7f40b7e00640 1 -- 172.21.15.131:0/3081399974 wait complete. 2024-04-26T20:13:41.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:13:41.817 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:38 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3081399974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:13:41.896 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:39 smithi131 ceph-mon[27919]: pgmap v519: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:41.935 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:38 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3081399974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:13:41.942 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:39 smithi148 ceph-mon[32281]: pgmap v519: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:41.949 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:38 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3081399974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:13:42.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:39 smithi157 ceph-mon[32703]: pgmap v519: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:42.179 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:41 smithi148 ceph-mon[32281]: pgmap v520: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:42.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:41 smithi157 ceph-mon[32703]: pgmap v520: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:42.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:41 smithi131 ceph-mon[27919]: pgmap v520: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:42.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 -- 172.21.15.131:0/3942566786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 msgr2=0x7f217010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:42.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 --2- 172.21.15.131:0/3942566786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 0x7f217010a400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2158009900 tx=0x7f215802a410 comp rx=0 tx=0).stop 2024-04-26T20:13:42.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 -- 172.21.15.131:0/3942566786 shutdown_connections 2024-04-26T20:13:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 --2- 172.21.15.131:0/3942566786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217010a940 0x7f21701111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:42.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 --2- 172.21.15.131:0/3942566786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 0x7f217010a400 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:42.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 --2- 172.21.15.131:0/3942566786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2170108d90 0x7f21701091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:42.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 -- 172.21.15.131:0/3942566786 >> 172.21.15.131:0/3942566786 conn(0x7f2170077d30 msgr2=0x7f217007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:42.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 -- 172.21.15.131:0/3942566786 shutdown_connections 2024-04-26T20:13:42.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.133+0000 7f2174a00640 1 -- 172.21.15.131:0/3942566786 wait complete. 2024-04-26T20:13:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 Processor -- start 2024-04-26T20:13:42.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 -- start start 2024-04-26T20:13:42.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2170108d90 0x7f217019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:42.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 0x7f217019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217010a940 0x7f21701a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:43.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2170114910 con 0x7f2170109f80 2024-04-26T20:13:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2170114790 con 0x7f2170108d90 2024-04-26T20:13:43.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.134+0000 7f2174a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2170114a90 con 0x7f217010a940 2024-04-26T20:13:43.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f2167e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 0x7f217019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:43.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2170108d90 0x7f217019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:43.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f2167e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 0x7f217019f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46490/0 (socket says 172.21.15.131:46490) 2024-04-26T20:13:43.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f2167e00640 1 -- 172.21.15.131:0/594609844 learned_addr learned my addr 172.21.15.131:0/594609844 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:43.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216fe00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217010a940 0x7f21701a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216f400640 1 -- 172.21.15.131:0/594609844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217010a940 msgr2=0x7f21701a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:43.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216f400640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217010a940 0x7f21701a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:43.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216f400640 1 -- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 msgr2=0x7f217019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:43.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216f400640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 0x7f217019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:43.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216f400640 1 -- 172.21.15.131:0/594609844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21701a6bb0 con 0x7f2170108d90 2024-04-26T20:13:43.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.135+0000 7f216f400640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2170108d90 0x7f217019ef90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f216000e9c0 tx=0x7f216000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:44.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.136+0000 7f216cc00640 1 -- 172.21.15.131:0/594609844 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216000ccc0 con 0x7f2170108d90 2024-04-26T20:13:44.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.136+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21701a6ea0 con 0x7f2170108d90 2024-04-26T20:13:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.136+0000 7f216cc00640 1 -- 172.21.15.131:0/594609844 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21600035f0 con 0x7f2170108d90 2024-04-26T20:13:44.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.136+0000 7f216cc00640 1 -- 172.21.15.131:0/594609844 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216000ce60 con 0x7f2170108d90 2024-04-26T20:13:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.136+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21701a7310 con 0x7f2170108d90 2024-04-26T20:13:44.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.137+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21701100b0 con 0x7f2170108d90 2024-04-26T20:13:44.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.139+0000 7f216cc00640 1 -- 172.21.15.131:0/594609844 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2160004000 con 0x7f2170108d90 2024-04-26T20:13:44.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.140+0000 7f216cc00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f21540a3cc0 0x7f21540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:45.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.140+0000 7f216cc00640 1 -- 172.21.15.131:0/594609844 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f21600d0f00 con 0x7f2170108d90 2024-04-26T20:13:45.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.141+0000 7f2167e00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f21540a3cc0 0x7f21540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:45.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.141+0000 7f2167e00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f21540a3cc0 0x7f21540a6180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f21701a04a0 tx=0x7f2158005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.143+0000 7f216cc00640 1 -- 172.21.15.131:0/594609844 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f216009d3c0 con 0x7f2170108d90 2024-04-26T20:13:45.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.383+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f217002dd30 con 0x7f2170108d90 2024-04-26T20:13:45.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.384+0000 7f216cc00640 1 -- 172.21.15.131:0/594609844 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7f216009ccc0 con 0x7f2170108d90 2024-04-26T20:13:45.833 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:42 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/594609844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:13:45.964 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:43 smithi148 ceph-mon[32281]: pgmap v521: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:46.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:42 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/594609844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:13:46.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:43 smithi157 ceph-mon[32703]: pgmap v521: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:46.035 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:13:46.042 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:12:47.696526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24571,24557]},"id":1}]} 2024-04-26T20:13:46.174 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-04-26T20:13:46.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f21540a3cc0 msgr2=0x7f21540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:46.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f21540a3cc0 0x7f21540a6180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f21701a04a0 tx=0x7f2158005a20 comp rx=0 tx=0).stop 2024-04-26T20:13:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2170108d90 msgr2=0x7f217019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:46.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2170108d90 0x7f217019ef90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f216000e9c0 tx=0x7f216000ee80 comp rx=0 tx=0).stop 2024-04-26T20:13:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 shutdown_connections 2024-04-26T20:13:46.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f21540a3cc0 0x7f21540a6180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:46.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217010a940 0x7f21701a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:46.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170109f80 0x7f217019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:46.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 --2- 172.21.15.131:0/594609844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2170108d90 0x7f217019ef90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:46.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 >> 172.21.15.131:0/594609844 conn(0x7f2170077d30 msgr2=0x7f217010d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 shutdown_connections 2024-04-26T20:13:47.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:42.387+0000 7f2174a00640 1 -- 172.21.15.131:0/594609844 wait complete. 2024-04-26T20:13:47.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:13:47.371 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:42 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/594609844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:13:47.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:43 smithi131 ceph-mon[27919]: pgmap v521: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:47.469 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:45 smithi131 ceph-mon[27919]: pgmap v522: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:47.477 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:45 smithi148 ceph-mon[32281]: pgmap v522: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:47.483 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:45 smithi157 ceph-mon[32703]: pgmap v522: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:13:47.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 -- 172.21.15.131:0/2272486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244102750 msgr2=0x7fe244102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:47.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 --2- 172.21.15.131:0/2272486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244102750 0x7fe244102b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe234009900 tx=0x7fe23402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:13:47.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 -- 172.21.15.131:0/2272486085 shutdown_connections 2024-04-26T20:13:47.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 --2- 172.21.15.131:0/2272486085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244104300 0x7fe24410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 --2- 172.21.15.131:0/2272486085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe244103940 0x7fe244103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:48.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 --2- 172.21.15.131:0/2272486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244102750 0x7fe244102b70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:48.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 -- 172.21.15.131:0/2272486085 >> 172.21.15.131:0/2272486085 conn(0x7fe2440fdcc0 msgr2=0x7fe244100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 -- 172.21.15.131:0/2272486085 shutdown_connections 2024-04-26T20:13:48.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.683+0000 7fe249c00640 1 -- 172.21.15.131:0/2272486085 wait complete. 2024-04-26T20:13:48.262 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:47 smithi148 ceph-mon[32281]: pgmap v523: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:48.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:47 smithi157 ceph-mon[32703]: pgmap v523: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:48.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.684+0000 7fe249c00640 1 Processor -- start 2024-04-26T20:13:48.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.684+0000 7fe249c00640 1 -- start start 2024-04-26T20:13:48.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe249c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244102750 0x7fe24419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:48.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe249c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244103940 0x7fe24419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:48.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe249c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe244104300 0x7fe2441a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:48.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe249c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410e270 con 0x7fe244103940 2024-04-26T20:13:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe249c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410e0f0 con 0x7fe244102750 2024-04-26T20:13:48.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe249c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410e3f0 con 0x7fe244104300 2024-04-26T20:13:49.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244103940 0x7fe24419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:49.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244103940 0x7fe24419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38682/0 (socket says 172.21.15.131:38682) 2024-04-26T20:13:49.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 -- 172.21.15.131:0/1068397792 learned_addr learned my addr 172.21.15.131:0/1068397792 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:49.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243e00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244102750 0x7fe24419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:49.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe249200640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe244104300 0x7fe2441a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:49.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 -- 172.21.15.131:0/1068397792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe244104300 msgr2=0x7fe2441a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:49.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe244104300 0x7fe2441a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:49.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 -- 172.21.15.131:0/1068397792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244102750 msgr2=0x7fe24419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244102750 0x7fe24419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.685+0000 7fe243400640 1 -- 172.21.15.131:0/1068397792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2441a27f0 con 0x7fe244103940 2024-04-26T20:13:49.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.686+0000 7fe243400640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244103940 0x7fe24419b110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe23000e9f0 tx=0x7fe23000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:49.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.686+0000 7fe240c00640 1 -- 172.21.15.131:0/1068397792 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23000cd80 con 0x7fe244103940 2024-04-26T20:13:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.686+0000 7fe240c00640 1 -- 172.21.15.131:0/1068397792 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2300054d0 con 0x7fe244103940 2024-04-26T20:13:49.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.686+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2441a2ae0 con 0x7fe244103940 2024-04-26T20:13:50.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.686+0000 7fe240c00640 1 -- 172.21.15.131:0/1068397792 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230010690 con 0x7fe244103940 2024-04-26T20:13:50.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.686+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2441a3030 con 0x7fe244103940 2024-04-26T20:13:50.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.687+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2441a3260 con 0x7fe244103940 2024-04-26T20:13:50.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.690+0000 7fe240c00640 1 -- 172.21.15.131:0/1068397792 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe230004e90 con 0x7fe244103940 2024-04-26T20:13:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.691+0000 7fe240c00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2280a3cc0 0x7fe2280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:50.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.692+0000 7fe240c00640 1 -- 172.21.15.131:0/1068397792 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe2300d0d40 con 0x7fe244103940 2024-04-26T20:13:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.692+0000 7fe243e00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2280a3cc0 0x7fe2280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:50.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.692+0000 7fe243e00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2280a3cc0 0x7fe2280a6180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe234009230 tx=0x7fe2340314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:50.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.699+0000 7fe240c00640 1 -- 172.21.15.131:0/1068397792 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe23009d170 con 0x7fe244103940 2024-04-26T20:13:50.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.906+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2441a34d0 con 0x7fe244103940 2024-04-26T20:13:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.909+0000 7fe240c00640 1 -- 172.21.15.131:0/1068397792 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7fe23009ca70 con 0x7fe244103940 2024-04-26T20:13:51.028 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:13:51.113 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:13:51.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2280a3cc0 msgr2=0x7fe2280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:51.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2280a3cc0 0x7fe2280a6180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe234009230 tx=0x7fe2340314a0 comp rx=0 tx=0).stop 2024-04-26T20:13:51.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244103940 msgr2=0x7fe24419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:51.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244103940 0x7fe24419b110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe23000e9f0 tx=0x7fe23000edf0 comp rx=0 tx=0).stop 2024-04-26T20:13:51.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 shutdown_connections 2024-04-26T20:13:51.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2280a3cc0 0x7fe2280a6180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:51.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe244104300 0x7fe2441a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:52.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe244103940 0x7fe24419b110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 --2- 172.21.15.131:0/1068397792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244102750 0x7fe24419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:52.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 >> 172.21.15.131:0/1068397792 conn(0x7fe2440fdcc0 msgr2=0x7fe244109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:52.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 shutdown_connections 2024-04-26T20:13:52.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:47.912+0000 7fe249c00640 1 -- 172.21.15.131:0/1068397792 wait complete. 2024-04-26T20:13:52.251 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:47 smithi131 ceph-mon[27919]: pgmap v523: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:13:52.260 DEBUG:tasks.fwd_scrub:Yielding 2024-04-26T20:13:52.341 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T20:13:52.473 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:13:52.506 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T20:13:52.514 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-26T20:13:52.638 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T20:13:52.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:52.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T20:13:52.877 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:48 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1068397792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:13:52.996 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:49 smithi131 ceph-mon[27919]: pgmap v524: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:53.254 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:51 smithi131 ceph-mon[27919]: pgmap v525: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:53.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:48 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1068397792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:13:53.385 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:49 smithi148 ceph-mon[32281]: pgmap v524: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:53.488 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:51 smithi148 ceph-mon[32281]: pgmap v525: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:53.495 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:48 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1068397792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:13:53.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:49 smithi157 ceph-mon[32703]: pgmap v524: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T20:13:53.572 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:51 smithi157 ceph-mon[32703]: pgmap v525: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T20:13:53.666 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T20:13:53.687 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:13:53.735 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T20:13:53.802 INFO:tasks.workunit:timeout=3h 2024-04-26T20:13:53.880 INFO:tasks.workunit:cleanup=True 2024-04-26T20:13:53.987 DEBUG:teuthology.orchestra.run.smithi131:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:13:53.997 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:53 smithi157 ceph-mon[32703]: pgmap v526: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:13:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.529+0000 7fb4f4e00640 1 -- 172.21.15.131:0/1734689377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0105d80 msgr2=0x7fb4f0106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.529+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/1734689377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0105d80 0x7fb4f0106200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009900 tx=0x7fb4d802a400 comp rx=0 tx=0).stop 2024-04-26T20:13:54.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.530+0000 7fb4f4e00640 1 -- 172.21.15.131:0/1734689377 shutdown_connections 2024-04-26T20:13:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.530+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/1734689377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4f0106740 0x7fb4f010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.530+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/1734689377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0105d80 0x7fb4f0106200 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.530+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/1734689377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 0x7fb4f0104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.530+0000 7fb4f4e00640 1 -- 172.21.15.131:0/1734689377 >> 172.21.15.131:0/1734689377 conn(0x7fb4f01000e0 msgr2=0x7fb4f0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:54.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.530+0000 7fb4f4e00640 1 -- 172.21.15.131:0/1734689377 shutdown_connections 2024-04-26T20:13:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.530+0000 7fb4f4e00640 1 -- 172.21.15.131:0/1734689377 wait complete. 2024-04-26T20:13:54.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 Processor -- start 2024-04-26T20:13:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 -- start start 2024-04-26T20:13:54.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 0x7fb4f019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4f0105d80 0x7fb4f019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0106740 0x7fb4f01a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:55.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0110710 con 0x7fb4f0105d80 2024-04-26T20:13:55.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0110590 con 0x7fb4f0106740 2024-04-26T20:13:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.531+0000 7fb4f4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0110890 con 0x7fb4f0104b90 2024-04-26T20:13:55.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4f0105d80 0x7fb4f019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:55.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 0x7fb4f019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:55.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0106740 0x7fb4f01a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4f0105d80 0x7fb4f019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38708/0 (socket says 172.21.15.131:38708) 2024-04-26T20:13:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4e7e00640 1 -- 172.21.15.131:0/2334298509 learned_addr learned my addr 172.21.15.131:0/2334298509 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 0x7fb4f019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40674/0 (socket says 172.21.15.131:40674) 2024-04-26T20:13:55.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 -- 172.21.15.131:0/2334298509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0106740 msgr2=0x7fb4f01a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0106740 0x7fb4f01a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:55.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 -- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4f0105d80 msgr2=0x7fb4f019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4f0105d80 0x7fb4f019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 -- 172.21.15.131:0/2334298509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f01a2a30 con 0x7fb4f0104b90 2024-04-26T20:13:56.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ee800640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0106740 0x7fb4f01a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:13:56.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.532+0000 7fb4ede00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 0x7fb4f019ae10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e9c0 tx=0x7fb4e000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:56.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.533+0000 7fb4e6a00640 1 -- 172.21.15.131:0/2334298509 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000ccc0 con 0x7fb4f0104b90 2024-04-26T20:13:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.533+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f01a2d20 con 0x7fb4f0104b90 2024-04-26T20:13:56.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.533+0000 7fb4e6a00640 1 -- 172.21.15.131:0/2334298509 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e00035f0 con 0x7fb4f0104b90 2024-04-26T20:13:56.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.533+0000 7fb4e6a00640 1 -- 172.21.15.131:0/2334298509 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000ce60 con 0x7fb4f0104b90 2024-04-26T20:13:56.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.533+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f01a31c0 con 0x7fb4f0104b90 2024-04-26T20:13:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.536+0000 7fb4f7d97740 1 Processor -- start 2024-04-26T20:13:56.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.536+0000 7fb4f7d97740 1 -- start start 2024-04-26T20:13:56.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341736c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:56.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4e6a00640 1 -- 172.21.15.131:0/2334298509 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb4e0004000 con 0x7fb4f0104b90 2024-04-26T20:13:57.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341737250 0x55f34173bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:57.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 0x55f34173e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:57.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f34169b980 con 0x55f341737250 2024-04-26T20:13:57.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f341638220 con 0x55f34173c3c0 2024-04-26T20:13:57.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f341697940 con 0x55f341736860 2024-04-26T20:13:57.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341737250 0x55f34173bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:57.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341736c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:57.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4e7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 0x55f34173e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:57.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341736c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40680/0 (socket says 172.21.15.131:40680) 2024-04-26T20:13:58.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 -- 172.21.15.131:0/1103574631 learned_addr learned my addr 172.21.15.131:0/1103574631 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:13:58.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 -- 172.21.15.131:0/1103574631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 msgr2=0x55f34173e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:13:58.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 --2- 172.21.15.131:0/1103574631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 0x55f34173e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:58.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 -- 172.21.15.131:0/1103574631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341737250 msgr2=0x55f34173bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:13:58.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 --2- 172.21.15.131:0/1103574631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341737250 0x55f34173bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:58.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.537+0000 7fb4ee800640 1 -- 172.21.15.131:0/1103574631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3414757e0 con 0x55f341736860 2024-04-26T20:13:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.538+0000 7fb4ee800640 1 --2- 172.21.15.131:0/1103574631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341736c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80117e0 tx=0x7fb4e8011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21b1876195cdc1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.538+0000 7fb4df400640 1 -- 172.21.15.131:0/1103574631 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8011310 con 0x55f341736860 2024-04-26T20:13:58.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.538+0000 7fb4df400640 1 -- 172.21.15.131:0/1103574631 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e8003e20 con 0x55f341736860 2024-04-26T20:13:58.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.538+0000 7fb4df400640 1 -- 172.21.15.131:0/1103574631 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80044d0 con 0x55f341736860 2024-04-26T20:13:58.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.538+0000 7fb4e6a00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb4d40a3c40 0x7fb4d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:58.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.538+0000 7fb4e6a00640 1 -- 172.21.15.131:0/2334298509 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb4e00d0d50 con 0x7fb4f0104b90 2024-04-26T20:13:58.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4ede00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb4d40a3c40 0x7fb4d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:13:58.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 -- 172.21.15.131:0/1103574631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 msgr2=0x55f341736c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:13:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/1103574631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341736c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80117e0 tx=0x7fb4e8011ca0 comp rx=0 tx=0).stop 2024-04-26T20:13:59.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 -- 172.21.15.131:0/1103574631 shutdown_connections 2024-04-26T20:13:59.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/1103574631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 0x55f34173e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/1103574631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341737250 0x55f34173bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:59.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/1103574631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341736c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:13:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 -- 172.21.15.131:0/1103574631 >> 172.21.15.131:0/1103574631 conn(0x55f341731ed0 msgr2=0x55f3417322d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:13:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 -- 172.21.15.131:0/1103574631 shutdown_connections 2024-04-26T20:13:59.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 -- 172.21.15.131:0/1103574631 wait complete. 2024-04-26T20:13:59.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4ede00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb4d40a3c40 0x7fb4d40a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f341737650 tx=0x7fb4e00d4800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:13:59.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.539+0000 7fb4f7d97740 1 -- start start 2024-04-26T20:13:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 1 Processor -- start 2024-04-26T20:13:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341736860 0x55f341740c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f341748210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:13:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f34173f370 con 0x55f34173c3c0 2024-04-26T20:13:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f341623170 con 0x55f341736860 2024-04-26T20:13:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f34173f4f0 con 0x55f341737250 2024-04-26T20:13:59.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4f7d97740 10 client.0 _ll_register_callbacks cb 0x55f341692770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:14:00.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4e7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:00.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f341748210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:00.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4e7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40696/0 (socket says 172.21.15.131:40696) 2024-04-26T20:14:00.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.540+0000 7fb4e7e00640 1 -- 172.21.15.131:0/3962092941 learned_addr learned my addr 172.21.15.131:0/3962092941 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4ee800640 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341736860 0x55f341740c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:00.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4e7e00640 1 -- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341736860 msgr2=0x55f341740c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341736860 0x55f341740c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:00.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4e7e00640 1 -- 172.21.15.131:0/3962092941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 msgr2=0x55f341748210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:00.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f341748210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4e7e00640 1 -- 172.21.15.131:0/3962092941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3414d5f70 con 0x55f341737250 2024-04-26T20:14:00.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4ee800640 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341736860 0x55f341740c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:14:00.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417411d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4d802a9f0 tx=0x7fb4d80043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=134c9e44492322c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:00.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:14:01.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.541+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3962092941 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8035040 con 0x55f341737250 2024-04-26T20:14:01.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e8003e20 con 0x55f341737250 2024-04-26T20:14:01.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3962092941 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb4d8003a00 con 0x55f341737250 2024-04-26T20:14:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3962092941 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4d804e050 con 0x55f341737250 2024-04-26T20:14:01.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4f7d97740 10 client.25510 fetch_fsmap learned FSMap version 27 2024-04-26T20:14:01.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f34136d170 con 0x55f341737250 2024-04-26T20:14:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3962092941 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e8003e20 con 0x55f341737250 2024-04-26T20:14:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3962092941 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb4d803c440 con 0x55f341737250 2024-04-26T20:14:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4f7d97740 10 client.25510 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:14:01.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.542+0000 7fb4f7d97740 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:14:01.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.543+0000 7fb4f7d97740 20 client.25510 populate_metadata read hostname 'smithi131' 2024-04-26T20:14:01.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.543+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f34174c870 0x55f34174ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:01.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.543+0000 7fb4f7d97740 4 client.25510 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:14:01.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.543+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3416a9590 con 0x55f34174c870 2024-04-26T20:14:02.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.543+0000 7fb4ee800640 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f34174c870 0x55f34174ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:02.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.543+0000 7fb4ee800640 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f34174c870 0x55f34174ecb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace70bd1b5d85d9a server_cookie=19b99359660ed44 in_seq=0 out_seq=0 2024-04-26T20:14:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.543+0000 7fb4d3e00640 10 client.25510 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:02.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3962092941 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3416a9590 con 0x55f34174c870 2024-04-26T20:14:02.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4d3e00640 10 client.25510 handle_command_reply: tid=0 2024-04-26T20:14:02.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 client.25510 shutdown 2024-04-26T20:14:02.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 msgr2=0x55f3417411d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417411d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4d802a9f0 tx=0x7fb4d80043b0 comp rx=0 tx=0).stop 2024-04-26T20:14:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 shutdown_connections 2024-04-26T20:14:02.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f34174c870 0x55f34174ecb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:02.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f341748210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417411d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3962092941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341736860 0x55f341740c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:02.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4d3e00640 0 client.25510 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:02.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.544+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 >> 172.21.15.131:0/3962092941 conn(0x55f341731ed0 msgr2=0x55f341730ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:02.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.545+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 shutdown_connections 2024-04-26T20:14:03.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.545+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3962092941 wait complete. 2024-04-26T20:14:03.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.545+0000 7fb4f7d97740 20 client.25510 trim_cache size 0 max 16384 2024-04-26T20:14:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 Processor -- start 2024-04-26T20:14:03.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 -- start start 2024-04-26T20:14:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341735670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:03.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341737250 0x55f3417481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:03.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f3417535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01b00a0 con 0x55f34173c3c0 2024-04-26T20:14:03.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01ad970 con 0x55f341737250 2024-04-26T20:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.564+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01ab7c0 con 0x55f341736860 2024-04-26T20:14:04.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f3417535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341737250 0x55f3417481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:04.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f3417535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38732/0 (socket says 172.21.15.131:38732) 2024-04-26T20:14:04.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341737250 0x55f3417481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:53228/0 (socket says 172.21.15.131:53228) 2024-04-26T20:14:04.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4ee800640 1 -- 172.21.15.131:0/2195911514 learned_addr learned my addr 172.21.15.131:0/2195911514 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:04.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4ee800640 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341735670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 -- 172.21.15.131:0/2195911514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 msgr2=0x55f341735670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:04.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341735670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 -- 172.21.15.131:0/2195911514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 msgr2=0x55f3417535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:05.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f3417535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:05.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 -- 172.21.15.131:0/2195911514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f341731a70 con 0x55f341737250 2024-04-26T20:14:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4ee800640 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f3417535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:05.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.565+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341737250 0x55f3417481d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80096c0 tx=0x7fb4d803ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c2a4b36c8cc8a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:05.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4dea00640 1 -- 172.21.15.131:0/2195911514 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8035070 con 0x55f341737250 2024-04-26T20:14:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4dea00640 1 -- 172.21.15.131:0/2195911514 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d8005850 con 0x55f341737250 2024-04-26T20:14:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4dea00640 1 -- 172.21.15.131:0/2195911514 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8003a20 con 0x55f341737250 2024-04-26T20:14:05.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 -- 172.21.15.131:0/2195911514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341737250 msgr2=0x55f3417481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:05.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341737250 0x55f3417481d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80096c0 tx=0x7fb4d803ef90 comp rx=0 tx=0).stop 2024-04-26T20:14:05.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 -- 172.21.15.131:0/2195911514 shutdown_connections 2024-04-26T20:14:05.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f34173c3c0 0x55f3417535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:05.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f341737250 0x55f3417481d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/2195911514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341736860 0x55f341735670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:05.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 -- 172.21.15.131:0/2195911514 >> 172.21.15.131:0/2195911514 conn(0x55f341731ed0 msgr2=0x55f341734f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.566+0000 7fb4f7d97740 1 -- 172.21.15.131:0/2195911514 shutdown_connections 2024-04-26T20:14:06.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.567+0000 7fb4f7d97740 1 -- 172.21.15.131:0/2195911514 wait complete. 2024-04-26T20:14:06.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.567+0000 7fb4f7d97740 1 -- start start 2024-04-26T20:14:06.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.567+0000 7fb4f7d97740 1 Processor -- start 2024-04-26T20:14:06.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.567+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341736860 0x55f341747fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:06.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.567+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:06.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4f7d97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 0x55f341746b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:06.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01abe30 con 0x55f341736860 2024-04-26T20:14:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01ad2a0 con 0x55f34173c3c0 2024-04-26T20:14:06.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4f7d97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01ac500 con 0x55f341737250 2024-04-26T20:14:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4f7d97740 10 client.0 _ll_register_callbacks cb 0x7fb4d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:14:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341736860 0x55f341747fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:06.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341736860 0x55f341747fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38744/0 (socket says 172.21.15.131:38744) 2024-04-26T20:14:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 -- 172.21.15.131:0/3728234923 learned_addr learned my addr 172.21.15.131:0/3728234923 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:06.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:06.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 -- 172.21.15.131:0/3728234923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 msgr2=0x55f3417465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:06.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:06.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 -- 172.21.15.131:0/3728234923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 msgr2=0x55f341746b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:06.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 0x55f341746b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:07.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 -- 172.21.15.131:0/3728234923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f341747240 con 0x55f341736860 2024-04-26T20:14:07.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4ee800640 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341736860 0x55f341747fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800a010 tx=0x7fb4e80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65245b4d573c7b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:07.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.568+0000 7fb4d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:14:07.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3728234923 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8025040 con 0x55f341736860 2024-04-26T20:14:07.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3728234923 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb4e800f9a0 con 0x55f341736860 2024-04-26T20:14:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d8005850 con 0x55f341736860 2024-04-26T20:14:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3728234923 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4e800fbc0 con 0x55f341736860 2024-04-26T20:14:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4f7d97740 10 client.15396 fetch_fsmap learned FSMap version 27 2024-04-26T20:14:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3417bc620 con 0x55f341736860 2024-04-26T20:14:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3728234923 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d8005850 con 0x55f341736860 2024-04-26T20:14:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.569+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3728234923 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb4e8018020 con 0x55f341736860 2024-04-26T20:14:07.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4f7d97740 10 client.15396 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:14:07.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4f7d97740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:14:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4f7d97740 20 client.15396 populate_metadata read hostname 'smithi131' 2024-04-26T20:14:07.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3417bcec0 0x55f3417bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4f7d97740 4 client.15396 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T20:14:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f3416a3400 con 0x55f3417bcec0 2024-04-26T20:14:08.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3417bcec0 0x55f3417bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:08.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4d3e00640 10 client.15396 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.570+0000 7fb4e7e00640 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3417bcec0 0x55f3417bf2e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c858103579c36df5 server_cookie=b7bab0a9a460d8cb in_seq=0 out_seq=0 2024-04-26T20:14:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4d3e00640 1 -- 172.21.15.131:0/3728234923 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f3416a3400 con 0x55f3417bcec0 2024-04-26T20:14:08.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4d3e00640 10 client.15396 handle_command_reply: tid=0 2024-04-26T20:14:08.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 client.15396 shutdown 2024-04-26T20:14:08.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341736860 msgr2=0x55f341747fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:08.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341736860 0x55f341747fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800a010 tx=0x7fb4e80096c0 comp rx=0 tx=0).stop 2024-04-26T20:14:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 shutdown_connections 2024-04-26T20:14:08.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3417bcec0 0x55f3417bf2e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f34173c3c0 0x55f341746b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f341737250 0x55f3417465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 --2- 172.21.15.131:0/3728234923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f341736860 0x55f341747fb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:08.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 >> 172.21.15.131:0/3728234923 conn(0x55f341731ed0 msgr2=0x55f3417357f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:08.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4d3e00640 0 client.15396 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:08.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 shutdown_connections 2024-04-26T20:14:09.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 1 -- 172.21.15.131:0/3728234923 wait complete. 2024-04-26T20:14:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.572+0000 7fb4f7d97740 20 client.15396 trim_cache size 0 max 16384 2024-04-26T20:14:09.204 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:14:09.293 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-04-26T20:14:09.437 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "b780555d-d008-4808-8633-59bc8d1efeaa", 2024-04-26T20:14:09.550 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-04-26T20:14:09.556 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:14:09.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb4d40a3c40 msgr2=0x7fb4d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb4d40a3c40 0x7fb4d40a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f341737650 tx=0x7fb4e00d4800 comp rx=0 tx=0).stop 2024-04-26T20:14:09.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 msgr2=0x7fb4f019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:09.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 0x7fb4f019ae10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e9c0 tx=0x7fb4e000ee80 comp rx=0 tx=0).stop 2024-04-26T20:14:09.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 shutdown_connections 2024-04-26T20:14:09.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f0106740 0x7fb4f01a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:09.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4f0105d80 0x7fb4f019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:09.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb4d40a3c40 0x7fb4d40a6100 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:09.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 --2- 172.21.15.131:0/2334298509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0104b90 0x7fb4f019ae10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:09.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.573+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 >> 172.21.15.131:0/2334298509 conn(0x7fb4f01000e0 msgr2=0x7fb4f01090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:10.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.574+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 shutdown_connections 2024-04-26T20:14:10.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:13:53.574+0000 7fb4f4e00640 1 -- 172.21.15.131:0/2334298509 wait complete. 2024-04-26T20:14:10.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T20:14:10.344 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T20:14:10.439 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:53 smithi131 ceph-mon[27919]: pgmap v526: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:10.445 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:54 smithi131 ceph-mon[27919]: scrub queued for path: / 2024-04-26T20:14:10.542 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:54 smithi131 ceph-mon[27919]: scrub summary: idle+waiting paths [/] 2024-04-26T20:14:10.609 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:54 smithi131 ceph-mon[27919]: scrub summary: active paths [/] 2024-04-26T20:14:10.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:54 smithi131 ceph-mon[27919]: scrub summary: idle+waiting paths [/] 2024-04-26T20:14:10.731 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:55 smithi131 ceph-mon[27919]: pgmap v527: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T20:14:10.797 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:56 smithi131 ceph-mon[27919]: pgmap v528: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:10.824 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:13:58 smithi131 ceph-mon[27919]: pgmap v529: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:14:10.866 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:00 smithi131 ceph-mon[27919]: pgmap v530: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:14:10.900 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:02 smithi131 ceph-mon[27919]: pgmap v531: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:10.928 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:04 smithi131 ceph-mon[27919]: pgmap v532: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-26T20:14:10.964 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:54 smithi157 ceph-mon[32703]: scrub queued for path: / 2024-04-26T20:14:11.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:54 smithi157 ceph-mon[32703]: scrub summary: idle+waiting paths [/] 2024-04-26T20:14:11.209 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:54 smithi157 ceph-mon[32703]: scrub summary: active paths [/] 2024-04-26T20:14:11.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:54 smithi157 ceph-mon[32703]: scrub summary: idle+waiting paths [/] 2024-04-26T20:14:11.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:55 smithi157 ceph-mon[32703]: pgmap v527: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T20:14:11.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:56 smithi157 ceph-mon[32703]: pgmap v528: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:11.473 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:13:58 smithi157 ceph-mon[32703]: pgmap v529: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:14:11.480 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:00 smithi157 ceph-mon[32703]: pgmap v530: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:14:11.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:02 smithi157 ceph-mon[32703]: pgmap v531: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:11.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:04 smithi157 ceph-mon[32703]: pgmap v532: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-26T20:14:11.612 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:06 smithi157 ceph-mon[32703]: pgmap v533: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T20:14:11.623 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:07 smithi157 ceph-mon[32703]: scrub summary: idle 2024-04-26T20:14:11.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:09 smithi157 ceph-mon[32703]: pgmap v534: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T20:14:11.746 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:53 smithi148 ceph-mon[32281]: pgmap v526: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:11.752 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:54 smithi148 ceph-mon[32281]: scrub queued for path: / 2024-04-26T20:14:11.759 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:54 smithi148 ceph-mon[32281]: scrub summary: idle+waiting paths [/] 2024-04-26T20:14:11.881 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:54 smithi148 ceph-mon[32281]: scrub summary: active paths [/] 2024-04-26T20:14:11.889 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:54 smithi148 ceph-mon[32281]: scrub summary: idle+waiting paths [/] 2024-04-26T20:14:12.019 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:55 smithi148 ceph-mon[32281]: pgmap v527: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T20:14:12.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:56 smithi148 ceph-mon[32281]: pgmap v528: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:12.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:13:58 smithi148 ceph-mon[32281]: pgmap v529: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:14:12.354 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:00 smithi148 ceph-mon[32281]: pgmap v530: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:14:12.470 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:02 smithi148 ceph-mon[32281]: pgmap v531: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:14:12.586 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:04 smithi148 ceph-mon[32281]: pgmap v532: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-26T20:14:12.701 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:06 smithi148 ceph-mon[32281]: pgmap v533: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T20:14:12.707 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:07 smithi148 ceph-mon[32281]: scrub summary: idle 2024-04-26T20:14:12.714 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:09 smithi148 ceph-mon[32281]: pgmap v534: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T20:14:12.767 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:11 smithi157 ceph-mon[32703]: pgmap v535: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-26T20:14:12.810 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:11 smithi148 ceph-mon[32281]: pgmap v535: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-26T20:14:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 -- 172.21.15.131:0/874856334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 msgr2=0x7f2590104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 --2- 172.21.15.131:0/874856334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 0x7f2590104e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2580009be0 tx=0x7f258002a6d0 comp rx=0 tx=0).stop 2024-04-26T20:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 -- 172.21.15.131:0/874856334 shutdown_connections 2024-04-26T20:14:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 --2- 172.21.15.131:0/874856334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25901065a0 0x7f259010ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 --2- 172.21.15.131:0/874856334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2590105be0 0x7f2590106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 --2- 172.21.15.131:0/874856334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 0x7f2590104e10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 -- 172.21.15.131:0/874856334 >> 172.21.15.131:0/874856334 conn(0x7f25900fff60 msgr2=0x7f25901023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 -- 172.21.15.131:0/874856334 shutdown_connections 2024-04-26T20:14:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.578+0000 7f2595c00640 1 -- 172.21.15.131:0/874856334 wait complete. 2024-04-26T20:14:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.579+0000 7f2595c00640 1 Processor -- start 2024-04-26T20:14:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2595c00640 1 -- start start 2024-04-26T20:14:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2595c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 0x7f259019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2595c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2590105be0 0x7f259019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2595c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25901065a0 0x7f25901a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2595c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25901104f0 con 0x7f2590105be0 2024-04-26T20:14:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2595c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2590110370 con 0x7f25901049f0 2024-04-26T20:14:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2595c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2590110670 con 0x7f25901065a0 2024-04-26T20:14:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2587400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2590105be0 0x7f259019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f258fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 0x7f259019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2587400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2590105be0 0x7f259019b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40406/0 (socket says 172.21.15.131:40406) 2024-04-26T20:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.580+0000 7f2587400640 1 -- 172.21.15.131:0/3468669190 learned_addr learned my addr 172.21.15.131:0/3468669190 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f2595200640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25901065a0 0x7f25901a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f258fe00640 1 -- 172.21.15.131:0/3468669190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25901065a0 msgr2=0x7f25901a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f258fe00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25901065a0 0x7f25901a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f258fe00640 1 -- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2590105be0 msgr2=0x7f259019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f258fe00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2590105be0 0x7f259019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f258fe00640 1 -- 172.21.15.131:0/3468669190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25901a2830 con 0x7f25901049f0 2024-04-26T20:14:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f2587400640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2590105be0 0x7f259019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.581+0000 7f258fe00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 0x7f259019ac70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2580034040 tx=0x7f2580030d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.582+0000 7f258d600640 1 -- 172.21.15.131:0/3468669190 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580041020 con 0x7f25901049f0 2024-04-26T20:14:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.582+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25901a2ac0 con 0x7f25901049f0 2024-04-26T20:14:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.582+0000 7f258d600640 1 -- 172.21.15.131:0/3468669190 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25800318f0 con 0x7f25901049f0 2024-04-26T20:14:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.582+0000 7f258d600640 1 -- 172.21.15.131:0/3468669190 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580036040 con 0x7f25901049f0 2024-04-26T20:14:14.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.582+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25901a2fd0 con 0x7f25901049f0 2024-04-26T20:14:14.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 Processor -- start 2024-04-26T20:14:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 -- start start 2024-04-26T20:14:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f258d600640 1 -- 172.21.15.131:0/3468669190 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2580044020 con 0x7f25901049f0 2024-04-26T20:14:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480dde0 0x556ed480e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480e7d0 0x556ed48132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed4815dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556ed4772380 con 0x556ed480dde0 2024-04-26T20:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556ed470ec30 con 0x556ed480e7d0 2024-04-26T20:14:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.585+0000 7f259a1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556ed476e340 con 0x556ed4813940 2024-04-26T20:14:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480dde0 0x556ed480e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480dde0 0x556ed480e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40414/0 (socket says 172.21.15.131:40414) 2024-04-26T20:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 -- 172.21.15.131:0/947060474 learned_addr learned my addr 172.21.15.131:0/947060474 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f258fe00640 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480e7d0 0x556ed48132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2587400640 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed4815dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 -- 172.21.15.131:0/947060474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 msgr2=0x556ed4815dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed4815dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 -- 172.21.15.131:0/947060474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480e7d0 msgr2=0x556ed48132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480e7d0 0x556ed48132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f2595200640 1 -- 172.21.15.131:0/947060474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed4730940 con 0x556ed480dde0 2024-04-26T20:14:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.586+0000 7f258fe00640 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480e7d0 0x556ed48132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.587+0000 7f2595200640 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480dde0 0x556ed480e200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2588009ca0 tx=0x7f2588011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba63bbedea01e92d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.587+0000 7f2585600640 1 -- 172.21.15.131:0/947060474 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258801e070 con 0x556ed480dde0 2024-04-26T20:14:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.587+0000 7f258d600640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25740a3c30 0x7f25740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.587+0000 7f2585600640 1 -- 172.21.15.131:0/947060474 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2588009e50 con 0x556ed480dde0 2024-04-26T20:14:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.587+0000 7f2585600640 1 -- 172.21.15.131:0/947060474 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258800e9e0 con 0x556ed480dde0 2024-04-26T20:14:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.587+0000 7f258d600640 1 -- 172.21.15.131:0/3468669190 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f25800f1660 con 0x7f25901049f0 2024-04-26T20:14:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.587+0000 7f258fe00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25740a3c30 0x7f25740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 -- 172.21.15.131:0/947060474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480dde0 msgr2=0x556ed480e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480dde0 0x556ed480e200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2588009ca0 tx=0x7f2588011810 comp rx=0 tx=0).stop 2024-04-26T20:14:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 -- 172.21.15.131:0/947060474 shutdown_connections 2024-04-26T20:14:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed4815dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480e7d0 0x556ed48132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 --2- 172.21.15.131:0/947060474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480dde0 0x556ed480e200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 -- 172.21.15.131:0/947060474 >> 172.21.15.131:0/947060474 conn(0x556ed4809450 msgr2=0x556ed4809850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f258fe00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25740a3c30 0x7f25740a60f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x556ed480ebd0 tx=0x7f25800f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 -- 172.21.15.131:0/947060474 shutdown_connections 2024-04-26T20:14:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 -- 172.21.15.131:0/947060474 wait complete. 2024-04-26T20:14:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.588+0000 7f259a1c8740 1 -- start start 2024-04-26T20:14:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 1 Processor -- start 2024-04-26T20:14:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed4818000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480e7d0 0x556ed4818540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed4813940 0x556ed481f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556ed48168f0 con 0x556ed4813940 2024-04-26T20:14:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556ed46f9c60 con 0x556ed480dde0 2024-04-26T20:14:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556ed4816a70 con 0x556ed480e7d0 2024-04-26T20:14:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f259a1c8740 10 client.0 _ll_register_callbacks cb 0x556ed4725e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:14:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f2587400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480e7d0 0x556ed4818540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f2595200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed4818000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f2595200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed4818000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:51722/0 (socket says 172.21.15.131:51722) 2024-04-26T20:14:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f2595200640 1 -- 172.21.15.131:0/4158598202 learned_addr learned my addr 172.21.15.131:0/4158598202 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.589+0000 7f2587400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480e7d0 0x556ed4818540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56592/0 (socket says 172.21.15.131:56592) 2024-04-26T20:14:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f2595200640 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed4813940 0x556ed481f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f2587400640 1 -- 172.21.15.131:0/4158598202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 msgr2=0x556ed4818000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f2587400640 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed4818000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f2587400640 1 -- 172.21.15.131:0/4158598202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed4813940 msgr2=0x556ed481f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f2587400640 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed4813940 0x556ed481f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f2587400640 1 -- 172.21.15.131:0/4158598202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed4708510 con 0x556ed480e7d0 2024-04-26T20:14:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f2587400640 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480e7d0 0x556ed4818540 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2578005c80 tx=0x7f257800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcff2ccf35b4f821 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.590+0000 7f257e000640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:14:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f257e000640 1 -- 172.21.15.131:0/4158598202 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578004050 con 0x556ed480e7d0 2024-04-26T20:14:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f257e000640 1 -- 172.21.15.131:0/4158598202 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f25780041f0 con 0x556ed480e7d0 2024-04-26T20:14:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2588009e50 con 0x556ed480e7d0 2024-04-26T20:14:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f257e000640 1 -- 172.21.15.131:0/4158598202 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2578025050 con 0x556ed480e7d0 2024-04-26T20:14:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f259a1c8740 10 client.25528 fetch_fsmap learned FSMap version 27 2024-04-26T20:14:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ed4484210 con 0x556ed480e7d0 2024-04-26T20:14:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f257e000640 1 -- 172.21.15.131:0/4158598202 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2588009e50 con 0x556ed480e7d0 2024-04-26T20:14:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f257e000640 1 -- 172.21.15.131:0/4158598202 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2578004ac0 con 0x556ed480e7d0 2024-04-26T20:14:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.591+0000 7f259a1c8740 10 client.25528 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:14:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f259a1c8740 10 client.25528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:14:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f259a1c8740 20 client.25528 populate_metadata read hostname 'smithi131' 2024-04-26T20:14:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed4823be0 0x556ed4826020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f259a1c8740 4 client.25528 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:14:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ed477ff90 con 0x556ed4823be0 2024-04-26T20:14:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f2595200640 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed4823be0 0x556ed4826020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f2595200640 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed4823be0 0x556ed4826020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88eebec240a2c214 server_cookie=241f0b2ad3b0a222 in_seq=0 out_seq=0 2024-04-26T20:14:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.592+0000 7f257e000640 10 client.25528 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.593+0000 7f257e000640 1 -- 172.21.15.131:0/4158598202 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ed477ff90 con 0x556ed4823be0 2024-04-26T20:14:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.593+0000 7f257e000640 10 client.25528 handle_command_reply: tid=0 2024-04-26T20:14:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.593+0000 7f259a1c8740 1 client.25528 shutdown 2024-04-26T20:14:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480e7d0 msgr2=0x556ed4818540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480e7d0 0x556ed4818540 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2578005c80 tx=0x7f257800f790 comp rx=0 tx=0).stop 2024-04-26T20:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 shutdown_connections 2024-04-26T20:14:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed4823be0 0x556ed4826020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed4813940 0x556ed481f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480e7d0 0x556ed4818540 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4158598202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed4818000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f257e000640 0 client.25528 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 >> 172.21.15.131:0/4158598202 conn(0x556ed4809450 msgr2=0x556ed4808650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 shutdown_connections 2024-04-26T20:14:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 1 -- 172.21.15.131:0/4158598202 wait complete. 2024-04-26T20:14:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.594+0000 7f259a1c8740 20 client.25528 trim_cache size 0 max 16384 2024-04-26T20:14:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 Processor -- start 2024-04-26T20:14:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 -- start start 2024-04-26T20:14:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480dde0 0x556ed480f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed480fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed4813940 0x556ed481f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25901afed0 con 0x556ed480e7d0 2024-04-26T20:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25901ad7a0 con 0x556ed4813940 2024-04-26T20:14:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f259a1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25901ab5f0 con 0x556ed480dde0 2024-04-26T20:14:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.611+0000 7f2587400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed480fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed480fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40428/0 (socket says 172.21.15.131:40428) 2024-04-26T20:14:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 -- 172.21.15.131:0/2222817176 learned_addr learned my addr 172.21.15.131:0/2222817176 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2595200640 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480dde0 0x556ed480f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2595200640 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed4813940 0x556ed481f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 -- 172.21.15.131:0/2222817176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480dde0 msgr2=0x556ed480f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480dde0 0x556ed480f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 -- 172.21.15.131:0/2222817176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed4813940 msgr2=0x556ed481f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed4813940 0x556ed481f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 -- 172.21.15.131:0/2222817176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed4550620 con 0x556ed480e7d0 2024-04-26T20:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2595200640 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480dde0 0x556ed480f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.612+0000 7f2587400640 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed480fef0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2578009340 tx=0x7f2578004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ebc698c8c8bdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f2584c00640 1 -- 172.21.15.131:0/2222817176 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578005000 con 0x556ed480e7d0 2024-04-26T20:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f2584c00640 1 -- 172.21.15.131:0/2222817176 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2578005650 con 0x556ed480e7d0 2024-04-26T20:14:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f2584c00640 1 -- 172.21.15.131:0/2222817176 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578003d00 con 0x556ed480e7d0 2024-04-26T20:14:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f259a1c8740 1 -- 172.21.15.131:0/2222817176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 msgr2=0x556ed480fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f259a1c8740 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed480fef0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2578009340 tx=0x7f2578004d50 comp rx=0 tx=0).stop 2024-04-26T20:14:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f259a1c8740 1 -- 172.21.15.131:0/2222817176 shutdown_connections 2024-04-26T20:14:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f259a1c8740 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed4813940 0x556ed481f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f259a1c8740 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed480fef0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f259a1c8740 1 --2- 172.21.15.131:0/2222817176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed480dde0 0x556ed480f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.613+0000 7f259a1c8740 1 -- 172.21.15.131:0/2222817176 >> 172.21.15.131:0/2222817176 conn(0x556ed4809450 msgr2=0x556ed4818000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.614+0000 7f259a1c8740 1 -- 172.21.15.131:0/2222817176 shutdown_connections 2024-04-26T20:14:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.614+0000 7f259a1c8740 1 -- 172.21.15.131:0/2222817176 wait complete. 2024-04-26T20:14:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.614+0000 7f259a1c8740 1 -- start start 2024-04-26T20:14:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.614+0000 7f259a1c8740 1 Processor -- start 2024-04-26T20:14:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.614+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed481ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed481cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f259a1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed481df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f259a1c8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25901abc60 con 0x556ed480e7d0 2024-04-26T20:14:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f259a1c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25901ad0d0 con 0x556ed480dde0 2024-04-26T20:14:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f259a1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25901ac330 con 0x556ed4813940 2024-04-26T20:14:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f259a1c8740 10 client.0 _ll_register_callbacks cb 0x7f2578009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:14:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed481ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2587400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed481cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed481ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:51744/0 (socket says 172.21.15.131:51744) 2024-04-26T20:14:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 -- 172.21.15.131:0/4151592944 learned_addr learned my addr 172.21.15.131:0/4151592944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed481df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 -- 172.21.15.131:0/4151592944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 msgr2=0x556ed481df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed481df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 -- 172.21.15.131:0/4151592944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 msgr2=0x556ed481cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed481cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 -- 172.21.15.131:0/4151592944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed472ccb0 con 0x556ed480dde0 2024-04-26T20:14:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f2595200640 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed481ca30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x556ed48268a0 tx=0x7f2588000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6054974d221e950 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.615+0000 7f257e000640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:14:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f257e000640 1 -- 172.21.15.131:0/4151592944 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258801e040 con 0x556ed480dde0 2024-04-26T20:14:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2578005650 con 0x556ed480dde0 2024-04-26T20:14:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f257e000640 1 -- 172.21.15.131:0/4151592944 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2588019430 con 0x556ed480dde0 2024-04-26T20:14:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f257e000640 1 -- 172.21.15.131:0/4151592944 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2588019630 con 0x556ed480dde0 2024-04-26T20:14:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 10 client.25535 fetch_fsmap learned FSMap version 27 2024-04-26T20:14:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ed481f660 con 0x556ed480dde0 2024-04-26T20:14:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f257e000640 1 -- 172.21.15.131:0/4151592944 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2578005650 con 0x556ed480dde0 2024-04-26T20:14:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f257e000640 1 -- 172.21.15.131:0/4151592944 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2588024040 con 0x556ed480dde0 2024-04-26T20:14:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 10 client.25535 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:14:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 10 client.25535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:14:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 20 client.25535 populate_metadata read hostname 'smithi131' 2024-04-26T20:14:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed488a580 0x556ed488c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 4 client.25535 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:14:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.616+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ed4779e00 con 0x556ed488a580 2024-04-26T20:14:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.617+0000 7f2587400640 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed488a580 0x556ed488c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.617+0000 7f2587400640 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed488a580 0x556ed488c970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3aee54a700dec1 server_cookie=a1354f64610bd174 in_seq=0 out_seq=0 2024-04-26T20:14:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.617+0000 7f257e000640 10 client.25535 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.629+0000 7f257e000640 1 -- 172.21.15.131:0/4151592944 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ed4779e00 con 0x556ed488a580 2024-04-26T20:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.629+0000 7f257e000640 10 client.25535 handle_command_reply: tid=0 2024-04-26T20:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 client.25535 shutdown 2024-04-26T20:14:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 msgr2=0x556ed481ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed481ca30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x556ed48268a0 tx=0x7f2588000bc0 comp rx=0 tx=0).stop 2024-04-26T20:14:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 shutdown_connections 2024-04-26T20:14:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556ed488a580 0x556ed488c970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ed4813940 0x556ed481df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ed480e7d0 0x556ed481cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 --2- 172.21.15.131:0/4151592944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556ed480dde0 0x556ed481ca30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f257e000640 0 client.25535 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 >> 172.21.15.131:0/4151592944 conn(0x556ed4809450 msgr2=0x556ed4812b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 shutdown_connections 2024-04-26T20:14:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 1 -- 172.21.15.131:0/4151592944 wait complete. 2024-04-26T20:14:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.630+0000 7f259a1c8740 20 client.25535 trim_cache size 0 max 16384 2024-04-26T20:14:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:14:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:14:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:14:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:14:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25740a3c30 msgr2=0x7f25740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25740a3c30 0x7f25740a60f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x556ed480ebd0 tx=0x7f25800f7530 comp rx=0 tx=0).stop 2024-04-26T20:14:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 msgr2=0x7f259019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 0x7f259019ac70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2580034040 tx=0x7f2580030d20 comp rx=0 tx=0).stop 2024-04-26T20:14:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 shutdown_connections 2024-04-26T20:14:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25740a3c30 0x7f25740a60f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25901065a0 0x7f25901a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2590105be0 0x7f259019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 --2- 172.21.15.131:0/3468669190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25901049f0 0x7f259019ac70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 >> 172.21.15.131:0/3468669190 conn(0x7f25900fff60 msgr2=0x7f2590108fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 shutdown_connections 2024-04-26T20:14:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:06.631+0000 7f2595c00640 1 -- 172.21.15.131:0/3468669190 wait complete. 2024-04-26T20:14:30.628 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:06 smithi131 ceph-mon[27919]: pgmap v533: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T20:14:30.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:07 smithi131 ceph-mon[27919]: scrub summary: idle 2024-04-26T20:14:30.774 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:09 smithi131 ceph-mon[27919]: pgmap v534: 129 pgs: 129 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T20:14:30.827 INFO:teuthology.orchestra.run.smithi131.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T20:14:30.890 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T20:14:30.897 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-26T20:14:30.903 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:14:30.910 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T20:14:30.936 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-26 20:09:09.506506730 +0000 2024-04-26T20:14:30.947 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-26 20:09:09.506506730 +0000 2024-04-26T20:14:31.051 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-26 20:12:46.029201266 +0000 2024-04-26T20:14:31.206 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-26 20:09:09.506506730 +0000 2024-04-26T20:14:31.286 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T20:14:31.293 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T20:14:31.300 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:13 smithi157 ceph-mon[32703]: pgmap v536: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T20:14:31.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:15 smithi157 ceph-mon[32703]: pgmap v537: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-26T20:14:31.465 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:17 smithi157 ceph-mon[32703]: pgmap v538: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T20:14:31.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:19 smithi157 ceph-mon[32703]: pgmap v539: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-26T20:14:31.479 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:21 smithi157 ceph-mon[32703]: pgmap v540: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:14:31.594 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:23 smithi157 ceph-mon[32703]: pgmap v541: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:14:31.639 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:25 smithi157 ceph-mon[32703]: pgmap v542: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:14:31.673 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:27 smithi157 ceph-mon[32703]: pgmap v543: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:31.679 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:29 smithi157 ceph-mon[32703]: pgmap v544: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:31.688 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:13 smithi148 ceph-mon[32281]: pgmap v536: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T20:14:31.832 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:15 smithi148 ceph-mon[32281]: pgmap v537: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-26T20:14:31.942 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:17 smithi148 ceph-mon[32281]: pgmap v538: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T20:14:32.050 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:19 smithi148 ceph-mon[32281]: pgmap v539: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-26T20:14:32.166 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:21 smithi148 ceph-mon[32281]: pgmap v540: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:14:32.244 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:23 smithi148 ceph-mon[32281]: pgmap v541: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:14:32.251 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:25 smithi148 ceph-mon[32281]: pgmap v542: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:14:32.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:27 smithi148 ceph-mon[32281]: pgmap v543: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:32.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:29 smithi148 ceph-mon[32281]: pgmap v544: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:32.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:11 smithi131 ceph-mon[27919]: pgmap v535: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-26T20:14:32.446 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:13 smithi131 ceph-mon[27919]: pgmap v536: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T20:14:32.572 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:15 smithi131 ceph-mon[27919]: pgmap v537: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-26T20:14:32.689 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:17 smithi131 ceph-mon[27919]: pgmap v538: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T20:14:32.787 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:31 smithi157 ceph-mon[32703]: pgmap v545: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:32.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:31 smithi148 ceph-mon[32281]: pgmap v545: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 -- 172.21.15.131:0/2270352541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090105da0 msgr2=0x7ff090106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 --2- 172.21.15.131:0/2270352541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090105da0 0x7ff090106220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:14:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 -- 172.21.15.131:0/2270352541 shutdown_connections 2024-04-26T20:14:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 --2- 172.21.15.131:0/2270352541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff090106760 0x7ff09010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 --2- 172.21.15.131:0/2270352541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090105da0 0x7ff090106220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 --2- 172.21.15.131:0/2270352541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090104bb0 0x7ff090104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 -- 172.21.15.131:0/2270352541 >> 172.21.15.131:0/2270352541 conn(0x7ff090100120 msgr2=0x7ff090102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 -- 172.21.15.131:0/2270352541 shutdown_connections 2024-04-26T20:14:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.777+0000 7ff098800640 1 -- 172.21.15.131:0/2270352541 wait complete. 2024-04-26T20:14:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.778+0000 7ff098800640 1 Processor -- start 2024-04-26T20:14:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.778+0000 7ff098800640 1 -- start start 2024-04-26T20:14:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.778+0000 7ff098800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff090104bb0 0x7ff09019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.778+0000 7ff098800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090105da0 0x7ff09019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff098800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090106760 0x7ff0901a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff098800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901106c0 con 0x7ff090106760 2024-04-26T20:14:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff098800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff090110540 con 0x7ff090105da0 2024-04-26T20:14:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff098800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff090110840 con 0x7ff090104bb0 2024-04-26T20:14:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff096800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090106760 0x7ff0901a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff095400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090105da0 0x7ff09019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff095400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090105da0 0x7ff09019b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36324/0 (socket says 172.21.15.131:36324) 2024-04-26T20:14:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff095400640 1 -- 172.21.15.131:0/4089963260 learned_addr learned my addr 172.21.15.131:0/4089963260 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff095e00640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff090104bb0 0x7ff09019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff096800640 1 -- 172.21.15.131:0/4089963260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff090104bb0 msgr2=0x7ff09019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff096800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff090104bb0 0x7ff09019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff096800640 1 -- 172.21.15.131:0/4089963260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090105da0 msgr2=0x7ff09019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff096800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090105da0 0x7ff09019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff096800640 1 -- 172.21.15.131:0/4089963260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901a2960 con 0x7ff090106760 2024-04-26T20:14:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.779+0000 7ff095400640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090105da0 0x7ff09019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.780+0000 7ff096800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090106760 0x7ff0901a22c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e940 tx=0x7ff08c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.780+0000 7ff08aa00640 1 -- 172.21.15.131:0/4089963260 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004d20 con 0x7ff090106760 2024-04-26T20:14:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.780+0000 7ff08aa00640 1 -- 172.21.15.131:0/4089963260 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08c005020 con 0x7ff090106760 2024-04-26T20:14:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.780+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901a2c50 con 0x7ff090106760 2024-04-26T20:14:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.780+0000 7ff08aa00640 1 -- 172.21.15.131:0/4089963260 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0116d0 con 0x7ff090106760 2024-04-26T20:14:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.780+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a3150 con 0x7ff090106760 2024-04-26T20:14:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.783+0000 7ff09b615740 1 Processor -- start 2024-04-26T20:14:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.783+0000 7ff08aa00640 1 -- 172.21.15.131:0/4089963260 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff08c005460 con 0x7ff090106760 2024-04-26T20:14:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff09b615740 1 -- start start 2024-04-26T20:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff09b615740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f4bd0 0x55e2111f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff09b615740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff09b615740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff09b615740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e211159380 con 0x55e2111f55c0 2024-04-26T20:14:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff09b615740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e2110f5c30 con 0x55e2111fa730 2024-04-26T20:14:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff09b615740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e211155340 con 0x55e2111f4bd0 2024-04-26T20:14:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff095e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff095400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f4bd0 0x55e2111f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff095e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111fa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33088/0 (socket says 172.21.15.131:33088) 2024-04-26T20:14:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.784+0000 7ff095e00640 1 -- 172.21.15.131:0/2555714885 learned_addr learned my addr 172.21.15.131:0/2555714885 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff095400640 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff095e00640 1 -- 172.21.15.131:0/2555714885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f4bd0 msgr2=0x55e2111f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff095e00640 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f4bd0 0x55e2111f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff095e00640 1 -- 172.21.15.131:0/2555714885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 msgr2=0x55e2111fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff095e00640 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff095e00640 1 -- 172.21.15.131:0/2555714885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e211117940 con 0x55e2111f55c0 2024-04-26T20:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff095e00640 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111fa0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b750 tx=0x7ff08400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab61ac8b8589ffcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff08aa00640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0780a3cf0 0x7ff0780a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff07f000640 1 -- 172.21.15.131:0/2555714885 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400be60 con 0x55e2111f55c0 2024-04-26T20:14:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff07f000640 1 -- 172.21.15.131:0/2555714885 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff084015070 con 0x55e2111f55c0 2024-04-26T20:14:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.785+0000 7ff07f000640 1 -- 172.21.15.131:0/2555714885 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084010680 con 0x55e2111f55c0 2024-04-26T20:14:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff08aa00640 1 -- 172.21.15.131:0/4089963260 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff08c0d0a60 con 0x7ff090106760 2024-04-26T20:14:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff095400640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0780a3cf0 0x7ff0780a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff09b615740 1 -- 172.21.15.131:0/2555714885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 msgr2=0x55e2111fa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff09b615740 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111fa0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b750 tx=0x7ff08400bc10 comp rx=0 tx=0).stop 2024-04-26T20:14:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff095400640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0780a3cf0 0x7ff0780a61b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff080009230 tx=0x7ff08003f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff09b615740 1 -- 172.21.15.131:0/2555714885 shutdown_connections 2024-04-26T20:14:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff09b615740 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff09b615740 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111fa0c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff09b615740 1 --2- 172.21.15.131:0/2555714885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f4bd0 0x55e2111f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.786+0000 7ff09b615740 1 -- 172.21.15.131:0/2555714885 >> 172.21.15.131:0/2555714885 conn(0x55e2111f0240 msgr2=0x55e2111f0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.787+0000 7ff09b615740 1 -- 172.21.15.131:0/2555714885 shutdown_connections 2024-04-26T20:14:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.787+0000 7ff09b615740 1 -- 172.21.15.131:0/2555714885 wait complete. 2024-04-26T20:14:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.787+0000 7ff09b615740 1 -- start start 2024-04-26T20:14:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.787+0000 7ff09b615740 1 Processor -- start 2024-04-26T20:14:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff09b615740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111f4bd0 0x55e2111fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff09b615740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff09b615740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111fa730 0x55e211206330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff09b615740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e2111fd6e0 con 0x55e2111f55c0 2024-04-26T20:14:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff09b615740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e2110e0c60 con 0x55e2111f4bd0 2024-04-26T20:14:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff09b615740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e2111fd860 con 0x55e2111fa730 2024-04-26T20:14:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff09b615740 10 client.0 _ll_register_callbacks cb 0x55e21110ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:14:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111f4bd0 0x55e2111fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111f4bd0 0x55e2111fedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36350/0 (socket says 172.21.15.131:36350) 2024-04-26T20:14:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 -- 172.21.15.131:0/3881380741 learned_addr learned my addr 172.21.15.131:0/3881380741 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095400640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff096800640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111fa730 0x55e211206330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 -- 172.21.15.131:0/3881380741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111fa730 msgr2=0x55e211206330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111fa730 0x55e211206330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 -- 172.21.15.131:0/3881380741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 msgr2=0x55e2111ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff095e00640 1 -- 172.21.15.131:0/3881380741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2110ef510 con 0x55e2111f4bd0 2024-04-26T20:14:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.788+0000 7ff096800640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111fa730 0x55e211206330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff095e00640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111f4bd0 0x55e2111fedb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c00 tx=0x7ff084003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4248e58476b350e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff073400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:14:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff073400640 1 -- 172.21.15.131:0/3881380741 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08402a060 con 0x55e2111f4bd0 2024-04-26T20:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff073400640 1 -- 172.21.15.131:0/3881380741 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff084011650 con 0x55e2111f4bd0 2024-04-26T20:14:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff084015070 con 0x55e2111f4bd0 2024-04-26T20:14:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff073400640 1 -- 172.21.15.131:0/3881380741 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff084031050 con 0x55e2111f4bd0 2024-04-26T20:14:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff09b615740 10 client.25547 fetch_fsmap learned FSMap version 27 2024-04-26T20:14:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.789+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e210e6b210 con 0x55e2111f4bd0 2024-04-26T20:14:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff073400640 1 -- 172.21.15.131:0/3881380741 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff084015070 con 0x55e2111f4bd0 2024-04-26T20:14:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff073400640 1 -- 172.21.15.131:0/3881380741 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff08401aa90 con 0x55e2111f4bd0 2024-04-26T20:14:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff09b615740 10 client.25547 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:14:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff09b615740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:14:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff09b615740 20 client.25547 populate_metadata read hostname 'smithi131' 2024-04-26T20:14:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff09b615740 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e21120a540 0x55e21120c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff09b615740 4 client.25547 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:14:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e211166f90 con 0x55e21120a540 2024-04-26T20:14:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.790+0000 7ff095400640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e21120a540 0x55e21120c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.791+0000 7ff095400640 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e21120a540 0x55e21120c930 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ffbee03a9dfdc server_cookie=74b17e9c0f1a2d09 in_seq=0 out_seq=0 2024-04-26T20:14:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.791+0000 7ff073400640 10 client.25547 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff073400640 1 -- 172.21.15.131:0/3881380741 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e211166f90 con 0x55e21120a540 2024-04-26T20:14:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff073400640 10 client.25547 handle_command_reply: tid=0 2024-04-26T20:14:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 client.25547 shutdown 2024-04-26T20:14:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111f4bd0 msgr2=0x55e2111fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111f4bd0 0x55e2111fedb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c00 tx=0x7ff084003e00 comp rx=0 tx=0).stop 2024-04-26T20:14:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 shutdown_connections 2024-04-26T20:14:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e21120a540 0x55e21120c930 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111fa730 0x55e211206330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f55c0 0x55e2111ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff073400640 0 client.25547 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 --2- 172.21.15.131:0/3881380741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111f4bd0 0x55e2111fedb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 >> 172.21.15.131:0/3881380741 conn(0x55e2111f0240 msgr2=0x55e2111ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 shutdown_connections 2024-04-26T20:14:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.792+0000 7ff09b615740 1 -- 172.21.15.131:0/3881380741 wait complete. 2024-04-26T20:14:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.793+0000 7ff09b615740 20 client.25547 trim_cache size 0 max 16384 2024-04-26T20:14:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.808+0000 7ff09b615740 1 Processor -- start 2024-04-26T20:14:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff09b615740 1 -- start start 2024-04-26T20:14:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff09b615740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e2111f3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff09b615740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff09b615740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff09b615740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901b00f0 con 0x55e2111f4bd0 2024-04-26T20:14:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff09b615740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ad9c0 con 0x55e2111fa730 2024-04-26T20:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff09b615740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ab810 con 0x55e2111f55c0 2024-04-26T20:14:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff095e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e2111f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff095400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff096800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff095400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46168/0 (socket says 172.21.15.131:46168) 2024-04-26T20:14:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.809+0000 7ff095400640 1 -- 172.21.15.131:0/2786656279 learned_addr learned my addr 172.21.15.131:0/2786656279 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff095400640 1 -- 172.21.15.131:0/2786656279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 msgr2=0x55e2111ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff095400640 1 --2- 172.21.15.131:0/2786656279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff095400640 1 -- 172.21.15.131:0/2786656279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 msgr2=0x55e2111f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff095400640 1 --2- 172.21.15.131:0/2786656279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e2111f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff095400640 1 -- 172.21.15.131:0/2786656279 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21120cca0 con 0x55e2111f55c0 2024-04-26T20:14:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff095400640 1 --2- 172.21.15.131:0/2786656279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112062f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff080006010 tx=0x7ff08002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d69fd451d3fe18b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff07e600640 1 -- 172.21.15.131:0/2786656279 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080035040 con 0x55e2111f55c0 2024-04-26T20:14:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff07e600640 1 -- 172.21.15.131:0/2786656279 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff080056710 con 0x55e2111f55c0 2024-04-26T20:14:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.810+0000 7ff07e600640 1 -- 172.21.15.131:0/2786656279 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080003dd0 con 0x55e2111f55c0 2024-04-26T20:14:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 -- 172.21.15.131:0/2786656279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 msgr2=0x55e2112062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 --2- 172.21.15.131:0/2786656279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112062f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff080006010 tx=0x7ff08002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:14:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 -- 172.21.15.131:0/2786656279 shutdown_connections 2024-04-26T20:14:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 --2- 172.21.15.131:0/2786656279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2111ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 --2- 172.21.15.131:0/2786656279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112062f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 --2- 172.21.15.131:0/2786656279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e2111f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 -- 172.21.15.131:0/2786656279 >> 172.21.15.131:0/2786656279 conn(0x55e2111f0240 msgr2=0x55e2111eebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 -- 172.21.15.131:0/2786656279 shutdown_connections 2024-04-26T20:14:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 -- 172.21.15.131:0/2786656279 wait complete. 2024-04-26T20:14:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.811+0000 7ff09b615740 1 -- start start 2024-04-26T20:14:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 1 Processor -- start 2024-04-26T20:14:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e211203480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2112049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901abe80 con 0x55e2111f4bd0 2024-04-26T20:14:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ad2f0 con 0x55e2111fa730 2024-04-26T20:14:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ac550 con 0x55e2111f55c0 2024-04-26T20:14:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff09b615740 10 client.0 _ll_register_callbacks cb 0x7ff080005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:14:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff095400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff095e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e211203480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff096800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2112049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff095400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46184/0 (socket says 172.21.15.131:46184) 2024-04-26T20:14:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff095400640 1 -- 172.21.15.131:0/2687296944 learned_addr learned my addr 172.21.15.131:0/2687296944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.812+0000 7ff096800640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2112049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36360/0 (socket says 172.21.15.131:36360) 2024-04-26T20:14:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff096800640 1 -- 172.21.15.131:0/2687296944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 msgr2=0x55e2112039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff096800640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff096800640 1 -- 172.21.15.131:0/2687296944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 msgr2=0x55e211203480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff096800640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e211203480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff096800640 1 -- 172.21.15.131:0/2687296944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e210f37620 con 0x55e2111fa730 2024-04-26T20:14:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff095400640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff095e00640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e211203480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:14:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff096800640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2112049d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0cdfc0 tx=0x7ff08c09c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e769f0819ab58971 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff073400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:14:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff073400640 1 -- 172.21.15.131:0/2687296944 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c01f040 con 0x55e2111fa730 2024-04-26T20:14:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff073400640 1 -- 172.21.15.131:0/2687296944 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff08c09bc90 con 0x55e2111fa730 2024-04-26T20:14:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.813+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff080056710 con 0x55e2111fa730 2024-04-26T20:14:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff073400640 1 -- 172.21.15.131:0/2687296944 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff08c09c840 con 0x55e2111fa730 2024-04-26T20:14:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 10 client.25553 fetch_fsmap learned FSMap version 27 2024-04-26T20:14:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e211113cb0 con 0x55e2111fa730 2024-04-26T20:14:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff073400640 1 -- 172.21.15.131:0/2687296944 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff080056710 con 0x55e2111fa730 2024-04-26T20:14:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff073400640 1 -- 172.21.15.131:0/2687296944 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff08c00f040 con 0x55e2111fa730 2024-04-26T20:14:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 10 client.25553 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:14:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:14:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 20 client.25553 populate_metadata read hostname 'smithi131' 2024-04-26T20:14:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e211270a60 0x55e211272e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 4 client.25553 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:14:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.814+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e211160e00 con 0x55e211270a60 2024-04-26T20:14:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.815+0000 7ff095e00640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e211270a60 0x55e211272e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.815+0000 7ff095e00640 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e211270a60 0x55e211272e80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc51c15cc5b2baa server_cookie=b700e4a3db44cc50 in_seq=0 out_seq=0 2024-04-26T20:14:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.815+0000 7ff073400640 10 client.25553 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff073400640 1 -- 172.21.15.131:0/2687296944 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e211160e00 con 0x55e211270a60 2024-04-26T20:14:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff073400640 10 client.25553 handle_command_reply: tid=0 2024-04-26T20:14:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 client.25553 shutdown 2024-04-26T20:14:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 msgr2=0x55e2112049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2112049d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0cdfc0 tx=0x7ff08c09c730 comp rx=0 tx=0).stop 2024-04-26T20:14:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 shutdown_connections 2024-04-26T20:14:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e211270a60 0x55e211272e80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2111fa730 0x55e2112049d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2111f55c0 0x55e2112039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 --2- 172.21.15.131:0/2687296944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e2111f4bd0 0x55e211203480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff073400640 0 client.25553 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:14:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 >> 172.21.15.131:0/2687296944 conn(0x55e2111f0240 msgr2=0x55e2111eebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 shutdown_connections 2024-04-26T20:14:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.819+0000 7ff09b615740 1 -- 172.21.15.131:0/2687296944 wait complete. 2024-04-26T20:14:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff09b615740 20 client.25553 trim_cache size 0 max 16384 2024-04-26T20:14:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:14:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:14:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:14:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:14:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0780a3cf0 msgr2=0x7ff0780a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0780a3cf0 0x7ff0780a61b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff080009230 tx=0x7ff08003f040 comp rx=0 tx=0).stop 2024-04-26T20:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090106760 msgr2=0x7ff0901a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090106760 0x7ff0901a22c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e940 tx=0x7ff08c00ee00 comp rx=0 tx=0).stop 2024-04-26T20:14:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 shutdown_connections 2024-04-26T20:14:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090106760 0x7ff0901a22c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff090105da0 0x7ff09019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff0780a3cf0 0x7ff0780a61b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 --2- 172.21.15.131:0/4089963260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff090104bb0 0x7ff09019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 >> 172.21.15.131:0/4089963260 conn(0x7ff090100120 msgr2=0x7ff0901090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.820+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 shutdown_connections 2024-04-26T20:14:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:19.821+0000 7ff098800640 1 -- 172.21.15.131:0/4089963260 wait complete. 2024-04-26T20:14:50.589 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:19 smithi131 ceph-mon[27919]: pgmap v539: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-26T20:14:50.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:21 smithi131 ceph-mon[27919]: pgmap v540: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:14:50.739 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:23 smithi131 ceph-mon[27919]: pgmap v541: 129 pgs: 129 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:14:50.836 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:25 smithi131 ceph-mon[27919]: pgmap v542: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:14:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:27 smithi131 ceph-mon[27919]: pgmap v543: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:51.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:29 smithi131 ceph-mon[27919]: pgmap v544: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:51.214 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:33 smithi157 ceph-mon[32703]: pgmap v546: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:51.360 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:35 smithi157 ceph-mon[32703]: pgmap v547: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:14:51.440 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:37 smithi157 ceph-mon[32703]: pgmap v548: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:14:51.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:37 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:14:51.722 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:39 smithi157 ceph-mon[32703]: pgmap v549: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:14:51.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:14:51.934 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:14:51.941 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:39 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:14:52.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:41 smithi157 ceph-mon[32703]: pgmap v550: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:52.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:43 smithi157 ceph-mon[32703]: pgmap v551: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:52.269 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:45 smithi157 ceph-mon[32703]: pgmap v552: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:14:52.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:47 smithi157 ceph-mon[32703]: pgmap v553: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:52.400 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:49 smithi157 ceph-mon[32703]: pgmap v554: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:52.406 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:33 smithi148 ceph-mon[32281]: pgmap v546: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:52.498 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:35 smithi148 ceph-mon[32281]: pgmap v547: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:14:52.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:37 smithi148 ceph-mon[32281]: pgmap v548: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:14:52.642 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:37 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:14:52.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:39 smithi148 ceph-mon[32281]: pgmap v549: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:14:52.672 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:14:52.819 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:14:52.841 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:39 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:14:52.892 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:41 smithi148 ceph-mon[32281]: pgmap v550: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:52.978 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:43 smithi148 ceph-mon[32281]: pgmap v551: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:52.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:45 smithi148 ceph-mon[32281]: pgmap v552: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:14:52.991 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:47 smithi148 ceph-mon[32281]: pgmap v553: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:14:52.997 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:49 smithi148 ceph-mon[32281]: pgmap v554: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:14:53.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:51 smithi148 ceph-mon[32281]: pgmap v555: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:14:53.152 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:51 smithi157 ceph-mon[32703]: pgmap v555: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:14:53.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 -- 172.21.15.131:0/3793005395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc105c60 msgr2=0x7feadc1060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:53.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 --2- 172.21.15.131:0/3793005395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc105c60 0x7feadc1060e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feac4009900 tx=0x7feac402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:14:53.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 -- 172.21.15.131:0/3793005395 shutdown_connections 2024-04-26T20:14:53.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 --2- 172.21.15.131:0/3793005395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc106620 0x7feadc10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:53.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 --2- 172.21.15.131:0/3793005395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc105c60 0x7feadc1060e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:53.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 --2- 172.21.15.131:0/3793005395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feadc104a70 0x7feadc104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:53.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 -- 172.21.15.131:0/3793005395 >> 172.21.15.131:0/3793005395 conn(0x7feadc0fffe0 msgr2=0x7feadc102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 -- 172.21.15.131:0/3793005395 shutdown_connections 2024-04-26T20:14:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.489+0000 7feadbe00640 1 -- 172.21.15.131:0/3793005395 wait complete. 2024-04-26T20:14:54.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.490+0000 7feadbe00640 1 Processor -- start 2024-04-26T20:14:54.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadbe00640 1 -- start start 2024-04-26T20:14:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc104a70 0x7feadc19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc105c60 0x7feadc19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:54.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feadc106620 0x7feadc1a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1105d0 con 0x7feadc104a70 2024-04-26T20:14:54.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feadc110450 con 0x7feadc105c60 2024-04-26T20:14:54.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feadc110750 con 0x7feadc106620 2024-04-26T20:14:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc104a70 0x7feadc19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:54.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc104a70 0x7feadc19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55478/0 (socket says 172.21.15.131:55478) 2024-04-26T20:14:54.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadaa00640 1 -- 172.21.15.131:0/3009872525 learned_addr learned my addr 172.21.15.131:0/3009872525 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:54.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7feadb400640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feadc106620 0x7feadc1a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:54.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.491+0000 7fead2000640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc105c60 0x7feadc19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:55.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadaa00640 1 -- 172.21.15.131:0/3009872525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feadc106620 msgr2=0x7feadc1a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:55.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadaa00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feadc106620 0x7feadc1a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:55.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadaa00640 1 -- 172.21.15.131:0/3009872525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc105c60 msgr2=0x7feadc19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:55.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadaa00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc105c60 0x7feadc19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:55.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadaa00640 1 -- 172.21.15.131:0/3009872525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1a28b0 con 0x7feadc104a70 2024-04-26T20:14:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadaa00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc104a70 0x7feadc19ac70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feacc00d950 tx=0x7feacc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:55.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7fead3e00640 1 -- 172.21.15.131:0/3009872525 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc017020 con 0x7feadc104a70 2024-04-26T20:14:55.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7fead3e00640 1 -- 172.21.15.131:0/3009872525 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feacc009c30 con 0x7feadc104a70 2024-04-26T20:14:55.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7fead3e00640 1 -- 172.21.15.131:0/3009872525 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc00f690 con 0x7feadc104a70 2024-04-26T20:14:55.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc1a2ba0 con 0x7feadc104a70 2024-04-26T20:14:55.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.492+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a3110 con 0x7feadc104a70 2024-04-26T20:14:55.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.495+0000 7feae49b0740 1 Processor -- start 2024-04-26T20:14:55.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feae49b0740 1 -- start start 2024-04-26T20:14:55.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7fead3e00640 1 -- 172.21.15.131:0/3009872525 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7feacc01a020 con 0x7feadc104a70 2024-04-26T20:14:55.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feae49b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7e960 0x55f43fa7ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feae49b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa83e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:56.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feae49b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa86950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:56.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feae49b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f43f98f030 con 0x55f43fa844c0 2024-04-26T20:14:56.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feae49b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f43f9aa7f0 con 0x55f43fa7e960 2024-04-26T20:14:56.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feae49b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f43f9657e0 con 0x55f43fa7f350 2024-04-26T20:14:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feadb400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7e960 0x55f43fa7ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7fead2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa86950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:56.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feadaa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa83e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feadb400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7e960 0x55f43fa7ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50948/0 (socket says 172.21.15.131:50948) 2024-04-26T20:14:56.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.496+0000 7feadb400640 1 -- 172.21.15.131:0/379647244 learned_addr learned my addr 172.21.15.131:0/379647244 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feadb400640 1 -- 172.21.15.131:0/379647244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 msgr2=0x55f43fa83e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feadb400640 1 --2- 172.21.15.131:0/379647244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa83e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:56.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feadb400640 1 -- 172.21.15.131:0/379647244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 msgr2=0x55f43fa86950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:56.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feadb400640 1 --2- 172.21.15.131:0/379647244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa86950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:56.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feadb400640 1 -- 172.21.15.131:0/379647244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43f971050 con 0x55f43fa7e960 2024-04-26T20:14:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feadb400640 1 --2- 172.21.15.131:0/379647244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7e960 0x55f43fa7ed80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fead4009f50 tx=0x7fead40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461edbb00f57df66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feacbe00640 1 -- 172.21.15.131:0/379647244 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4004d20 con 0x55f43fa7e960 2024-04-26T20:14:57.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feacbe00640 1 -- 172.21.15.131:0/379647244 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fead4004050 con 0x55f43fa7e960 2024-04-26T20:14:57.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.497+0000 7feacbe00640 1 -- 172.21.15.131:0/379647244 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4005280 con 0x55f43fa7e960 2024-04-26T20:14:57.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7fead3e00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7feac00a3cc0 0x7feac00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feadaa00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7feac00a3cc0 0x7feac00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:57.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7fead3e00640 1 -- 172.21.15.131:0/3009872525 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7feacc0d07b0 con 0x7feadc104a70 2024-04-26T20:14:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feae49b0740 1 -- 172.21.15.131:0/379647244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7e960 msgr2=0x55f43fa7ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:14:57.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feae49b0740 1 --2- 172.21.15.131:0/379647244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7e960 0x55f43fa7ed80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fead4009f50 tx=0x7fead40118d0 comp rx=0 tx=0).stop 2024-04-26T20:14:57.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feadaa00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7feac00a3cc0 0x7feac00a6180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55f43fa7f750 tx=0x7feacc0d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feae49b0740 1 -- 172.21.15.131:0/379647244 shutdown_connections 2024-04-26T20:14:57.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feae49b0740 1 --2- 172.21.15.131:0/379647244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa86950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feae49b0740 1 --2- 172.21.15.131:0/379647244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa83e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feae49b0740 1 --2- 172.21.15.131:0/379647244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7e960 0x55f43fa7ed80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:57.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.498+0000 7feae49b0740 1 -- 172.21.15.131:0/379647244 >> 172.21.15.131:0/379647244 conn(0x55f43fa79fd0 msgr2=0x55f43fa7a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:14:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.499+0000 7feae49b0740 1 -- 172.21.15.131:0/379647244 shutdown_connections 2024-04-26T20:14:57.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.499+0000 7feae49b0740 1 -- 172.21.15.131:0/379647244 wait complete. 2024-04-26T20:14:57.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.499+0000 7feae49b0740 1 -- start start 2024-04-26T20:14:57.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.499+0000 7feae49b0740 1 Processor -- start 2024-04-26T20:14:57.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feae49b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7e960 0x55f43fa887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feae49b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa88cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:58.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feae49b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa8fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:14:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feae49b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f43f9e7200 con 0x55f43fa844c0 2024-04-26T20:14:58.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feae49b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f43f9e1ee0 con 0x55f43fa7f350 2024-04-26T20:14:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feae49b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f43f9eee70 con 0x55f43fa7e960 2024-04-26T20:14:58.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feae49b0740 10 client.0 _ll_register_callbacks cb 0x55f43f9d8270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:14:58.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa88cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:58.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feadb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa8fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:58.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa88cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50962/0 (socket says 172.21.15.131:50962) 2024-04-26T20:14:58.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 -- 172.21.15.131:0/3413306005 learned_addr learned my addr 172.21.15.131:0/3413306005 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:14:58.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feadb400640 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7e960 0x55f43fa887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:14:58.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 -- 172.21.15.131:0/3413306005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7e960 msgr2=0x55f43fa887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7e960 0x55f43fa887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:58.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 -- 172.21.15.131:0/3413306005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 msgr2=0x55f43fa8fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:14:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa8fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:14:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7fead2000640 1 -- 172.21.15.131:0/3413306005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43f7be7d0 con 0x55f43fa7f350 2024-04-26T20:14:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.500+0000 7feadb400640 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa8fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:14:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7fead2000640 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa88cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feac4006010 tx=0x7feac4004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d2a632bbf285e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:14:59.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7feac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:14:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7feac8c00640 1 -- 172.21.15.131:0/3413306005 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4035040 con 0x55f43fa7f350 2024-04-26T20:14:59.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead4004050 con 0x55f43fa7f350 2024-04-26T20:14:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7feac8c00640 1 -- 172.21.15.131:0/3413306005 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7feac4003ac0 con 0x55f43fa7f350 2024-04-26T20:14:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7feac8c00640 1 -- 172.21.15.131:0/3413306005 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feac404e050 con 0x55f43fa7f350 2024-04-26T20:14:59.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7feae49b0740 10 client.25565 fetch_fsmap learned FSMap version 27 2024-04-26T20:14:59.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.501+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f43f6f43f0 con 0x55f43fa7f350 2024-04-26T20:14:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feac8c00640 1 -- 172.21.15.131:0/3413306005 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fead4004050 con 0x55f43fa7f350 2024-04-26T20:14:59.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feac8c00640 1 -- 172.21.15.131:0/3413306005 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7feac403c440 con 0x55f43fa7f350 2024-04-26T20:14:59.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feae49b0740 10 client.25565 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:14:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feae49b0740 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:14:59.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feae49b0740 20 client.25565 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:00.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feae49b0740 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fa942c0 0x55f43fa966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:00.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feae49b0740 4 client.25565 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:15:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f43f9f07e0 con 0x55f43fa942c0 2024-04-26T20:15:00.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.502+0000 7feadb400640 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fa942c0 0x55f43fa966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:00.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.503+0000 7feadb400640 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fa942c0 0x55f43fa966e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cfd957344c8c448 server_cookie=978fc379e0985646 in_seq=0 out_seq=0 2024-04-26T20:15:00.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.503+0000 7feac8c00640 10 client.25565 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:00.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feac8c00640 1 -- 172.21.15.131:0/3413306005 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f43f9f07e0 con 0x55f43fa942c0 2024-04-26T20:15:00.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feac8c00640 10 client.25565 handle_command_reply: tid=0 2024-04-26T20:15:01.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 client.25565 shutdown 2024-04-26T20:15:01.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 msgr2=0x55f43fa88cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:01.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa88cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feac4006010 tx=0x7feac4004440 comp rx=0 tx=0).stop 2024-04-26T20:15:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 shutdown_connections 2024-04-26T20:15:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fa942c0 0x55f43fa966e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:01.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa844c0 0x55f43fa8fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:01.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa88cf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:01.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 --2- 172.21.15.131:0/3413306005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7e960 0x55f43fa887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:01.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feac8c00640 0 client.25565 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:01.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 >> 172.21.15.131:0/3413306005 conn(0x55f43fa79fd0 msgr2=0x55f43fa791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 shutdown_connections 2024-04-26T20:15:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 1 -- 172.21.15.131:0/3413306005 wait complete. 2024-04-26T20:15:01.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.504+0000 7feae49b0740 20 client.25565 trim_cache size 0 max 16384 2024-04-26T20:15:01.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.523+0000 7feae49b0740 1 Processor -- start 2024-04-26T20:15:02.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.523+0000 7feae49b0740 1 -- start start 2024-04-26T20:15:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.523+0000 7feae49b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa78960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:02.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.523+0000 7feae49b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa8fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feae49b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa844c0 0x55f43fa7cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:02.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feae49b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1ad7e0 con 0x55f43fa7e960 2024-04-26T20:15:02.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feae49b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1ab630 con 0x55f43fa844c0 2024-04-26T20:15:02.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feae49b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1ad110 con 0x55f43fa7f350 2024-04-26T20:15:02.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feadb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa78960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:02.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feadb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa78960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55512/0 (socket says 172.21.15.131:55512) 2024-04-26T20:15:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feadb400640 1 -- 172.21.15.131:0/696185992 learned_addr learned my addr 172.21.15.131:0/696185992 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:02.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7fead2000640 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa8fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:02.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feadb400640 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa844c0 0x55f43fa7cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:02.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7fead2000640 1 -- 172.21.15.131:0/696185992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa844c0 msgr2=0x55f43fa7cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:02.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7fead2000640 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa844c0 0x55f43fa7cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:02.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7fead2000640 1 -- 172.21.15.131:0/696185992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 msgr2=0x55f43fa78960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:02.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7fead2000640 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa78960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:03.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7fead2000640 1 -- 172.21.15.131:0/696185992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43f6b40f0 con 0x55f43fa7f350 2024-04-26T20:15:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7feadb400640 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa844c0 0x55f43fa7cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:15:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.524+0000 7fead2000640 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa8fc80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feac4005ab0 tx=0x7feac402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=364594924b24a4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:03.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feacb400640 1 -- 172.21.15.131:0/696185992 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac404b070 con 0x55f43fa7f350 2024-04-26T20:15:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feacb400640 1 -- 172.21.15.131:0/696185992 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac4005850 con 0x55f43fa7f350 2024-04-26T20:15:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feacb400640 1 -- 172.21.15.131:0/696185992 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4035070 con 0x55f43fa7f350 2024-04-26T20:15:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feae49b0740 1 -- 172.21.15.131:0/696185992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 msgr2=0x55f43fa8fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:03.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feae49b0740 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa8fc80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feac4005ab0 tx=0x7feac402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:15:03.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feae49b0740 1 -- 172.21.15.131:0/696185992 shutdown_connections 2024-04-26T20:15:03.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feae49b0740 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa844c0 0x55f43fa7cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:03.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feae49b0740 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa7f350 0x55f43fa8fc80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:04.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feae49b0740 1 --2- 172.21.15.131:0/696185992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa78960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:04.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.525+0000 7feae49b0740 1 -- 172.21.15.131:0/696185992 >> 172.21.15.131:0/696185992 conn(0x55f43fa79fd0 msgr2=0x55f43fa87fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:04.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.526+0000 7feae49b0740 1 -- 172.21.15.131:0/696185992 shutdown_connections 2024-04-26T20:15:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.526+0000 7feae49b0740 1 -- 172.21.15.131:0/696185992 wait complete. 2024-04-26T20:15:04.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.526+0000 7feae49b0740 1 -- start start 2024-04-26T20:15:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.526+0000 7feae49b0740 1 Processor -- start 2024-04-26T20:15:04.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.526+0000 7feae49b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa8d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:04.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.526+0000 7feae49b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa8da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:04.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.526+0000 7feae49b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa844c0 0x55f43fa8e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feae49b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1ac370 con 0x55f43fa7e960 2024-04-26T20:15:04.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feae49b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1abca0 con 0x55f43fa7f350 2024-04-26T20:15:04.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feae49b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1aca40 con 0x55f43fa844c0 2024-04-26T20:15:04.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feae49b0740 10 client.0 _ll_register_callbacks cb 0x7feac4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:15:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa8d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:04.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa8d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55524/0 (socket says 172.21.15.131:55524) 2024-04-26T20:15:04.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 -- 172.21.15.131:0/2704104747 learned_addr learned my addr 172.21.15.131:0/2704104747 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:04.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7fead2000640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa8da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:04.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa844c0 0x55f43fa8e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:04.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 -- 172.21.15.131:0/2704104747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa844c0 msgr2=0x55f43fa8e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:04.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa844c0 0x55f43fa8e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:05.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 -- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 msgr2=0x55f43fa8da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:05.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa8da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:05.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7feadb400640 1 -- 172.21.15.131:0/2704104747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43f9aaa30 con 0x55f43fa7e960 2024-04-26T20:15:05.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.527+0000 7fead2000640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa8da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feadb400640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa8d530 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fead40041d0 tx=0x7fead40046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0b41c59deadb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:05.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:15:05.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feac8c00640 1 -- 172.21.15.131:0/2704104747 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead402ae60 con 0x55f43fa7e960 2024-04-26T20:15:05.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feac8c00640 1 -- 172.21.15.131:0/2704104747 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fead40047c0 con 0x55f43fa7e960 2024-04-26T20:15:05.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac4005850 con 0x55f43fa7e960 2024-04-26T20:15:05.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feac8c00640 1 -- 172.21.15.131:0/2704104747 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fead4039050 con 0x55f43fa7e960 2024-04-26T20:15:05.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feae49b0740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-04-26T20:15:05.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f43fb03070 con 0x55f43fa7e960 2024-04-26T20:15:05.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.528+0000 7feac8c00640 1 -- 172.21.15.131:0/2704104747 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac4005850 con 0x55f43fa7e960 2024-04-26T20:15:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7feac8c00640 1 -- 172.21.15.131:0/2704104747 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fead402a3c0 con 0x55f43fa7e960 2024-04-26T20:15:05.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7feae49b0740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:15:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7feae49b0740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:15:05.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7feae49b0740 20 client.15450 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:05.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7feae49b0740 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fb038b0 0x55f43fb05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:06.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7feae49b0740 4 client.15450 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T20:15:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f43f9ea650 con 0x55f43fb038b0 2024-04-26T20:15:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.529+0000 7fead2000640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fb038b0 0x55f43fb05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7fead2000640 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fb038b0 0x55f43fb05cd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1038cec1c2395c5f server_cookie=5170d631483a98f2 in_seq=0 out_seq=0 2024-04-26T20:15:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feac8c00640 10 client.15450 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:06.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feac8c00640 1 -- 172.21.15.131:0/2704104747 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f43f9ea650 con 0x55f43fb038b0 2024-04-26T20:15:06.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feac8c00640 10 client.15450 handle_command_reply: tid=0 2024-04-26T20:15:06.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 client.15450 shutdown 2024-04-26T20:15:06.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 msgr2=0x55f43fa8d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa8d530 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fead40041d0 tx=0x7fead40046e0 comp rx=0 tx=0).stop 2024-04-26T20:15:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 shutdown_connections 2024-04-26T20:15:06.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f43fb038b0 0x55f43fb05cd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:06.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43fa844c0 0x55f43fa8e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:07.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f43fa7f350 0x55f43fa8da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 --2- 172.21.15.131:0/2704104747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f43fa7e960 0x55f43fa8d530 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feac8c00640 0 client.15450 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:07.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 >> 172.21.15.131:0/2704104747 conn(0x55f43fa79fd0 msgr2=0x55f43fa7fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:07.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 shutdown_connections 2024-04-26T20:15:07.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 1 -- 172.21.15.131:0/2704104747 wait complete. 2024-04-26T20:15:07.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.530+0000 7feae49b0740 20 client.15450 trim_cache size 0 max 16384 2024-04-26T20:15:07.515 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:15:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-04-26T20:15:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-04-26T20:15:07.618 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:15:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7feac00a3cc0 msgr2=0x7feac00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7feac00a3cc0 0x7feac00a6180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55f43fa7f750 tx=0x7feacc0d6530 comp rx=0 tx=0).stop 2024-04-26T20:15:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc104a70 msgr2=0x7feadc19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc104a70 0x7feadc19ac70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feacc00d950 tx=0x7feacc00de10 comp rx=0 tx=0).stop 2024-04-26T20:15:08.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 shutdown_connections 2024-04-26T20:15:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7feac00a3cc0 0x7feac00a6180 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:08.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feadc106620 0x7feadc1a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:08.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc105c60 0x7feadc19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 --2- 172.21.15.131:0/3009872525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feadc104a70 0x7feadc19ac70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 >> 172.21.15.131:0/3009872525 conn(0x7feadc0fffe0 msgr2=0x7feadc108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:08.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 shutdown_connections 2024-04-26T20:15:08.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:14:31.531+0000 7feadbe00640 1 -- 172.21.15.131:0/3009872525 wait complete. 2024-04-26T20:15:08.463 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T20:15:08.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:15:08.480 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:31 smithi131 ceph-mon[27919]: pgmap v545: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:15:08.588 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:53 smithi148 ceph-mon[32281]: pgmap v556: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:08.789 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:55 smithi148 ceph-mon[32281]: pgmap v557: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:15:08.894 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:57 smithi148 ceph-mon[32281]: pgmap v558: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T20:15:08.992 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:14:59 smithi148 ceph-mon[32281]: pgmap v559: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T20:15:09.093 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:01 smithi148 ceph-mon[32281]: pgmap v560: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T20:15:09.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:03 smithi148 ceph-mon[32281]: pgmap v561: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:15:09.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:05 smithi148 ceph-mon[32281]: pgmap v562: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-26T20:15:09.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:07 smithi148 ceph-mon[32281]: pgmap v563: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:15:09.617 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:53 smithi157 ceph-mon[32703]: pgmap v556: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:09.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:55 smithi157 ceph-mon[32703]: pgmap v557: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:15:09.864 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:57 smithi157 ceph-mon[32703]: pgmap v558: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T20:15:09.904 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:14:59 smithi157 ceph-mon[32703]: pgmap v559: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T20:15:10.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:01 smithi157 ceph-mon[32703]: pgmap v560: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T20:15:10.098 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:03 smithi157 ceph-mon[32703]: pgmap v561: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:15:10.157 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:05 smithi157 ceph-mon[32703]: pgmap v562: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-26T20:15:10.271 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:07 smithi157 ceph-mon[32703]: pgmap v563: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:15:10.302 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:09 smithi148 ceph-mon[32281]: pgmap v564: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T20:15:10.315 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:09 smithi157 ceph-mon[32703]: pgmap v564: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T20:15:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 -- 172.21.15.131:0/2757435815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa610075330 msgr2=0x7fa610075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 --2- 172.21.15.131:0/2757435815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa610075330 0x7fa610075790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009900 tx=0x7fa5fc02a480 comp rx=0 tx=0).stop 2024-04-26T20:15:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 -- 172.21.15.131:0/2757435815 shutdown_connections 2024-04-26T20:15:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 --2- 172.21.15.131:0/2757435815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa61010a820 0x7fa61010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 --2- 172.21.15.131:0/2757435815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa610075330 0x7fa610075790 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 --2- 172.21.15.131:0/2757435815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610076910 0x7fa610074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 -- 172.21.15.131:0/2757435815 >> 172.21.15.131:0/2757435815 conn(0x7fa6100fdf00 msgr2=0x7fa610100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 -- 172.21.15.131:0/2757435815 shutdown_connections 2024-04-26T20:15:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.960+0000 7fa617400640 1 -- 172.21.15.131:0/2757435815 wait complete. 2024-04-26T20:15:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.961+0000 7fa617400640 1 Processor -- start 2024-04-26T20:15:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa617400640 1 -- start start 2024-04-26T20:15:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa617400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610075330 0x7fa61019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa617400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa610076910 0x7fa61019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa617400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa61010a820 0x7fa6101a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa617400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110710 con 0x7fa61010a820 2024-04-26T20:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa617400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110590 con 0x7fa610076910 2024-04-26T20:15:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa617400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110890 con 0x7fa610075330 2024-04-26T20:15:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa615400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa61010a820 0x7fa6101a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa614a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610075330 0x7fa61019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa614a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610075330 0x7fa61019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34384/0 (socket says 172.21.15.131:34384) 2024-04-26T20:15:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa614a00640 1 -- 172.21.15.131:0/340125061 learned_addr learned my addr 172.21.15.131:0/340125061 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa615400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa61010a820 0x7fa6101a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55544/0 (socket says 172.21.15.131:55544) 2024-04-26T20:15:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.962+0000 7fa60fe00640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa610076910 0x7fa61019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa614a00640 1 -- 172.21.15.131:0/340125061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa610076910 msgr2=0x7fa61019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa614a00640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa610076910 0x7fa61019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa614a00640 1 -- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa61010a820 msgr2=0x7fa6101a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa614a00640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa61010a820 0x7fa6101a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa614a00640 1 -- 172.21.15.131:0/340125061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101a2960 con 0x7fa610075330 2024-04-26T20:15:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa615400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa61010a820 0x7fa6101a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa614a00640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610075330 0x7fa61019ad50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa60400e9f0 tx=0x7fa60400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa60d600640 1 -- 172.21.15.131:0/340125061 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60400ccc0 con 0x7fa610075330 2024-04-26T20:15:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.963+0000 7fa60d600640 1 -- 172.21.15.131:0/340125061 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa60400ce60 con 0x7fa610075330 2024-04-26T20:15:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.964+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6101a2c50 con 0x7fa610075330 2024-04-26T20:15:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.964+0000 7fa60d600640 1 -- 172.21.15.131:0/340125061 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604010430 con 0x7fa610075330 2024-04-26T20:15:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.964+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6101a3150 con 0x7fa610075330 2024-04-26T20:15:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.967+0000 7fa60d600640 1 -- 172.21.15.131:0/340125061 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa6040046c0 con 0x7fa610075330 2024-04-26T20:15:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.967+0000 7fa61a1f5740 1 Processor -- start 2024-04-26T20:15:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.967+0000 7fa61a1f5740 1 -- start start 2024-04-26T20:15:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2094ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d209cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5638d1ff9380 con 0x5638d20955c0 2024-04-26T20:15:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5638d1f95c30 con 0x5638d2094bd0 2024-04-26T20:15:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5638d1ff5340 con 0x5638d209a730 2024-04-26T20:15:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa614a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d209cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa615400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2094ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa614a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55550/0 (socket says 172.21.15.131:55550) 2024-04-26T20:15:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.968+0000 7fa614a00640 1 -- 172.21.15.131:0/2458940239 learned_addr learned my addr 172.21.15.131:0/2458940239 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60fe00640 1 -- 172.21.15.131:0/2458940239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 msgr2=0x5638d2094ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2094ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60fe00640 1 -- 172.21.15.131:0/2458940239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 msgr2=0x5638d209a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60fe00640 1 -- 172.21.15.131:0/2458940239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638d1fb7940 con 0x5638d209a730 2024-04-26T20:15:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa615400640 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2094ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d209cbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc02aa70 tx=0x7fa5fc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef323f97db1c614 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60d600640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa5f80a3cf0 0x7fa5f80a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa601c00640 1 -- 172.21.15.131:0/2458940239 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc035030 con 0x5638d209a730 2024-04-26T20:15:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa601c00640 1 -- 172.21.15.131:0/2458940239 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5fc040070 con 0x5638d209a730 2024-04-26T20:15:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa601c00640 1 -- 172.21.15.131:0/2458940239 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc031800 con 0x5638d209a730 2024-04-26T20:15:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa60d600640 1 -- 172.21.15.131:0/340125061 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa6040d0e20 con 0x7fa610075330 2024-04-26T20:15:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.969+0000 7fa615400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa5f80a3cf0 0x7fa5f80a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2458940239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 msgr2=0x5638d209cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d209cbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc02aa70 tx=0x7fa5fc005c00 comp rx=0 tx=0).stop 2024-04-26T20:15:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2458940239 shutdown_connections 2024-04-26T20:15:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d209cbc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2458940239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2094ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2458940239 >> 172.21.15.131:0/2458940239 conn(0x5638d2090240 msgr2=0x5638d2090640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa615400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa5f80a3cf0 0x7fa5f80a61b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5638d2095e20 tx=0x7fa608009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2458940239 shutdown_connections 2024-04-26T20:15:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.970+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2458940239 wait complete. 2024-04-26T20:15:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 -- start start 2024-04-26T20:15:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 Processor -- start 2024-04-26T20:15:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d2094bd0 0x5638d209ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d209a730 0x5638d20a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5638d209d680 con 0x5638d20955c0 2024-04-26T20:15:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5638d1f80c60 con 0x5638d209a730 2024-04-26T20:15:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.971+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5638d209d800 con 0x5638d2094bd0 2024-04-26T20:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa61a1f5740 10 client.0 _ll_register_callbacks cb 0x5638d1face50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:15:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55552/0 (socket says 172.21.15.131:55552) 2024-04-26T20:15:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 -- 172.21.15.131:0/2226331448 learned_addr learned my addr 172.21.15.131:0/2226331448 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa615400640 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d209a730 0x5638d20a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa614a00640 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d2094bd0 0x5638d209ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 -- 172.21.15.131:0/2226331448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d2094bd0 msgr2=0x5638d209ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d2094bd0 0x5638d209ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 -- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d209a730 msgr2=0x5638d20a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d209a730 0x5638d20a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 -- 172.21.15.131:0/2226331448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638d1f8f510 con 0x5638d20955c0 2024-04-26T20:15:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209f320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc006010 tx=0x7fa5fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2be20319571fb0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.972+0000 7fa5f6000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:15:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.973+0000 7fa5f6000640 1 -- 172.21.15.131:0/2226331448 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc052060 con 0x5638d20955c0 2024-04-26T20:15:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.973+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5fc040070 con 0x5638d20955c0 2024-04-26T20:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.973+0000 7fa5f6000640 1 -- 172.21.15.131:0/2226331448 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa5fc003a00 con 0x5638d20955c0 2024-04-26T20:15:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.973+0000 7fa5f6000640 1 -- 172.21.15.131:0/2226331448 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5fc059050 con 0x5638d20955c0 2024-04-26T20:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.973+0000 7fa61a1f5740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-04-26T20:15:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.973+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638d1d0b210 con 0x5638d20955c0 2024-04-26T20:15:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.973+0000 7fa5f6000640 1 -- 172.21.15.131:0/2226331448 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5fc040070 con 0x5638d20955c0 2024-04-26T20:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa5f6000640 1 -- 172.21.15.131:0/2226331448 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa5fc045500 con 0x5638d20955c0 2024-04-26T20:15:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa61a1f5740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa61a1f5740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:15:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa61a1f5740 20 client.15462 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d20aa450 0x5638d20ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa61a1f5740 4 client.15462 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:15:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638d2006f90 con 0x5638d20aa450 2024-04-26T20:15:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.974+0000 7fa614a00640 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d20aa450 0x5638d20ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.975+0000 7fa614a00640 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d20aa450 0x5638d20ac840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e51312a4fdc51b server_cookie=44ea29983efa1b39 in_seq=0 out_seq=0 2024-04-26T20:15:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.975+0000 7fa5f6000640 10 client.15462 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.975+0000 7fa5f6000640 1 -- 172.21.15.131:0/2226331448 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638d2006f90 con 0x5638d20aa450 2024-04-26T20:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.975+0000 7fa5f6000640 10 client.15462 handle_command_reply: tid=0 2024-04-26T20:15:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.975+0000 7fa61a1f5740 1 client.15462 shutdown 2024-04-26T20:15:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.975+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 msgr2=0x5638d209f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.975+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209f320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc006010 tx=0x7fa5fc000bc0 comp rx=0 tx=0).stop 2024-04-26T20:15:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 shutdown_connections 2024-04-26T20:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d20aa450 0x5638d20ac840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d209a730 0x5638d20a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d209f320 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2226331448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d2094bd0 0x5638d209ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa5f6000640 0 client.15462 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 >> 172.21.15.131:0/2226331448 conn(0x5638d2090240 msgr2=0x5638d208f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 shutdown_connections 2024-04-26T20:15:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2226331448 wait complete. 2024-04-26T20:15:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.976+0000 7fa61a1f5740 20 client.15462 trim_cache size 0 max 16384 2024-04-26T20:15:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.992+0000 7fa61a1f5740 1 Processor -- start 2024-04-26T20:15:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.992+0000 7fa61a1f5740 1 -- start start 2024-04-26T20:15:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.992+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2092fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.992+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.992+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d2093500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.992+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101aff70 con 0x5638d20955c0 2024-04-26T20:15:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ad840 con 0x5638d2094bd0 2024-04-26T20:15:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ab690 con 0x5638d209a730 2024-04-26T20:15:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d2093500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d2093500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34414/0 (socket says 172.21.15.131:34414) 2024-04-26T20:15:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 -- 172.21.15.131:0/1122965687 learned_addr learned my addr 172.21.15.131:0/1122965687 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa60fe00640 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa614a00640 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2092fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 -- 172.21.15.131:0/1122965687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 msgr2=0x5638d2092fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2092fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 -- 172.21.15.131:0/1122965687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 msgr2=0x5638d20a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.993+0000 7fa615400640 1 -- 172.21.15.131:0/1122965687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638d20acbb0 con 0x5638d209a730 2024-04-26T20:15:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.994+0000 7fa615400640 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d2093500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6080120d0 tx=0x7fa608017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5b459b91030dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.994+0000 7fa601200640 1 -- 172.21.15.131:0/1122965687 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608003a30 con 0x5638d209a730 2024-04-26T20:15:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.994+0000 7fa601200640 1 -- 172.21.15.131:0/1122965687 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa608003bd0 con 0x5638d209a730 2024-04-26T20:15:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.994+0000 7fa601200640 1 -- 172.21.15.131:0/1122965687 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608010560 con 0x5638d209a730 2024-04-26T20:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.994+0000 7fa61a1f5740 1 -- 172.21.15.131:0/1122965687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 msgr2=0x5638d2093500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d2093500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6080120d0 tx=0x7fa608017000 comp rx=0 tx=0).stop 2024-04-26T20:15:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 -- 172.21.15.131:0/1122965687 shutdown_connections 2024-04-26T20:15:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d2093500 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/1122965687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d2092fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 -- 172.21.15.131:0/1122965687 >> 172.21.15.131:0/1122965687 conn(0x5638d2090240 msgr2=0x5638d2096840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 -- 172.21.15.131:0/1122965687 shutdown_connections 2024-04-26T20:15:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 -- 172.21.15.131:0/1122965687 wait complete. 2024-04-26T20:15:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 -- start start 2024-04-26T20:15:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.995+0000 7fa61a1f5740 1 Processor -- start 2024-04-26T20:15:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d20a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa61a1f5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d20a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101abd00 con 0x5638d20955c0 2024-04-26T20:15:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ad170 con 0x5638d2094bd0 2024-04-26T20:15:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa61a1f5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ac3d0 con 0x5638d209a730 2024-04-26T20:15:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa61a1f5740 10 client.0 _ll_register_callbacks cb 0x7fa6040d47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:15:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa614a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d20a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55562/0 (socket says 172.21.15.131:55562) 2024-04-26T20:15:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa60fe00640 1 -- 172.21.15.131:0/2493501955 learned_addr learned my addr 172.21.15.131:0/2493501955 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa615400640 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d20a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa614a00640 1 -- 172.21.15.131:0/2493501955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 msgr2=0x5638d20a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa614a00640 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d20a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa614a00640 1 -- 172.21.15.131:0/2493501955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 msgr2=0x5638d20a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa614a00640 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa614a00640 1 -- 172.21.15.131:0/2493501955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638d1dd7620 con 0x5638d2094bd0 2024-04-26T20:15:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.996+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa614a00640 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d20a3640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa60409cae0 tx=0x7fa604098e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97a38957048f9f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa5f6000640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:15:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa5f6000640 1 -- 172.21.15.131:0/2493501955 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60409c4b0 con 0x5638d2094bd0 2024-04-26T20:15:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa5f6000640 1 -- 172.21.15.131:0/2493501955 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa6040ce5d0 con 0x5638d2094bd0 2024-04-26T20:15:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa608003bd0 con 0x5638d2094bd0 2024-04-26T20:15:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa5f6000640 1 -- 172.21.15.131:0/2493501955 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa60409ac90 con 0x5638d2094bd0 2024-04-26T20:15:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa61a1f5740 10 client.25589 fetch_fsmap learned FSMap version 27 2024-04-26T20:15:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638d1fb3cb0 con 0x5638d2094bd0 2024-04-26T20:15:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.997+0000 7fa5f6000640 1 -- 172.21.15.131:0/2493501955 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa608003bd0 con 0x5638d2094bd0 2024-04-26T20:15:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa5f6000640 1 -- 172.21.15.131:0/2493501955 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa6040a5330 con 0x5638d2094bd0 2024-04-26T20:15:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa61a1f5740 10 client.25589 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:15:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa61a1f5740 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:15:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa61a1f5740 20 client.25589 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d2110b40 0x5638d2112f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa61a1f5740 4 client.25589 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:15:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638d2000e00 con 0x5638d2110b40 2024-04-26T20:15:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d2110b40 0x5638d2112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa5f6000640 10 client.25589 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:32.998+0000 7fa60fe00640 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d2110b40 0x5638d2112f60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1aa41d97b03f3 server_cookie=af993e8fad52454a in_seq=0 out_seq=0 2024-04-26T20:15:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.002+0000 7fa5f6000640 1 -- 172.21.15.131:0/2493501955 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638d2000e00 con 0x5638d2110b40 2024-04-26T20:15:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.002+0000 7fa5f6000640 10 client.25589 handle_command_reply: tid=0 2024-04-26T20:15:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.002+0000 7fa61a1f5740 1 client.25589 shutdown 2024-04-26T20:15:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 msgr2=0x5638d20a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d20a3640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa60409cae0 tx=0x7fa604098e20 comp rx=0 tx=0).stop 2024-04-26T20:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 shutdown_connections 2024-04-26T20:15:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5638d2110b40 0x5638d2112f60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638d209a730 0x5638d20a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638d20955c0 0x5638d20a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa5f6000640 0 client.25589 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 --2- 172.21.15.131:0/2493501955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638d2094bd0 0x5638d20a3640 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 >> 172.21.15.131:0/2493501955 conn(0x5638d2090240 msgr2=0x5638d2092fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 shutdown_connections 2024-04-26T20:15:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 1 -- 172.21.15.131:0/2493501955 wait complete. 2024-04-26T20:15:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.003+0000 7fa61a1f5740 20 client.25589 trim_cache size 0 max 16384 2024-04-26T20:15:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:15:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:15:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:15:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:15:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa5f80a3cf0 msgr2=0x7fa5f80a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa5f80a3cf0 0x7fa5f80a61b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5638d2095e20 tx=0x7fa608009500 comp rx=0 tx=0).stop 2024-04-26T20:15:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610075330 msgr2=0x7fa61019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610075330 0x7fa61019ad50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa60400e9f0 tx=0x7fa60400edf0 comp rx=0 tx=0).stop 2024-04-26T20:15:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 shutdown_connections 2024-04-26T20:15:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa61010a820 0x7fa6101a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa610076910 0x7fa61019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa5f80a3cf0 0x7fa5f80a61b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 --2- 172.21.15.131:0/340125061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa610075330 0x7fa61019ad50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 >> 172.21.15.131:0/340125061 conn(0x7fa6100fdf00 msgr2=0x7fa6101067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 shutdown_connections 2024-04-26T20:15:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:33.004+0000 7fa617400640 1 -- 172.21.15.131:0/340125061 wait complete. 2024-04-26T20:15:27.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:33 smithi131 ceph-mon[27919]: pgmap v546: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:27.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:35 smithi131 ceph-mon[27919]: pgmap v547: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:15:27.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:37 smithi131 ceph-mon[27919]: pgmap v548: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:15:27.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:37 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:15:27.647 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:39 smithi131 ceph-mon[27919]: pgmap v549: 129 pgs: 129 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:15:27.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:15:27.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:15:27.971 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:39 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:15:28.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:41 smithi131 ceph-mon[27919]: pgmap v550: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:15:28.232 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:43 smithi131 ceph-mon[27919]: pgmap v551: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:28.357 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:45 smithi131 ceph-mon[27919]: pgmap v552: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:15:28.429 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:11 smithi148 ceph-mon[32281]: pgmap v565: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T20:15:28.476 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:13 smithi148 ceph-mon[32281]: pgmap v566: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-26T20:15:28.584 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:15 smithi148 ceph-mon[32281]: pgmap v567: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-26T20:15:28.591 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:17 smithi148 ceph-mon[32281]: pgmap v568: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:28.686 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:19 smithi148 ceph-mon[32281]: pgmap v569: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:15:28.802 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:21 smithi148 ceph-mon[32281]: pgmap v570: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:15:28.938 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:23 smithi148 ceph-mon[32281]: pgmap v571: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:29.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:25 smithi148 ceph-mon[32281]: pgmap v572: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:15:29.080 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:27 smithi148 ceph-mon[32281]: pgmap v573: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:15:29.152 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:11 smithi157 ceph-mon[32703]: pgmap v565: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T20:15:29.225 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:13 smithi157 ceph-mon[32703]: pgmap v566: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-26T20:15:29.294 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:15 smithi157 ceph-mon[32703]: pgmap v567: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-26T20:15:29.454 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:17 smithi157 ceph-mon[32703]: pgmap v568: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:29.618 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:19 smithi157 ceph-mon[32703]: pgmap v569: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:15:29.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:21 smithi157 ceph-mon[32703]: pgmap v570: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:15:29.723 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:23 smithi157 ceph-mon[32703]: pgmap v571: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:29.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:25 smithi157 ceph-mon[32703]: pgmap v572: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:15:29.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:27 smithi157 ceph-mon[32703]: pgmap v573: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:15:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 -- 172.21.15.131:0/3371493560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 msgr2=0x7fc94810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 --2- 172.21.15.131:0/3371493560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 0x7fc94810a450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc930009860 tx=0x7fc93002a340 comp rx=0 tx=0).stop 2024-04-26T20:15:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 -- 172.21.15.131:0/3371493560 shutdown_connections 2024-04-26T20:15:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 --2- 172.21.15.131:0/3371493560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc94810a990 0x7fc948111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 --2- 172.21.15.131:0/3371493560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 0x7fc94810a450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 --2- 172.21.15.131:0/3371493560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 0x7fc948109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 -- 172.21.15.131:0/3371493560 >> 172.21.15.131:0/3371493560 conn(0x7fc948077dc0 msgr2=0x7fc94807a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 -- 172.21.15.131:0/3371493560 shutdown_connections 2024-04-26T20:15:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.146+0000 7fc947200640 1 -- 172.21.15.131:0/3371493560 wait complete. 2024-04-26T20:15:30.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.147+0000 7fc947200640 1 Processor -- start 2024-04-26T20:15:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.147+0000 7fc947200640 1 -- start start 2024-04-26T20:15:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc947200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 0x7fc94819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc947200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 0x7fc94819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc947200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc94810a990 0x7fc9481a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc947200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481148f0 con 0x7fc948108de0 2024-04-26T20:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc947200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc948114770 con 0x7fc94810a990 2024-04-26T20:15:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc947200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc948114a70 con 0x7fc948109fd0 2024-04-26T20:15:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc945e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 0x7fc94819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc945e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 0x7fc94819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47252/0 (socket says 172.21.15.131:47252) 2024-04-26T20:15:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc946800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc94810a990 0x7fc9481a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc945e00640 1 -- 172.21.15.131:0/844051995 learned_addr learned my addr 172.21.15.131:0/844051995 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.148+0000 7fc93d400640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 0x7fc94819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc945e00640 1 -- 172.21.15.131:0/844051995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 msgr2=0x7fc94819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc945e00640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 0x7fc94819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc945e00640 1 -- 172.21.15.131:0/844051995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc94810a990 msgr2=0x7fc9481a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc945e00640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc94810a990 0x7fc9481a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc945e00640 1 -- 172.21.15.131:0/844051995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9481a6c20 con 0x7fc948108de0 2024-04-26T20:15:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc945e00640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 0x7fc94819f050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc93800e9c0 tx=0x7fc93800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc93f400640 1 -- 172.21.15.131:0/844051995 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93800cd80 con 0x7fc948108de0 2024-04-26T20:15:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc93f400640 1 -- 172.21.15.131:0/844051995 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9380035f0 con 0x7fc948108de0 2024-04-26T20:15:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9481a6f10 con 0x7fc948108de0 2024-04-26T20:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc93f400640 1 -- 172.21.15.131:0/844051995 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc938010690 con 0x7fc948108de0 2024-04-26T20:15:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.149+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9481a7370 con 0x7fc948108de0 2024-04-26T20:15:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc93f400640 1 -- 172.21.15.131:0/844051995 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc9380037a0 con 0x7fc948108de0 2024-04-26T20:15:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 Processor -- start 2024-04-26T20:15:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 -- start start 2024-04-26T20:15:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561f7f12f380 con 0x561f7f1d0940 2024-04-26T20:15:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561f7f0cbc30 con 0x561f7f1cade0 2024-04-26T20:15:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f7f12b340 con 0x561f7f1cb7d0 2024-04-26T20:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.153+0000 7fc93d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc946800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1d2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47262/0 (socket says 172.21.15.131:47262) 2024-04-26T20:15:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 -- 172.21.15.131:0/278040445 learned_addr learned my addr 172.21.15.131:0/278040445 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc945e00640 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 -- 172.21.15.131:0/278040445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 msgr2=0x561f7f1d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 -- 172.21.15.131:0/278040445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 msgr2=0x561f7f1cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 -- 172.21.15.131:0/278040445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7f0ed940 con 0x561f7f1d0940 2024-04-26T20:15:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc946800640 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93f400640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc92c0a3c30 0x7fc92c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93d400640 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1d2dd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc9300057c0 tx=0x7fc930005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f1fd719aa7adb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc93f400640 1 -- 172.21.15.131:0/844051995 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc9380d0d20 con 0x7fc948108de0 2024-04-26T20:15:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.154+0000 7fc946800640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc92c0a3c30 0x7fc92c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.155+0000 7fc937400640 1 -- 172.21.15.131:0/278040445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930041020 con 0x561f7f1d0940 2024-04-26T20:15:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.155+0000 7fc937400640 1 -- 172.21.15.131:0/278040445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc930030dd0 con 0x561f7f1d0940 2024-04-26T20:15:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.155+0000 7fc937400640 1 -- 172.21.15.131:0/278040445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930037040 con 0x561f7f1d0940 2024-04-26T20:15:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.155+0000 7fc94fd9b740 1 -- 172.21.15.131:0/278040445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 msgr2=0x561f7f1d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.155+0000 7fc946800640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc92c0a3c30 0x7fc92c0a60f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc940009fd0 tx=0x7fc940010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.155+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1d2dd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc9300057c0 tx=0x7fc930005b20 comp rx=0 tx=0).stop 2024-04-26T20:15:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 -- 172.21.15.131:0/278040445 shutdown_connections 2024-04-26T20:15:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1d2dd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/278040445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 -- 172.21.15.131:0/278040445 >> 172.21.15.131:0/278040445 conn(0x561f7f1c6450 msgr2=0x561f7f1c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 -- 172.21.15.131:0/278040445 shutdown_connections 2024-04-26T20:15:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 -- 172.21.15.131:0/278040445 wait complete. 2024-04-26T20:15:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 -- start start 2024-04-26T20:15:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.156+0000 7fc94fd9b740 1 Processor -- start 2024-04-26T20:15:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cade0 0x561f7f1d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1dc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561f7f1d3890 con 0x561f7f1d0940 2024-04-26T20:15:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561f7f0b6c60 con 0x561f7f1cb7d0 2024-04-26T20:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f7f1d3a10 con 0x561f7f1cade0 2024-04-26T20:15:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc94fd9b740 10 client.0 _ll_register_callbacks cb 0x561f7f0e2e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:15:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc946800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1dc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc945e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cade0 0x561f7f1d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc946800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1dc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47272/0 (socket says 172.21.15.131:47272) 2024-04-26T20:15:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc946800640 1 -- 172.21.15.131:0/2731320236 learned_addr learned my addr 172.21.15.131:0/2731320236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc945e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cade0 0x561f7f1d5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58610/0 (socket says 172.21.15.131:58610) 2024-04-26T20:15:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc93d400640 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc945e00640 1 -- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cb7d0 msgr2=0x561f7f1d5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc945e00640 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc945e00640 1 -- 172.21.15.131:0/2731320236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 msgr2=0x561f7f1dc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.157+0000 7fc945e00640 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1dc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc945e00640 1 -- 172.21.15.131:0/2731320236 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7f0c5510 con 0x561f7f1cade0 2024-04-26T20:15:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc945e00640 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cade0 0x561f7f1d5020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ce280 tx=0x7fc938098690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be431c5bd6ec16f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc92be00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:15:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc92be00640 1 -- 172.21.15.131:0/2731320236 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93809c3b0 con 0x561f7f1cade0 2024-04-26T20:15:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc92be00640 1 -- 172.21.15.131:0/2731320236 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc9380ce4d0 con 0x561f7f1cade0 2024-04-26T20:15:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc930030dd0 con 0x561f7f1cade0 2024-04-26T20:15:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc92be00640 1 -- 172.21.15.131:0/2731320236 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc93809d150 con 0x561f7f1cade0 2024-04-26T20:15:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc94fd9b740 10 client.25597 fetch_fsmap learned FSMap version 27 2024-04-26T20:15:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.158+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f7ee41210 con 0x561f7f1cade0 2024-04-26T20:15:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc92be00640 1 -- 172.21.15.131:0/2731320236 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc930030dd0 con 0x561f7f1cade0 2024-04-26T20:15:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc92be00640 1 -- 172.21.15.131:0/2731320236 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc9380987a0 con 0x561f7f1cade0 2024-04-26T20:15:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc94fd9b740 10 client.25597 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:15:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc94fd9b740 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:15:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc94fd9b740 20 client.25597 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f1e0b00 0x561f7f1e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc94fd9b740 4 client.25597 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:15:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f7f13cf90 con 0x561f7f1e0b00 2024-04-26T20:15:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.159+0000 7fc93d400640 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f1e0b00 0x561f7f1e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.160+0000 7fc93d400640 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f1e0b00 0x561f7f1e2f50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9feb322294a6d6b3 server_cookie=9f108e29738ac333 in_seq=0 out_seq=0 2024-04-26T20:15:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.160+0000 7fc92be00640 10 client.25597 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc92be00640 1 -- 172.21.15.131:0/2731320236 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f7f13cf90 con 0x561f7f1e0b00 2024-04-26T20:15:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc92be00640 10 client.25597 handle_command_reply: tid=0 2024-04-26T20:15:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 client.25597 shutdown 2024-04-26T20:15:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cade0 msgr2=0x561f7f1d5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cade0 0x561f7f1d5020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ce280 tx=0x7fc938098690 comp rx=0 tx=0).stop 2024-04-26T20:15:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 shutdown_connections 2024-04-26T20:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f1e0b00 0x561f7f1e2f50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f1dc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2731320236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cade0 0x561f7f1d5020 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc92be00640 0 client.25597 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 >> 172.21.15.131:0/2731320236 conn(0x561f7f1c6450 msgr2=0x561f7f1c5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 shutdown_connections 2024-04-26T20:15:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2731320236 wait complete. 2024-04-26T20:15:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.161+0000 7fc94fd9b740 20 client.25597 trim_cache size 0 max 16384 2024-04-26T20:15:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 Processor -- start 2024-04-26T20:15:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 -- start start 2024-04-26T20:15:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1cb7d0 0x561f7f1dc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1d0940 0x561f7f1c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481b4370 con 0x561f7f1cb7d0 2024-04-26T20:15:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481b1c40 con 0x561f7f1cade0 2024-04-26T20:15:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.177+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481afa90 con 0x561f7f1d0940 2024-04-26T20:15:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc946800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1d0940 0x561f7f1c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:40766/0 (socket says 172.21.15.131:40766) 2024-04-26T20:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc946800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1d0940 0x561f7f1c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58626/0 (socket says 172.21.15.131:58626) 2024-04-26T20:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 -- 172.21.15.131:0/2778847482 learned_addr learned my addr 172.21.15.131:0/2778847482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc93d400640 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1cb7d0 0x561f7f1dc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 -- 172.21.15.131:0/2778847482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1d0940 msgr2=0x561f7f1c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1d0940 0x561f7f1c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 -- 172.21.15.131:0/2778847482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1cb7d0 msgr2=0x561f7f1dc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1cb7d0 0x561f7f1dc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc945e00640 1 -- 172.21.15.131:0/2778847482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7ef0d620 con 0x561f7f1cade0 2024-04-26T20:15:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.178+0000 7fc946800640 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1d0940 0x561f7f1c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.179+0000 7fc945e00640 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1c94e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc938098eb0 tx=0x7fc93809af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de80c987385e0c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.179+0000 7fc936a00640 1 -- 172.21.15.131:0/2778847482 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93807f070 con 0x561f7f1cade0 2024-04-26T20:15:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.179+0000 7fc936a00640 1 -- 172.21.15.131:0/2778847482 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc93809ca50 con 0x561f7f1cade0 2024-04-26T20:15:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.179+0000 7fc936a00640 1 -- 172.21.15.131:0/2778847482 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380d3030 con 0x561f7f1cade0 2024-04-26T20:15:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.179+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2778847482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 msgr2=0x561f7f1c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.179+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1c94e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc938098eb0 tx=0x7fc93809af60 comp rx=0 tx=0).stop 2024-04-26T20:15:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2778847482 shutdown_connections 2024-04-26T20:15:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1d0940 0x561f7f1c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1cb7d0 0x561f7f1dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/2778847482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1c94e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2778847482 >> 172.21.15.131:0/2778847482 conn(0x561f7f1c6450 msgr2=0x561f7f1cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2778847482 shutdown_connections 2024-04-26T20:15:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 -- 172.21.15.131:0/2778847482 wait complete. 2024-04-26T20:15:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 -- start start 2024-04-26T20:15:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.180+0000 7fc94fd9b740 1 Processor -- start 2024-04-26T20:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc94fd9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f245090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481b0100 con 0x561f7f1d0940 2024-04-26T20:15:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481b1570 con 0x561f7f1cade0 2024-04-26T20:15:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc94fd9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481b07d0 con 0x561f7f1cb7d0 2024-04-26T20:15:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc94fd9b740 10 client.0 _ll_register_callbacks cb 0x7fc9380d44c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:15:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1d9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:40782/0 (socket says 172.21.15.131:40782) 2024-04-26T20:15:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 -- 172.21.15.131:0/1933171077 learned_addr learned my addr 172.21.15.131:0/1933171077 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc946800640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f245090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc93d400640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 -- 172.21.15.131:0/1933171077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 msgr2=0x561f7f1d9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 -- 172.21.15.131:0/1933171077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 msgr2=0x561f7f245090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f245090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 -- 172.21.15.131:0/1933171077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7f0e9cb0 con 0x561f7f1cade0 2024-04-26T20:15:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc946800640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f245090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:15:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc945e00640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1d9530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c6a0 tx=0x7fc938003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c842312d424f9be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.181+0000 7fc92be00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:15:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc92be00640 1 -- 172.21.15.131:0/1933171077 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93807f070 con 0x561f7f1cade0 2024-04-26T20:15:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc92be00640 1 -- 172.21.15.131:0/1933171077 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc9380a5230 con 0x561f7f1cade0 2024-04-26T20:15:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93809ca50 con 0x561f7f1cade0 2024-04-26T20:15:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc92be00640 1 -- 172.21.15.131:0/1933171077 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9380d3070 con 0x561f7f1cade0 2024-04-26T20:15:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 10 client.25607 fetch_fsmap learned FSMap version 27 2024-04-26T20:15:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f7f2467c0 con 0x561f7f1cade0 2024-04-26T20:15:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc92be00640 1 -- 172.21.15.131:0/1933171077 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc93809ca50 con 0x561f7f1cade0 2024-04-26T20:15:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc92be00640 1 -- 172.21.15.131:0/1933171077 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc93809b910 con 0x561f7f1cade0 2024-04-26T20:15:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 10 client.25607 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:15:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:15:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 20 client.25607 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f246e80 0x561f7f2492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 4 client.25607 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:15:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.182+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f7f136e00 con 0x561f7f246e80 2024-04-26T20:15:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.183+0000 7fc93d400640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f246e80 0x561f7f2492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.183+0000 7fc93d400640 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f246e80 0x561f7f2492a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4de2009d29ddc24 server_cookie=9b65f1bf162f4acd in_seq=0 out_seq=0 2024-04-26T20:15:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.183+0000 7fc92be00640 10 client.25607 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc92be00640 1 -- 172.21.15.131:0/1933171077 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f7f136e00 con 0x561f7f246e80 2024-04-26T20:15:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc92be00640 10 client.25607 handle_command_reply: tid=0 2024-04-26T20:15:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 client.25607 shutdown 2024-04-26T20:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 msgr2=0x561f7f1d9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1d9530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c6a0 tx=0x7fc938003cf0 comp rx=0 tx=0).stop 2024-04-26T20:15:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 shutdown_connections 2024-04-26T20:15:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561f7f246e80 0x561f7f2492a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f7f1d0940 0x561f7f245090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f7f1cb7d0 0x561f7f1d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 --2- 172.21.15.131:0/1933171077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f7f1cade0 0x561f7f1d9530 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc92be00640 0 client.25607 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 >> 172.21.15.131:0/1933171077 conn(0x561f7f1c6450 msgr2=0x561f7f1cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 shutdown_connections 2024-04-26T20:15:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 1 -- 172.21.15.131:0/1933171077 wait complete. 2024-04-26T20:15:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.187+0000 7fc94fd9b740 20 client.25607 trim_cache size 0 max 16384 2024-04-26T20:15:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:15:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:15:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:15:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:15:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc92c0a3c30 msgr2=0x7fc92c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc92c0a3c30 0x7fc92c0a60f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc940009fd0 tx=0x7fc940010040 comp rx=0 tx=0).stop 2024-04-26T20:15:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 msgr2=0x7fc94819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 0x7fc94819f050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc93800e9c0 tx=0x7fc93800ee80 comp rx=0 tx=0).stop 2024-04-26T20:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 shutdown_connections 2024-04-26T20:15:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc94810a990 0x7fc9481a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc92c0a3c30 0x7fc92c0a60f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc948109fd0 0x7fc94819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 --2- 172.21.15.131:0/844051995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc948108de0 0x7fc94819f050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 >> 172.21.15.131:0/844051995 conn(0x7fc948077dc0 msgr2=0x7fc94810d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 shutdown_connections 2024-04-26T20:15:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:46.188+0000 7fc947200640 1 -- 172.21.15.131:0/844051995 wait complete. 2024-04-26T20:15:43.590 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:47 smithi131 ceph-mon[27919]: pgmap v553: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:43.595 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:49 smithi131 ceph-mon[27919]: pgmap v554: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:15:43.688 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:29 smithi148 ceph-mon[32281]: pgmap v574: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:15:43.788 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:30 smithi148 ceph-mon[32281]: pgmap v575: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T20:15:43.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:32 smithi148 ceph-mon[32281]: pgmap v576: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T20:15:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:34 smithi148 ceph-mon[32281]: pgmap v577: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T20:15:43.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:36 smithi148 ceph-mon[32281]: pgmap v578: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T20:15:43.937 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:38 smithi148 ceph-mon[32281]: pgmap v579: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:15:43.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:38 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:15:43.949 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: pgmap v580: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:15:44.033 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:44.126 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:44.231 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:44.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:44.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:15:44.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:15:44.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:15:44.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:15:44.443 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:15:44.510 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:42 smithi148 ceph-mon[32281]: pgmap v581: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T20:15:44.629 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:29 smithi157 ceph-mon[32703]: pgmap v574: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:15:44.645 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:30 smithi157 ceph-mon[32703]: pgmap v575: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T20:15:44.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:32 smithi157 ceph-mon[32703]: pgmap v576: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T20:15:44.768 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:34 smithi157 ceph-mon[32703]: pgmap v577: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T20:15:44.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:36 smithi157 ceph-mon[32703]: pgmap v578: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T20:15:44.883 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:38 smithi157 ceph-mon[32703]: pgmap v579: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:15:44.890 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:38 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:15:44.898 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: pgmap v580: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:15:44.906 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:45.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:45.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:45.148 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:15:45.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:15:45.263 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:15:45.348 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:15:45.357 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:15:45.418 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:15:45.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:42 smithi157 ceph-mon[32703]: pgmap v581: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T20:15:45.505 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:15:45.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:51 smithi131 ceph-mon[27919]: pgmap v555: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:15:45.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:53 smithi131 ceph-mon[27919]: pgmap v556: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:15:45.526 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:55 smithi131 ceph-mon[27919]: pgmap v557: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:15:45.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:57 smithi131 ceph-mon[27919]: pgmap v558: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T20:15:45.541 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:44 smithi148 ceph-mon[32281]: pgmap v582: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T20:15:45.648 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:44 smithi157 ceph-mon[32703]: pgmap v582: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T20:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 -- 172.21.15.131:0/3477637410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc1084b0 msgr2=0x7f0abc10a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:45.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/3477637410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc1084b0 0x7f0abc10a900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009ca0 tx=0x7f0ab402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:15:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 -- 172.21.15.131:0/3477637410 shutdown_connections 2024-04-26T20:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/3477637410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc1084b0 0x7f0abc10a900 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/3477637410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0abc105b80 0x7f0abc107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/3477637410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 0x7f0abc105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 -- 172.21.15.131:0/3477637410 >> 172.21.15.131:0/3477637410 conn(0x7f0abc0fcc60 msgr2=0x7f0abc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 -- 172.21.15.131:0/3477637410 shutdown_connections 2024-04-26T20:15:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.332+0000 7f0ac2e00640 1 -- 172.21.15.131:0/3477637410 wait complete. 2024-04-26T20:15:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.333+0000 7f0ac2e00640 1 Processor -- start 2024-04-26T20:15:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.333+0000 7f0ac2e00640 1 -- start start 2024-04-26T20:15:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 0x7f0abc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac2e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc105b80 0x7f0abc198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0abc1084b0 0x7f0abc19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac2e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10e350 con 0x7f0abc105b80 2024-04-26T20:15:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10e1d0 con 0x7f0abc1084b0 2024-04-26T20:15:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10e4d0 con 0x7f0abc103250 2024-04-26T20:15:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 0x7f0abc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc105b80 0x7f0abc198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 0x7f0abc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43532/0 (socket says 172.21.15.131:43532) 2024-04-26T20:15:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 -- 172.21.15.131:0/701091216 learned_addr learned my addr 172.21.15.131:0/701091216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac2400640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0abc1084b0 0x7f0abc19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 -- 172.21.15.131:0/701091216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0abc1084b0 msgr2=0x7f0abc19ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0abc1084b0 0x7f0abc19ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 -- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc105b80 msgr2=0x7f0abc198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc105b80 0x7f0abc198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1a00640 1 -- 172.21.15.131:0/701091216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc1a05f0 con 0x7f0abc103250 2024-04-26T20:15:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.334+0000 7f0ac1000640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc105b80 0x7f0abc198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.335+0000 7f0ac1a00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 0x7f0abc1989d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000ea60 tx=0x7f0ab000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.335+0000 7f0aba000640 1 -- 172.21.15.131:0/701091216 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000cde0 con 0x7f0abc103250 2024-04-26T20:15:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.335+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc1010a0 con 0x7f0abc103250 2024-04-26T20:15:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.335+0000 7f0aba000640 1 -- 172.21.15.131:0/701091216 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab00035f0 con 0x7f0abc103250 2024-04-26T20:15:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.335+0000 7f0aba000640 1 -- 172.21.15.131:0/701091216 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0010690 con 0x7f0abc103250 2024-04-26T20:15:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.335+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc1015f0 con 0x7f0abc103250 2024-04-26T20:15:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.338+0000 7f0ac71fc740 1 Processor -- start 2024-04-26T20:15:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac71fc740 1 -- start start 2024-04-26T20:15:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0aba000640 1 -- 172.21.15.131:0/701091216 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0ab00037a0 con 0x7f0abc103250 2024-04-26T20:15:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f9142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f914940 0x55f58f916dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f58f873380 con 0x55f58f90f7d0 2024-04-26T20:15:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f58f80fc30 con 0x55f58f914940 2024-04-26T20:15:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f58f86f340 con 0x55f58f90ede0 2024-04-26T20:15:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f9142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43534/0 (socket says 172.21.15.131:43534) 2024-04-26T20:15:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac2400640 1 -- 172.21.15.131:0/1720055418 learned_addr learned my addr 172.21.15.131:0/1720055418 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.339+0000 7f0ac1000640 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f914940 0x55f58f916dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0ac2400640 1 -- 172.21.15.131:0/1720055418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f914940 msgr2=0x55f58f916dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0ac2400640 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f914940 0x55f58f916dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0ac2400640 1 -- 172.21.15.131:0/1720055418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 msgr2=0x55f58f9142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0ac2400640 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f9142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0ac2400640 1 -- 172.21.15.131:0/1720055418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58f831940 con 0x55f58f90ede0 2024-04-26T20:15:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0ac1000640 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f914940 0x55f58f916dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:15:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0ac2400640 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90f200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009ca0 tx=0x7f0ab40093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=230d71ab030e03af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0aaaa00640 1 -- 172.21.15.131:0/1720055418 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4031650 con 0x55f58f90ede0 2024-04-26T20:15:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0aaaa00640 1 -- 172.21.15.131:0/1720055418 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab4036030 con 0x55f58f90ede0 2024-04-26T20:15:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.340+0000 7f0aaaa00640 1 -- 172.21.15.131:0/1720055418 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4004680 con 0x55f58f90ede0 2024-04-26T20:15:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0aba000640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0aa40a3c30 0x7f0aa40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0aba000640 1 -- 172.21.15.131:0/701091216 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0ab00d0c70 con 0x7f0abc103250 2024-04-26T20:15:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac1a00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0aa40a3c30 0x7f0aa40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 -- 172.21.15.131:0/1720055418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 msgr2=0x55f58f90f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90f200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009ca0 tx=0x7f0ab40093c0 comp rx=0 tx=0).stop 2024-04-26T20:15:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 -- 172.21.15.131:0/1720055418 shutdown_connections 2024-04-26T20:15:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f914940 0x55f58f916dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f9142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/1720055418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90f200 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 -- 172.21.15.131:0/1720055418 >> 172.21.15.131:0/1720055418 conn(0x55f58f90a450 msgr2=0x55f58f90a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac71fc740 1 -- 172.21.15.131:0/1720055418 shutdown_connections 2024-04-26T20:15:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.341+0000 7f0ac1a00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0aa40a3c30 0x7f0aa40a60f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55f58f90fbd0 tx=0x7f0ab00d4720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.342+0000 7f0ac71fc740 1 -- 172.21.15.131:0/1720055418 wait complete. 2024-04-26T20:15:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.342+0000 7f0ac71fc740 1 -- start start 2024-04-26T20:15:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.342+0000 7f0ac71fc740 1 Processor -- start 2024-04-26T20:15:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.342+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90ede0 0x55f58f918e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.342+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f919350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f914940 0x55f58f920390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f58f917710 con 0x55f58f90f7d0 2024-04-26T20:15:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f58f7fac60 con 0x55f58f90ede0 2024-04-26T20:15:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f58f917890 con 0x55f58f914940 2024-04-26T20:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac71fc740 10 client.0 _ll_register_callbacks cb 0x55f58f826e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:15:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f919350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f919350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33568/0 (socket says 172.21.15.131:33568) 2024-04-26T20:15:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 -- 172.21.15.131:0/95319483 learned_addr learned my addr 172.21.15.131:0/95319483 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac2400640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90ede0 0x55f58f918e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac2400640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f914940 0x55f58f920390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 -- 172.21.15.131:0/95319483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f914940 msgr2=0x55f58f920390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f914940 0x55f58f920390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 -- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90ede0 msgr2=0x55f58f918e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90ede0 0x55f58f918e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac1000640 1 -- 172.21.15.131:0/95319483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58f809510 con 0x55f58f90f7d0 2024-04-26T20:15:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.343+0000 7f0ac2400640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f914940 0x55f58f920390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:15:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0ac1000640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f919350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009f20 tx=0x7f0aac00b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0caead230f0f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0a9f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:15:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0a9f400640 1 -- 172.21.15.131:0/95319483 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac01f030 con 0x55f58f90f7d0 2024-04-26T20:15:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0a9f400640 1 -- 172.21.15.131:0/95319483 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0aac005020 con 0x55f58f90f7d0 2024-04-26T20:15:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab4036030 con 0x55f58f90f7d0 2024-04-26T20:15:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0a9f400640 1 -- 172.21.15.131:0/95319483 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0aac029050 con 0x55f58f90f7d0 2024-04-26T20:15:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0ac71fc740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-04-26T20:15:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f58f585210 con 0x55f58f90f7d0 2024-04-26T20:15:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.344+0000 7f0a9f400640 1 -- 172.21.15.131:0/95319483 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab4036030 con 0x55f58f90f7d0 2024-04-26T20:15:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0a9f400640 1 -- 172.21.15.131:0/95319483 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0aac01b680 con 0x55f58f90f7d0 2024-04-26T20:15:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0ac71fc740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:15:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0ac71fc740 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:15:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0ac71fc740 20 client.15498 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f924960 0x55f58f926dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0ac71fc740 4 client.15498 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f58f880f90 con 0x55f58f924960 2024-04-26T20:15:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.345+0000 7f0ac2400640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f924960 0x55f58f926dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.346+0000 7f0ac2400640 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f924960 0x55f58f926dc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b354c98239b91bd server_cookie=1fa9653555c20418 in_seq=0 out_seq=0 2024-04-26T20:15:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.346+0000 7f0a9f400640 10 client.15498 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0a9f400640 1 -- 172.21.15.131:0/95319483 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f58f880f90 con 0x55f58f924960 2024-04-26T20:15:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0a9f400640 10 client.15498 handle_command_reply: tid=0 2024-04-26T20:15:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 client.15498 shutdown 2024-04-26T20:15:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 msgr2=0x55f58f919350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f919350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009f20 tx=0x7f0aac00b750 comp rx=0 tx=0).stop 2024-04-26T20:15:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 shutdown_connections 2024-04-26T20:15:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f924960 0x55f58f926dc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f914940 0x55f58f920390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f90f7d0 0x55f58f919350 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0a9f400640 0 client.15498 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/95319483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90ede0 0x55f58f918e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 >> 172.21.15.131:0/95319483 conn(0x55f58f90a450 msgr2=0x55f58f909650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 shutdown_connections 2024-04-26T20:15:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 1 -- 172.21.15.131:0/95319483 wait complete. 2024-04-26T20:15:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.347+0000 7f0ac71fc740 20 client.15498 trim_cache size 0 max 16384 2024-04-26T20:15:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.363+0000 7f0ac71fc740 1 Processor -- start 2024-04-26T20:15:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.363+0000 7f0ac71fc740 1 -- start start 2024-04-26T20:15:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f920350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f910970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1abb10 con 0x55f58f914940 2024-04-26T20:15:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a93e0 con 0x55f58f90f7d0 2024-04-26T20:15:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a7230 con 0x55f58f90ede0 2024-04-26T20:15:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f910970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f910970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33576/0 (socket says 172.21.15.131:33576) 2024-04-26T20:15:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac2400640 1 -- 172.21.15.131:0/2378608232 learned_addr learned my addr 172.21.15.131:0/2378608232 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac1000640 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f920350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.364+0000 7f0ac2400640 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0ac1000640 1 -- 172.21.15.131:0/2378608232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 msgr2=0x55f58f90a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0ac1000640 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0ac1000640 1 -- 172.21.15.131:0/2378608232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 msgr2=0x55f58f910970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0ac1000640 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f910970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0ac1000640 1 -- 172.21.15.131:0/2378608232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58f651620 con 0x55f58f90f7d0 2024-04-26T20:15:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0ac2400640 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f910970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0ac1000640 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f920350 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0aac01baf0 tx=0x7f0aac0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322bd20a6d26cd4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0aaa000640 1 -- 172.21.15.131:0/2378608232 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac01f040 con 0x55f58f90f7d0 2024-04-26T20:15:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0aaa000640 1 -- 172.21.15.131:0/2378608232 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aac005460 con 0x55f58f90f7d0 2024-04-26T20:15:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.365+0000 7f0aaa000640 1 -- 172.21.15.131:0/2378608232 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac000e60 con 0x55f58f90f7d0 2024-04-26T20:15:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 -- 172.21.15.131:0/2378608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 msgr2=0x55f58f920350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f920350 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0aac01baf0 tx=0x7f0aac0092d0 comp rx=0 tx=0).stop 2024-04-26T20:15:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 -- 172.21.15.131:0/2378608232 shutdown_connections 2024-04-26T20:15:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f910970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f920350 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/2378608232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f90a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 -- 172.21.15.131:0/2378608232 >> 172.21.15.131:0/2378608232 conn(0x55f58f90a450 msgr2=0x55f58f913fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 -- 172.21.15.131:0/2378608232 shutdown_connections 2024-04-26T20:15:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.366+0000 7f0ac71fc740 1 -- 172.21.15.131:0/2378608232 wait complete. 2024-04-26T20:15:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 -- start start 2024-04-26T20:15:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 Processor -- start 2024-04-26T20:15:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f91d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f91d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f988f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a78a0 con 0x55f58f914940 2024-04-26T20:15:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a8d10 con 0x55f58f90f7d0 2024-04-26T20:15:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a7f70 con 0x55f58f90ede0 2024-04-26T20:15:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.367+0000 7f0ac71fc740 10 client.0 _ll_register_callbacks cb 0x7f0aac009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:15:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f988f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f988f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33580/0 (socket says 172.21.15.131:33580) 2024-04-26T20:15:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 -- 172.21.15.131:0/535372780 learned_addr learned my addr 172.21.15.131:0/535372780 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:15:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac1000640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f91d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f91d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 -- 172.21.15.131:0/535372780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 msgr2=0x55f58f91d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f91d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 -- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 msgr2=0x55f58f91d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:15:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f91d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 -- 172.21.15.131:0/535372780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58f82dcb0 con 0x55f58f914940 2024-04-26T20:15:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac1000640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f91d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:15:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0ac2400640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f988f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402aeb0 tx=0x7f0ab40093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3564cb06032ef9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:15:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.368+0000 7f0a9f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:15:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0a9f400640 1 -- 172.21.15.131:0/535372780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4060060 con 0x55f58f914940 2024-04-26T20:15:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0a9f400640 1 -- 172.21.15.131:0/535372780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0ab4003970 con 0x55f58f914940 2024-04-26T20:15:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aac005460 con 0x55f58f914940 2024-04-26T20:15:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0a9f400640 1 -- 172.21.15.131:0/535372780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ab4004b40 con 0x55f58f914940 2024-04-26T20:15:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0ac71fc740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-04-26T20:15:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f58f98a700 con 0x55f58f914940 2024-04-26T20:15:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0a9f400640 1 -- 172.21.15.131:0/535372780 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aac005460 con 0x55f58f914940 2024-04-26T20:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.369+0000 7f0a9f400640 1 -- 172.21.15.131:0/535372780 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ab4050040 con 0x55f58f914940 2024-04-26T20:15:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac71fc740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:15:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac71fc740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:15:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac71fc740 20 client.15504 populate_metadata read hostname 'smithi131' 2024-04-26T20:15:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f98ae30 0x55f58f98d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:15:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac71fc740 4 client.15504 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:15:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f58f87ae00 con 0x55f58f98ae30 2024-04-26T20:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac1000640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f98ae30 0x55f58f98d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:15:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0ac1000640 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f98ae30 0x55f58f98d250 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649099789056b5bb server_cookie=d81d8d981803241b in_seq=0 out_seq=0 2024-04-26T20:15:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.370+0000 7f0a9f400640 10 client.15504 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0a9f400640 1 -- 172.21.15.131:0/535372780 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f58f87ae00 con 0x55f58f98ae30 2024-04-26T20:15:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0a9f400640 10 client.15504 handle_command_reply: tid=0 2024-04-26T20:15:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 client.15504 shutdown 2024-04-26T20:15:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 msgr2=0x55f58f988f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f988f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402aeb0 tx=0x7f0ab40093c0 comp rx=0 tx=0).stop 2024-04-26T20:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 shutdown_connections 2024-04-26T20:15:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f58f98ae30 0x55f58f98d250 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f58f914940 0x55f58f988f30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f58f90f7d0 0x55f58f91d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0a9f400640 0 client.15504 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:15:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 --2- 172.21.15.131:0/535372780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f58f90ede0 0x55f58f91d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 >> 172.21.15.131:0/535372780 conn(0x55f58f90a450 msgr2=0x55f58f910970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 shutdown_connections 2024-04-26T20:15:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 1 -- 172.21.15.131:0/535372780 wait complete. 2024-04-26T20:15:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.379+0000 7f0ac71fc740 20 client.15504 trim_cache size 0 max 16384 2024-04-26T20:15:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:15:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:15:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:15:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0aa40a3c30 msgr2=0x7f0aa40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0aa40a3c30 0x7f0aa40a60f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55f58f90fbd0 tx=0x7f0ab00d4720 comp rx=0 tx=0).stop 2024-04-26T20:15:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 msgr2=0x7f0abc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:15:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 0x7f0abc1989d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000ea60 tx=0x7f0ab000ef20 comp rx=0 tx=0).stop 2024-04-26T20:15:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 shutdown_connections 2024-04-26T20:15:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0abc1084b0 0x7f0abc19ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc105b80 0x7f0abc198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:15:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0aa40a3c30 0x7f0aa40a60f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 --2- 172.21.15.131:0/701091216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0abc103250 0x7f0abc1989d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 >> 172.21.15.131:0/701091216 conn(0x7f0abc0fcc60 msgr2=0x7f0abc103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 shutdown_connections 2024-04-26T20:16:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:14:59.380+0000 7f0ac2e00640 1 -- 172.21.15.131:0/701091216 wait complete. 2024-04-26T20:16:00.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:46 smithi148 ceph-mon[32281]: pgmap v583: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T20:16:00.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:48 smithi148 ceph-mon[32281]: pgmap v584: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T20:16:00.474 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:50 smithi148 ceph-mon[32281]: pgmap v585: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T20:16:00.560 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:52 smithi148 ceph-mon[32281]: pgmap v586: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T20:16:00.617 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:54 smithi148 ceph-mon[32281]: pgmap v587: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T20:16:00.675 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:56 smithi148 ceph-mon[32281]: pgmap v588: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:00.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:15:58 smithi148 ceph-mon[32281]: pgmap v589: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:00.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:46 smithi157 ceph-mon[32703]: pgmap v583: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T20:16:00.764 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:48 smithi157 ceph-mon[32703]: pgmap v584: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T20:16:00.827 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:50 smithi157 ceph-mon[32703]: pgmap v585: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T20:16:00.833 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:52 smithi157 ceph-mon[32703]: pgmap v586: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T20:16:00.840 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:54 smithi157 ceph-mon[32703]: pgmap v587: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T20:16:00.847 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:56 smithi157 ceph-mon[32703]: pgmap v588: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:00.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:15:58 smithi157 ceph-mon[32703]: pgmap v589: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:00.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:14:59 smithi131 ceph-mon[27919]: pgmap v559: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T20:16:00.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:01 smithi131 ceph-mon[27919]: pgmap v560: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T20:16:01.168 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:03 smithi131 ceph-mon[27919]: pgmap v561: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:16:01.333 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:05 smithi131 ceph-mon[27919]: pgmap v562: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-26T20:16:01.442 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:07 smithi131 ceph-mon[27919]: pgmap v563: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:16:01.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:09 smithi131 ceph-mon[27919]: pgmap v564: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T20:16:01.573 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:11 smithi131 ceph-mon[27919]: pgmap v565: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T20:16:01.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:00 smithi148 ceph-mon[32281]: pgmap v590: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:01.657 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:00 smithi157 ceph-mon[32703]: pgmap v590: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 -- 172.21.15.131:0/65125891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 msgr2=0x7f925c111220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 --2- 172.21.15.131:0/65125891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 0x7f925c111220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9254009be0 tx=0x7f925402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:16:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 -- 172.21.15.131:0/65125891 shutdown_connections 2024-04-26T20:16:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 --2- 172.21.15.131:0/65125891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 0x7f925c111220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 --2- 172.21.15.131:0/65125891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f925c109fd0 0x7f925c10a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 --2- 172.21.15.131:0/65125891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 0x7f925c109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 -- 172.21.15.131:0/65125891 >> 172.21.15.131:0/65125891 conn(0x7f925c077dc0 msgr2=0x7f925c07a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 -- 172.21.15.131:0/65125891 shutdown_connections 2024-04-26T20:16:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.521+0000 7f9262200640 1 -- 172.21.15.131:0/65125891 wait complete. 2024-04-26T20:16:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.522+0000 7f9262200640 1 Processor -- start 2024-04-26T20:16:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9262200640 1 -- start start 2024-04-26T20:16:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9262200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 0x7f925c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9262200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f925c109fd0 0x7f925c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9262200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 0x7f925c1a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9262200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1148f0 con 0x7f925c108de0 2024-04-26T20:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9262200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f925c114770 con 0x7f925c109fd0 2024-04-26T20:16:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9262200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f925c114a70 con 0x7f925c10a990 2024-04-26T20:16:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9260e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 0x7f925c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9260e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 0x7f925c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51616/0 (socket says 172.21.15.131:51616) 2024-04-26T20:16:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9260e00640 1 -- 172.21.15.131:0/684320756 learned_addr learned my addr 172.21.15.131:0/684320756 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9261800640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 0x7f925c1a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.523+0000 7f9253e00640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f925c109fd0 0x7f925c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9260e00640 1 -- 172.21.15.131:0/684320756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 msgr2=0x7f925c1a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9260e00640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 0x7f925c1a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9260e00640 1 -- 172.21.15.131:0/684320756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f925c109fd0 msgr2=0x7f925c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9260e00640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f925c109fd0 0x7f925c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9260e00640 1 -- 172.21.15.131:0/684320756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c1a6b80 con 0x7f925c108de0 2024-04-26T20:16:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9253e00640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f925c109fd0 0x7f925c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9260e00640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 0x7f925c19ef60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9c0 tx=0x7f924c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f925a000640 1 -- 172.21.15.131:0/684320756 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c00cd80 con 0x7f925c108de0 2024-04-26T20:16:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f925a000640 1 -- 172.21.15.131:0/684320756 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924c0035f0 con 0x7f925c108de0 2024-04-26T20:16:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c1a6e70 con 0x7f925c108de0 2024-04-26T20:16:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.524+0000 7f925a000640 1 -- 172.21.15.131:0/684320756 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c010690 con 0x7f925c108de0 2024-04-26T20:16:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.525+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c1a72d0 con 0x7f925c108de0 2024-04-26T20:16:03.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.527+0000 7f92667e1740 1 Processor -- start 2024-04-26T20:16:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f92667e1740 1 -- start start 2024-04-26T20:16:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f925a000640 1 -- 172.21.15.131:0/684320756 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f924c0037a0 con 0x7f925c108de0 2024-04-26T20:16:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f92667e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f92667e1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f0f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f92667e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f0f940 0x556904f11dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f92667e1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556904e6e380 con 0x556904f0a7d0 2024-04-26T20:16:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f92667e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556904e0ac30 con 0x556904f09de0 2024-04-26T20:16:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f92667e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556904e6a340 con 0x556904f0f940 2024-04-26T20:16:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f9260e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f0f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f9260e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f0f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51620/0 (socket says 172.21.15.131:51620) 2024-04-26T20:16:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f9260e00640 1 -- 172.21.15.131:0/995191272 learned_addr learned my addr 172.21.15.131:0/995191272 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f9253e00640 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f0f940 0x556904f11dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.528+0000 7f9261800640 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9253e00640 1 -- 172.21.15.131:0/995191272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 msgr2=0x556904f0a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9253e00640 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f0a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9253e00640 1 -- 172.21.15.131:0/995191272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 msgr2=0x556904f0f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9253e00640 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f0f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9253e00640 1 -- 172.21.15.131:0/995191272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556904e2c940 con 0x556904f0f940 2024-04-26T20:16:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9261800640 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f0a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:16:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9260e00640 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f0f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9253e00640 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f0f940 0x556904f11dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9244009530 tx=0x7f92440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5aadf92f40160e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9252000640 1 -- 172.21.15.131:0/995191272 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244004020 con 0x556904f0f940 2024-04-26T20:16:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9252000640 1 -- 172.21.15.131:0/995191272 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92440041c0 con 0x556904f0f940 2024-04-26T20:16:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.529+0000 7f9252000640 1 -- 172.21.15.131:0/995191272 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924400f850 con 0x556904f0f940 2024-04-26T20:16:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f925a000640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92400a3c30 0x7f92400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f925a000640 1 -- 172.21.15.131:0/684320756 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f924c0d0d40 con 0x7f925c108de0 2024-04-26T20:16:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f9261800640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92400a3c30 0x7f92400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 -- 172.21.15.131:0/995191272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f0f940 msgr2=0x556904f11dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f0f940 0x556904f11dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9244009530 tx=0x7f92440118d0 comp rx=0 tx=0).stop 2024-04-26T20:16:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 -- 172.21.15.131:0/995191272 shutdown_connections 2024-04-26T20:16:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f0f940 0x556904f11dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f0f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 --2- 172.21.15.131:0/995191272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f0a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 -- 172.21.15.131:0/995191272 >> 172.21.15.131:0/995191272 conn(0x556904f05450 msgr2=0x556904f05850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.530+0000 7f92667e1740 1 -- 172.21.15.131:0/995191272 shutdown_connections 2024-04-26T20:16:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.531+0000 7f92667e1740 1 -- 172.21.15.131:0/995191272 wait complete. 2024-04-26T20:16:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.531+0000 7f9261800640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92400a3c30 0x7f92400a60f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9254006010 tx=0x7f9254030970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.531+0000 7f92667e1740 1 -- start start 2024-04-26T20:16:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.531+0000 7f92667e1740 1 Processor -- start 2024-04-26T20:16:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f92667e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f13f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f92667e1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f92667e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f0f940 0x556904f1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f92667e1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556904f128f0 con 0x556904f0a7d0 2024-04-26T20:16:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f92667e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556904df5c60 con 0x556904f0f940 2024-04-26T20:16:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f92667e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556904f12a70 con 0x556904f09de0 2024-04-26T20:16:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f92667e1740 10 client.0 _ll_register_callbacks cb 0x556904e21e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:16:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9253e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f13f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f13f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40352/0 (socket says 172.21.15.131:40352) 2024-04-26T20:16:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 -- 172.21.15.131:0/3131952394 learned_addr learned my addr 172.21.15.131:0/3131952394 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9253e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f144b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51628/0 (socket says 172.21.15.131:51628) 2024-04-26T20:16:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9261800640 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f0f940 0x556904f1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 -- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f0f940 msgr2=0x556904f1b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f0f940 0x556904f1b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 -- 172.21.15.131:0/3131952394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 msgr2=0x556904f144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f144b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.532+0000 7f9260e00640 1 -- 172.21.15.131:0/3131952394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556904e04510 con 0x556904f09de0 2024-04-26T20:16:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f9260e00640 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f13f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f924c09ca00 tx=0x7f924c0985f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cada33de4baab76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f924aa00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:16:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f924aa00640 1 -- 172.21.15.131:0/3131952394 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c09c5b0 con 0x556904f09de0 2024-04-26T20:16:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f924aa00640 1 -- 172.21.15.131:0/3131952394 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f924c0ce4f0 con 0x556904f09de0 2024-04-26T20:16:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92440041c0 con 0x556904f09de0 2024-04-26T20:16:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f924aa00640 1 -- 172.21.15.131:0/3131952394 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f924c09c3d0 con 0x556904f09de0 2024-04-26T20:16:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f92667e1740 10 client.25630 fetch_fsmap learned FSMap version 27 2024-04-26T20:16:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556904b80210 con 0x556904f09de0 2024-04-26T20:16:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.533+0000 7f924aa00640 1 -- 172.21.15.131:0/3131952394 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92440041c0 con 0x556904f09de0 2024-04-26T20:16:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f924aa00640 1 -- 172.21.15.131:0/3131952394 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f924c09ccd0 con 0x556904f09de0 2024-04-26T20:16:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f92667e1740 10 client.25630 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:16:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f92667e1740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f92667e1740 20 client.25630 populate_metadata read hostname 'smithi131' 2024-04-26T20:16:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f92667e1740 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f1fb10 0x556904f21f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f92667e1740 4 client.25630 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:16:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556904e7bf90 con 0x556904f1fb10 2024-04-26T20:16:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.534+0000 7f9253e00640 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f1fb10 0x556904f21f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.535+0000 7f9253e00640 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f1fb10 0x556904f21f60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7cf4530b25bc97 server_cookie=e16dc91557a59870 in_seq=0 out_seq=0 2024-04-26T20:16:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.535+0000 7f924aa00640 10 client.25630 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.535+0000 7f924aa00640 1 -- 172.21.15.131:0/3131952394 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556904e7bf90 con 0x556904f1fb10 2024-04-26T20:16:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f924aa00640 10 client.25630 handle_command_reply: tid=0 2024-04-26T20:16:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 client.25630 shutdown 2024-04-26T20:16:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 msgr2=0x556904f13f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f13f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f924c09ca00 tx=0x7f924c0985f0 comp rx=0 tx=0).stop 2024-04-26T20:16:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 shutdown_connections 2024-04-26T20:16:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f1fb10 0x556904f21f60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f0f940 0x556904f1b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f924aa00640 0 client.25630 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 --2- 172.21.15.131:0/3131952394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f13f70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 >> 172.21.15.131:0/3131952394 conn(0x556904f05450 msgr2=0x556904f04650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 shutdown_connections 2024-04-26T20:16:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 1 -- 172.21.15.131:0/3131952394 wait complete. 2024-04-26T20:16:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.536+0000 7f92667e1740 20 client.25630 trim_cache size 0 max 16384 2024-04-26T20:16:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.552+0000 7f92667e1740 1 Processor -- start 2024-04-26T20:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.552+0000 7f92667e1740 1 -- start start 2024-04-26T20:16:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.552+0000 7f92667e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f05450 0x556904f08b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f92667e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f1b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f92667e1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f27870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f92667e1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1b42e0 con 0x556904f0a7d0 2024-04-26T20:16:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f92667e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1b1bb0 con 0x556904f09de0 2024-04-26T20:16:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f92667e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1afa00 con 0x556904f05450 2024-04-26T20:16:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f27870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9260e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f05450 0x556904f08b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f27870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51630/0 (socket says 172.21.15.131:51630) 2024-04-26T20:16:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 -- 172.21.15.131:0/1076017479 learned_addr learned my addr 172.21.15.131:0/1076017479 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9253e00640 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f1b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 -- 172.21.15.131:0/1076017479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f05450 msgr2=0x556904f08b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f05450 0x556904f08b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 -- 172.21.15.131:0/1076017479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 msgr2=0x556904f1b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f1b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9261800640 1 -- 172.21.15.131:0/1076017479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556904c4c620 con 0x556904f0a7d0 2024-04-26T20:16:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9260e00640 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f05450 0x556904f08b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.553+0000 7f9253e00640 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f1b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.554+0000 7f9261800640 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f27870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9254009be0 tx=0x7f925403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cebefa67802676 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.554+0000 7f9251600640 1 -- 172.21.15.131:0/1076017479 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254036040 con 0x556904f0a7d0 2024-04-26T20:16:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.554+0000 7f9251600640 1 -- 172.21.15.131:0/1076017479 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9254003d70 con 0x556904f0a7d0 2024-04-26T20:16:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.554+0000 7f9251600640 1 -- 172.21.15.131:0/1076017479 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540318a0 con 0x556904f0a7d0 2024-04-26T20:16:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 -- 172.21.15.131:0/1076017479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 msgr2=0x556904f27870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f27870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9254009be0 tx=0x7f925403f000 comp rx=0 tx=0).stop 2024-04-26T20:16:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 -- 172.21.15.131:0/1076017479 shutdown_connections 2024-04-26T20:16:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f27870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f09de0 0x556904f1b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 --2- 172.21.15.131:0/1076017479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f05450 0x556904f08b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 -- 172.21.15.131:0/1076017479 >> 172.21.15.131:0/1076017479 conn(0x556904f0b230 msgr2=0x556904f08400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 -- 172.21.15.131:0/1076017479 shutdown_connections 2024-04-26T20:16:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 -- 172.21.15.131:0/1076017479 wait complete. 2024-04-26T20:16:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 -- start start 2024-04-26T20:16:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.555+0000 7f92667e1740 1 Processor -- start 2024-04-26T20:16:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f92667e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f05450 0x556904f18cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f92667e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f19200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f92667e1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f1a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f92667e1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1b0070 con 0x556904f0a7d0 2024-04-26T20:16:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f92667e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1b14e0 con 0x556904f05450 2024-04-26T20:16:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f92667e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1b0740 con 0x556904f09de0 2024-04-26T20:16:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f92667e1740 10 client.0 _ll_register_callbacks cb 0x7f924c0d44e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:16:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f9261800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f1a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f9260e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f05450 0x556904f18cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f9260e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f05450 0x556904f18cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:35462/0 (socket says 172.21.15.131:35462) 2024-04-26T20:16:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f9261800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f1a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51632/0 (socket says 172.21.15.131:51632) 2024-04-26T20:16:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f9260e00640 1 -- 172.21.15.131:0/2473207697 learned_addr learned my addr 172.21.15.131:0/2473207697 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.556+0000 7f9253e00640 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f19200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f9261800640 1 -- 172.21.15.131:0/2473207697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 msgr2=0x556904f19200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f9261800640 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f19200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f9261800640 1 -- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f05450 msgr2=0x556904f18cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f9261800640 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f05450 0x556904f18cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f9261800640 1 -- 172.21.15.131:0/2473207697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556904e28cb0 con 0x556904f0a7d0 2024-04-26T20:16:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f9260e00640 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f05450 0x556904f18cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f9261800640 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f1a1c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9254034040 tx=0x7f9254009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d7a17c11f9da2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f924aa00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:16:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f924aa00640 1 -- 172.21.15.131:0/2473207697 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254062060 con 0x556904f0a7d0 2024-04-26T20:16:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f924aa00640 1 -- 172.21.15.131:0/2473207697 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9254003b70 con 0x556904f0a7d0 2024-04-26T20:16:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9254003d70 con 0x556904f0a7d0 2024-04-26T20:16:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f924aa00640 1 -- 172.21.15.131:0/2473207697 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9254069050 con 0x556904f0a7d0 2024-04-26T20:16:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f92667e1740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-04-26T20:16:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.557+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556904f860e0 con 0x556904f0a7d0 2024-04-26T20:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f924aa00640 1 -- 172.21.15.131:0/2473207697 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9254003d70 con 0x556904f0a7d0 2024-04-26T20:16:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f924aa00640 1 -- 172.21.15.131:0/2473207697 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9254031c00 con 0x556904f0a7d0 2024-04-26T20:16:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f92667e1740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:16:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f92667e1740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:16:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f92667e1740 20 client.15522 populate_metadata read hostname 'smithi131' 2024-04-26T20:16:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f92667e1740 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f0f940 0x556904f88a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f92667e1740 4 client.15522 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:16:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556904e75e00 con 0x556904f0f940 2024-04-26T20:16:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.558+0000 7f9260e00640 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f0f940 0x556904f88a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.559+0000 7f9260e00640 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f0f940 0x556904f88a00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca6f3c306ff797a8 server_cookie=f14dd09e1242f153 in_seq=0 out_seq=0 2024-04-26T20:16:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.559+0000 7f924aa00640 10 client.15522 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.562+0000 7f924aa00640 1 -- 172.21.15.131:0/2473207697 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556904e75e00 con 0x556904f0f940 2024-04-26T20:16:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.562+0000 7f924aa00640 10 client.15522 handle_command_reply: tid=0 2024-04-26T20:16:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.562+0000 7f92667e1740 1 client.15522 shutdown 2024-04-26T20:16:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 msgr2=0x556904f1a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f1a1c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9254034040 tx=0x7f9254009510 comp rx=0 tx=0).stop 2024-04-26T20:16:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 shutdown_connections 2024-04-26T20:16:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556904f0f940 0x556904f88a00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556904f0a7d0 0x556904f1a1c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556904f09de0 0x556904f19200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 --2- 172.21.15.131:0/2473207697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556904f05450 0x556904f18cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f924aa00640 0 client.15522 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 >> 172.21.15.131:0/2473207697 conn(0x556904f0b230 msgr2=0x556904f13af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 shutdown_connections 2024-04-26T20:16:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 1 -- 172.21.15.131:0/2473207697 wait complete. 2024-04-26T20:16:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.563+0000 7f92667e1740 20 client.15522 trim_cache size 0 max 16384 2024-04-26T20:16:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:16:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:16:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:16:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:16:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92400a3c30 msgr2=0x7f92400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92400a3c30 0x7f92400a60f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9254006010 tx=0x7f9254030970 comp rx=0 tx=0).stop 2024-04-26T20:16:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 msgr2=0x7f925c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 0x7f925c19ef60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9c0 tx=0x7f924c00ee80 comp rx=0 tx=0).stop 2024-04-26T20:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 shutdown_connections 2024-04-26T20:16:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f92400a3c30 0x7f92400a60f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925c10a990 0x7f925c1a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f925c109fd0 0x7f925c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 --2- 172.21.15.131:0/684320756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f925c108de0 0x7f925c19ef60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 >> 172.21.15.131:0/684320756 conn(0x7f925c077dc0 msgr2=0x7f925c10d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 shutdown_connections 2024-04-26T20:16:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:12.564+0000 7f9262200640 1 -- 172.21.15.131:0/684320756 wait complete. 2024-04-26T20:16:18.239 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:13 smithi131 ceph-mon[27919]: pgmap v566: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-26T20:16:18.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:15 smithi131 ceph-mon[27919]: pgmap v567: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-26T20:16:18.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:17 smithi131 ceph-mon[27919]: pgmap v568: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:18.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:19 smithi131 ceph-mon[27919]: pgmap v569: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:18.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:21 smithi131 ceph-mon[27919]: pgmap v570: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:18.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:23 smithi131 ceph-mon[27919]: pgmap v571: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:18.814 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:01 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1005451432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:16:18.821 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:02 smithi148 ceph-mon[32281]: pgmap v591: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:18.827 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:04 smithi148 ceph-mon[32281]: pgmap v592: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:18.835 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:06 smithi148 ceph-mon[32281]: pgmap v593: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:16:18.842 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:08 smithi148 ceph-mon[32281]: pgmap v594: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:16:18.849 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:10 smithi148 ceph-mon[32281]: pgmap v595: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:18.857 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:12 smithi148 ceph-mon[32281]: pgmap v596: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:18.865 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:14 smithi148 ceph-mon[32281]: pgmap v597: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:19.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:17 smithi148 ceph-mon[32281]: pgmap v598: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:19.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:01 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1005451432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:16:19.295 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:02 smithi157 ceph-mon[32703]: pgmap v591: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:19.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:04 smithi157 ceph-mon[32703]: pgmap v592: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:19.309 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:06 smithi157 ceph-mon[32703]: pgmap v593: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:16:19.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:08 smithi157 ceph-mon[32703]: pgmap v594: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:16:19.451 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:10 smithi157 ceph-mon[32703]: pgmap v595: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:19.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:12 smithi157 ceph-mon[32703]: pgmap v596: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:19.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:14 smithi157 ceph-mon[32703]: pgmap v597: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:19.530 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:17 smithi157 ceph-mon[32703]: pgmap v598: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:19.609 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:19 smithi157 ceph-mon[32703]: pgmap v599: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:19.617 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:19 smithi148 ceph-mon[32281]: pgmap v599: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 -- 172.21.15.131:0/2632714938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8109fd0 msgr2=0x7f03d810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 --2- 172.21.15.131:0/2632714938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8109fd0 0x7f03d810a450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009900 tx=0x7f03c002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:16:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 -- 172.21.15.131:0/2632714938 shutdown_connections 2024-04-26T20:16:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 --2- 172.21.15.131:0/2632714938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d810a990 0x7f03d8111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 --2- 172.21.15.131:0/2632714938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8109fd0 0x7f03d810a450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 --2- 172.21.15.131:0/2632714938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8108de0 0x7f03d8109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 -- 172.21.15.131:0/2632714938 >> 172.21.15.131:0/2632714938 conn(0x7f03d8077dc0 msgr2=0x7f03d807a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 -- 172.21.15.131:0/2632714938 shutdown_connections 2024-04-26T20:16:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.707+0000 7f03dd400640 1 -- 172.21.15.131:0/2632714938 wait complete. 2024-04-26T20:16:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.708+0000 7f03dd400640 1 Processor -- start 2024-04-26T20:16:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.708+0000 7f03dd400640 1 -- start start 2024-04-26T20:16:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03dd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8108de0 0x7f03d819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03dd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8109fd0 0x7f03d819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03dd400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d810a990 0x7f03d81a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03dd400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8114960 con 0x7f03d8109fd0 2024-04-26T20:16:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03dd400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81147e0 con 0x7f03d810a990 2024-04-26T20:16:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03dd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8114ae0 con 0x7f03d8108de0 2024-04-26T20:16:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8109fd0 0x7f03d819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03d7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8108de0 0x7f03d819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8109fd0 0x7f03d819f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40414/0 (socket says 172.21.15.131:40414) 2024-04-26T20:16:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03d7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8108de0 0x7f03d819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45568/0 (socket says 172.21.15.131:45568) 2024-04-26T20:16:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03cf400640 1 -- 172.21.15.131:0/1521172118 learned_addr learned my addr 172.21.15.131:0/1521172118 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03dca00640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d810a990 0x7f03d81a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03d7e00640 1 -- 172.21.15.131:0/1521172118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d810a990 msgr2=0x7f03d81a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03d7e00640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d810a990 0x7f03d81a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03d7e00640 1 -- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8109fd0 msgr2=0x7f03d819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03d7e00640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8109fd0 0x7f03d819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03d7e00640 1 -- 172.21.15.131:0/1521172118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d81a6c00 con 0x7f03d8108de0 2024-04-26T20:16:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.709+0000 7f03cf400640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8109fd0 0x7f03d819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.710+0000 7f03d7e00640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8108de0 0x7f03d819efe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03c800e9c0 tx=0x7f03c800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.710+0000 7f03d5600640 1 -- 172.21.15.131:0/1521172118 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c800ccc0 con 0x7f03d8108de0 2024-04-26T20:16:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.710+0000 7f03d5600640 1 -- 172.21.15.131:0/1521172118 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c80035f0 con 0x7f03d8108de0 2024-04-26T20:16:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.710+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d81a6ef0 con 0x7f03d8108de0 2024-04-26T20:16:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.710+0000 7f03d5600640 1 -- 172.21.15.131:0/1521172118 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c800ce60 con 0x7f03d8108de0 2024-04-26T20:16:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.710+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d81a7350 con 0x7f03d8108de0 2024-04-26T20:16:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.714+0000 7f03e19a9740 1 Processor -- start 2024-04-26T20:16:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03e19a9740 1 -- start start 2024-04-26T20:16:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03d5600640 1 -- 172.21.15.131:0/1521172118 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f03c8004000 con 0x7f03d8108de0 2024-04-26T20:16:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03e19a9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4a61380 con 0x55dbe4b02940 2024-04-26T20:16:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03e19a9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dbe49fdc30 con 0x55dbe4afcde0 2024-04-26T20:16:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03e19a9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4a5d340 con 0x55dbe4afd7d0 2024-04-26T20:16:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03dca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03dca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:46026/0 (socket says 172.21.15.131:46026) 2024-04-26T20:16:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03dca00640 1 -- 172.21.15.131:0/2066399781 learned_addr learned my addr 172.21.15.131:0/2066399781 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03d7e00640 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.715+0000 7f03cf400640 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03dca00640 1 -- 172.21.15.131:0/2066399781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 msgr2=0x55dbe4b022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03dca00640 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03dca00640 1 -- 172.21.15.131:0/2066399781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 msgr2=0x55dbe4b04dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03dca00640 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b04dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03dca00640 1 -- 172.21.15.131:0/2066399781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe4a1f940 con 0x55dbe4afcde0 2024-04-26T20:16:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03d7e00640 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03dca00640 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afd200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03d0010370 tx=0x7f03d0010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc4af6bfc9f3686d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.716+0000 7f03cd200640 1 -- 172.21.15.131:0/2066399781 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0016070 con 0x55dbe4afcde0 2024-04-26T20:16:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.717+0000 7f03d5600640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03bc0a3c30 0x7f03bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.717+0000 7f03cd200640 1 -- 172.21.15.131:0/2066399781 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d0004d20 con 0x55dbe4afcde0 2024-04-26T20:16:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.717+0000 7f03cd200640 1 -- 172.21.15.131:0/2066399781 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0005420 con 0x55dbe4afcde0 2024-04-26T20:16:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.717+0000 7f03d5600640 1 -- 172.21.15.131:0/1521172118 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f03c80d0de0 con 0x7f03d8108de0 2024-04-26T20:16:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.717+0000 7f03d7e00640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03bc0a3c30 0x7f03bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.717+0000 7f03e19a9740 1 -- 172.21.15.131:0/2066399781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 msgr2=0x55dbe4afd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.717+0000 7f03e19a9740 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afd200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03d0010370 tx=0x7f03d0010830 comp rx=0 tx=0).stop 2024-04-26T20:16:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03d7e00640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03bc0a3c30 0x7f03bc0a60f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55dbe4afdbd0 tx=0x7f03c80d4890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 -- 172.21.15.131:0/2066399781 shutdown_connections 2024-04-26T20:16:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 --2- 172.21.15.131:0/2066399781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afd200 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 -- 172.21.15.131:0/2066399781 >> 172.21.15.131:0/2066399781 conn(0x55dbe4af8450 msgr2=0x55dbe4af8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 -- 172.21.15.131:0/2066399781 shutdown_connections 2024-04-26T20:16:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 -- 172.21.15.131:0/2066399781 wait complete. 2024-04-26T20:16:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 -- start start 2024-04-26T20:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.718+0000 7f03e19a9740 1 Processor -- start 2024-04-26T20:16:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4b06ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b07530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03e19a9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4b058f0 con 0x55dbe4b02940 2024-04-26T20:16:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03e19a9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dbe49e8c60 con 0x55dbe4afcde0 2024-04-26T20:16:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03e19a9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4b05a70 con 0x55dbe4afd7d0 2024-04-26T20:16:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03e19a9740 10 client.0 _ll_register_callbacks cb 0x55dbe4a14e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:16:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b07530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b07530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45578/0 (socket says 172.21.15.131:45578) 2024-04-26T20:16:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 -- 172.21.15.131:0/1562888327 learned_addr learned my addr 172.21.15.131:0/1562888327 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03dca00640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4b06ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03dca00640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 -- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 msgr2=0x55dbe4b06ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4b06ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 -- 172.21.15.131:0/1562888327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 msgr2=0x55dbe4b0e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b0e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03cf400640 1 -- 172.21.15.131:0/1562888327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe49f7510 con 0x55dbe4afd7d0 2024-04-26T20:16:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.719+0000 7f03dca00640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b0e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:16:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03cf400640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b07530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f03c002a9e0 tx=0x7f03c0003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b68ea5cdc42161f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03c5600640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:16:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03c5600640 1 -- 172.21.15.131:0/1562888327 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0035070 con 0x55dbe4afd7d0 2024-04-26T20:16:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d0004d20 con 0x55dbe4afd7d0 2024-04-26T20:16:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03c5600640 1 -- 172.21.15.131:0/1562888327 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f03c0003c20 con 0x55dbe4afd7d0 2024-04-26T20:16:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03c5600640 1 -- 172.21.15.131:0/1562888327 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03c004f050 con 0x55dbe4afd7d0 2024-04-26T20:16:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03e19a9740 10 client.25648 fetch_fsmap learned FSMap version 27 2024-04-26T20:16:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.720+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbe4773210 con 0x55dbe4afd7d0 2024-04-26T20:16:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03c5600640 1 -- 172.21.15.131:0/1562888327 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d0004d20 con 0x55dbe4afd7d0 2024-04-26T20:16:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03c5600640 1 -- 172.21.15.131:0/1562888327 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03c00464d0 con 0x55dbe4afd7d0 2024-04-26T20:16:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03e19a9740 10 client.25648 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:16:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03e19a9740 10 client.25648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:16:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03e19a9740 20 client.25648 populate_metadata read hostname 'smithi131' 2024-04-26T20:16:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03e19a9740 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b12b30 0x55dbe4b14f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03e19a9740 4 client.25648 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:16:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe4a6ef90 con 0x55dbe4b12b30 2024-04-26T20:16:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.721+0000 7f03dca00640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b12b30 0x55dbe4b14f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.722+0000 7f03dca00640 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b12b30 0x55dbe4b14f40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f28a05aebcd9efa server_cookie=3e58d42bfa409bf8 in_seq=0 out_seq=0 2024-04-26T20:16:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.722+0000 7f03c5600640 10 client.25648 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03c5600640 1 -- 172.21.15.131:0/1562888327 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbe4a6ef90 con 0x55dbe4b12b30 2024-04-26T20:16:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03c5600640 10 client.25648 handle_command_reply: tid=0 2024-04-26T20:16:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 client.25648 shutdown 2024-04-26T20:16:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 msgr2=0x55dbe4b07530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b07530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f03c002a9e0 tx=0x7f03c0003820 comp rx=0 tx=0).stop 2024-04-26T20:16:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 shutdown_connections 2024-04-26T20:16:27.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b12b30 0x55dbe4b14f40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:27.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b0e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b07530 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03c5600640 0 client.25648 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 --2- 172.21.15.131:0/1562888327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4b06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 >> 172.21.15.131:0/1562888327 conn(0x55dbe4af8450 msgr2=0x55dbe4af7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 shutdown_connections 2024-04-26T20:16:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 1 -- 172.21.15.131:0/1562888327 wait complete. 2024-04-26T20:16:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.723+0000 7f03e19a9740 20 client.25648 trim_cache size 0 max 16384 2024-04-26T20:16:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.739+0000 7f03e19a9740 1 Processor -- start 2024-04-26T20:16:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.739+0000 7f03e19a9740 1 -- start start 2024-04-26T20:16:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.739+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.739+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.739+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.739+0000 7f03e19a9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b4370 con 0x55dbe4b02940 2024-04-26T20:16:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.739+0000 7f03e19a9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b1c40 con 0x55dbe4afcde0 2024-04-26T20:16:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.740+0000 7f03e19a9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81afa90 con 0x55dbe4afd7d0 2024-04-26T20:16:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.740+0000 7f03dca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.740+0000 7f03dca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.740+0000 7f03dca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b19840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40448/0 (socket says 172.21.15.131:40448) 2024-04-26T20:16:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.740+0000 7f03cf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.740+0000 7f03dca00640 1 -- 172.21.15.131:0/3782180964 learned_addr learned my addr 172.21.15.131:0/3782180964 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.740+0000 7f03cf400640 1 -- 172.21.15.131:0/3782180964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 msgr2=0x55dbe4afb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03cf400640 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03cf400640 1 -- 172.21.15.131:0/3782180964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 msgr2=0x55dbe4b19840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03cf400640 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b19840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03cf400640 1 -- 172.21.15.131:0/3782180964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe483f620 con 0x55dbe4afd7d0 2024-04-26T20:16:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03dca00640 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b19840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03cf400640 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0e530 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f03c0046a90 tx=0x7f03c002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c147266749a8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03c7e00640 1 -- 172.21.15.131:0/3782180964 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0035070 con 0x55dbe4afd7d0 2024-04-26T20:16:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03c7e00640 1 -- 172.21.15.131:0/3782180964 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0046b10 con 0x55dbe4afd7d0 2024-04-26T20:16:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.741+0000 7f03c7e00640 1 -- 172.21.15.131:0/3782180964 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0003c20 con 0x55dbe4afd7d0 2024-04-26T20:16:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 -- 172.21.15.131:0/3782180964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 msgr2=0x55dbe4b0e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0e530 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f03c0046a90 tx=0x7f03c002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:16:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 -- 172.21.15.131:0/3782180964 shutdown_connections 2024-04-26T20:16:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4b02940 0x55dbe4b19840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0e530 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3782180964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4afcde0 0x55dbe4afb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 -- 172.21.15.131:0/3782180964 >> 172.21.15.131:0/3782180964 conn(0x55dbe4af8450 msgr2=0x55dbe4afb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 -- 172.21.15.131:0/3782180964 shutdown_connections 2024-04-26T20:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 -- 172.21.15.131:0/3782180964 wait complete. 2024-04-26T20:16:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 -- start start 2024-04-26T20:16:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.742+0000 7f03e19a9740 1 Processor -- start 2024-04-26T20:16:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4afcde0 0x55dbe4b0c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03e19a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4b02940 0x55dbe4b0d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03e19a9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b0100 con 0x55dbe4afcde0 2024-04-26T20:16:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03e19a9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b1570 con 0x55dbe4b02940 2024-04-26T20:16:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03e19a9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b07d0 con 0x55dbe4afd7d0 2024-04-26T20:16:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03e19a9740 10 client.0 _ll_register_callbacks cb 0x7f03c0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:16:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03dca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4afcde0 0x55dbe4b0c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45602/0 (socket says 172.21.15.131:45602) 2024-04-26T20:16:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 -- 172.21.15.131:0/3369534002 learned_addr learned my addr 172.21.15.131:0/3369534002 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03dca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4afcde0 0x55dbe4b0c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40454/0 (socket says 172.21.15.131:40454) 2024-04-26T20:16:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03dca00640 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4b02940 0x55dbe4b0d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 -- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4b02940 msgr2=0x55dbe4b0d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4b02940 0x55dbe4b0d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 -- 172.21.15.131:0/3369534002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4afcde0 msgr2=0x55dbe4b0c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4afcde0 0x55dbe4b0c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.743+0000 7f03cf400640 1 -- 172.21.15.131:0/3369534002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe4a1bcb0 con 0x55dbe4afd7d0 2024-04-26T20:16:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03dca00640 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4b02940 0x55dbe4b0d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:16:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03cf400640 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0c710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03c0046920 tx=0x7f03c0005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31279bb0fa1f49e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03c5600640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:16:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03c5600640 1 -- 172.21.15.131:0/3369534002 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0035070 con 0x55dbe4afd7d0 2024-04-26T20:16:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03c5600640 1 -- 172.21.15.131:0/3369534002 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f03c0003c20 con 0x55dbe4afd7d0 2024-04-26T20:16:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c0046b10 con 0x55dbe4afd7d0 2024-04-26T20:16:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03c5600640 1 -- 172.21.15.131:0/3369534002 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03c0063050 con 0x55dbe4afd7d0 2024-04-26T20:16:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03e19a9740 10 client.25654 fetch_fsmap learned FSMap version 27 2024-04-26T20:16:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.744+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbe4b79450 con 0x55dbe4afd7d0 2024-04-26T20:16:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03c5600640 1 -- 172.21.15.131:0/3369534002 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0046b10 con 0x55dbe4afd7d0 2024-04-26T20:16:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03c5600640 1 -- 172.21.15.131:0/3369534002 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03c003c440 con 0x55dbe4afd7d0 2024-04-26T20:16:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03e19a9740 10 client.25654 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:16:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03e19a9740 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:16:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03e19a9740 20 client.25654 populate_metadata read hostname 'smithi131' 2024-04-26T20:16:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b79dc0 0x55dbe4b7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03e19a9740 4 client.25654 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:16:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbe4a68e00 con 0x55dbe4b79dc0 2024-04-26T20:16:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.745+0000 7f03dca00640 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b79dc0 0x55dbe4b7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.746+0000 7f03dca00640 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b79dc0 0x55dbe4b7c1e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56f20fd90077f6 server_cookie=3212c7761580442 in_seq=0 out_seq=0 2024-04-26T20:16:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.746+0000 7f03c5600640 10 client.25654 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03c5600640 1 -- 172.21.15.131:0/3369534002 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dbe4a68e00 con 0x55dbe4b79dc0 2024-04-26T20:16:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03c5600640 10 client.25654 handle_command_reply: tid=0 2024-04-26T20:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 client.25654 shutdown 2024-04-26T20:16:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 msgr2=0x55dbe4b0c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0c710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03c0046920 tx=0x7f03c0005a20 comp rx=0 tx=0).stop 2024-04-26T20:16:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 shutdown_connections 2024-04-26T20:16:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dbe4b79dc0 0x55dbe4b7c1e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbe4b02940 0x55dbe4b0d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe4afd7d0 0x55dbe4b0c710 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03c5600640 0 client.25654 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 --2- 172.21.15.131:0/3369534002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe4afcde0 0x55dbe4b0c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 >> 172.21.15.131:0/3369534002 conn(0x55dbe4af8450 msgr2=0x55dbe4af6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 shutdown_connections 2024-04-26T20:16:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 1 -- 172.21.15.131:0/3369534002 wait complete. 2024-04-26T20:16:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.752+0000 7f03e19a9740 20 client.25654 trim_cache size 0 max 16384 2024-04-26T20:16:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:16:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:16:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:16:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:16:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03bc0a3c30 msgr2=0x7f03bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03bc0a3c30 0x7f03bc0a60f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55dbe4afdbd0 tx=0x7f03c80d4890 comp rx=0 tx=0).stop 2024-04-26T20:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8108de0 msgr2=0x7f03d819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8108de0 0x7f03d819efe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03c800e9c0 tx=0x7f03c800ee80 comp rx=0 tx=0).stop 2024-04-26T20:16:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 shutdown_connections 2024-04-26T20:16:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d810a990 0x7f03d81a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d8109fd0 0x7f03d819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03bc0a3c30 0x7f03bc0a60f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 --2- 172.21.15.131:0/1521172118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d8108de0 0x7f03d819efe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 >> 172.21.15.131:0/1521172118 conn(0x7f03d8077dc0 msgr2=0x7f03d810d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 shutdown_connections 2024-04-26T20:16:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:25.753+0000 7f03dd400640 1 -- 172.21.15.131:0/1521172118 wait complete. 2024-04-26T20:16:35.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:25 smithi131 ceph-mon[27919]: pgmap v572: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:35.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:27 smithi131 ceph-mon[27919]: pgmap v573: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:16:36.060 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:29 smithi131 ceph-mon[27919]: pgmap v574: 129 pgs: 129 active+clean; 651 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:16:36.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:30 smithi131 ceph-mon[27919]: pgmap v575: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T20:16:36.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:32 smithi131 ceph-mon[27919]: pgmap v576: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T20:16:36.121 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:34 smithi131 ceph-mon[27919]: pgmap v577: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T20:16:36.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:36 smithi131 ceph-mon[27919]: pgmap v578: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T20:16:36.280 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:21 smithi157 ceph-mon[32703]: pgmap v600: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:36.309 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:23 smithi157 ceph-mon[32703]: pgmap v601: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:16:36.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:25 smithi157 ceph-mon[32703]: pgmap v602: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:36.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:27 smithi157 ceph-mon[32703]: pgmap v603: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:36.456 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:29 smithi157 ceph-mon[32703]: pgmap v604: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:36.550 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:31 smithi157 ceph-mon[32703]: pgmap v605: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:36.620 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:33 smithi157 ceph-mon[32703]: pgmap v606: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:36.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:35 smithi157 ceph-mon[32703]: pgmap v607: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:16:36.963 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:21 smithi148 ceph-mon[32281]: pgmap v600: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:37.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:23 smithi148 ceph-mon[32281]: pgmap v601: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:16:37.057 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:25 smithi148 ceph-mon[32281]: pgmap v602: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:37.180 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:27 smithi148 ceph-mon[32281]: pgmap v603: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:37.290 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:29 smithi148 ceph-mon[32281]: pgmap v604: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:37.408 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:31 smithi148 ceph-mon[32281]: pgmap v605: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:37.511 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:33 smithi148 ceph-mon[32281]: pgmap v606: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:37.598 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:35 smithi148 ceph-mon[32281]: pgmap v607: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:16:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.899+0000 7f38c9c00640 1 -- 172.21.15.131:0/2724482065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4102990 msgr2=0x7f38c4102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.899+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2724482065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4102990 0x7f38c4102db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009900 tx=0x7f38b402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:16:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.899+0000 7f38c9c00640 1 -- 172.21.15.131:0/2724482065 shutdown_connections 2024-04-26T20:16:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.899+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2724482065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c4104540 0x7f38c410add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.899+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2724482065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c4103b80 0x7f38c4104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.899+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2724482065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4102990 0x7f38c4102db0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.899+0000 7f38c9c00640 1 -- 172.21.15.131:0/2724482065 >> 172.21.15.131:0/2724482065 conn(0x7f38c40fdf20 msgr2=0x7f38c4100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.900+0000 7f38c9c00640 1 -- 172.21.15.131:0/2724482065 shutdown_connections 2024-04-26T20:16:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.900+0000 7f38c9c00640 1 -- 172.21.15.131:0/2724482065 wait complete. 2024-04-26T20:16:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 Processor -- start 2024-04-26T20:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 -- start start 2024-04-26T20:16:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c4102990 0x7f38c419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4103b80 0x7f38c419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c4104540 0x7f38c41a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410e4e0 con 0x7f38c4104540 2024-04-26T20:16:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410e360 con 0x7f38c4102990 2024-04-26T20:16:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.901+0000 7f38c9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410e660 con 0x7f38c4103b80 2024-04-26T20:16:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4103b80 0x7f38c419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4103b80 0x7f38c419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52988/0 (socket says 172.21.15.131:52988) 2024-04-26T20:16:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c4102990 0x7f38c419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 -- 172.21.15.131:0/2635512012 learned_addr learned my addr 172.21.15.131:0/2635512012 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c3400640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c4104540 0x7f38c41a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 -- 172.21.15.131:0/2635512012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c4102990 msgr2=0x7f38c419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c4102990 0x7f38c419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 -- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c4104540 msgr2=0x7f38c41a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c4104540 0x7f38c41a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 -- 172.21.15.131:0/2635512012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c41a2980 con 0x7f38c4103b80 2024-04-26T20:16:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c3400640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c4104540 0x7f38c41a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:16:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.902+0000 7f38c2000640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4103b80 0x7f38c419b2a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38b000ea60 tx=0x7f38b000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.903+0000 7f38bb400640 1 -- 172.21.15.131:0/2635512012 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b000cde0 con 0x7f38c4103b80 2024-04-26T20:16:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.903+0000 7f38bb400640 1 -- 172.21.15.131:0/2635512012 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b00035f0 con 0x7f38c4103b80 2024-04-26T20:16:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.903+0000 7f38bb400640 1 -- 172.21.15.131:0/2635512012 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0010690 con 0x7f38c4103b80 2024-04-26T20:16:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.903+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c41a2c70 con 0x7f38c4103b80 2024-04-26T20:16:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.903+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41a31c0 con 0x7f38c4103b80 2024-04-26T20:16:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.906+0000 7f38cca54740 1 Processor -- start 2024-04-26T20:16:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38cca54740 1 -- start start 2024-04-26T20:16:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38cca54740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f8bd0 0x5642b39f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38cca54740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b39fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38cca54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39fe730 0x5642b3a00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38cca54740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5642b395d380 con 0x5642b39f95c0 2024-04-26T20:16:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38cca54740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5642b38f9c30 con 0x5642b39f8bd0 2024-04-26T20:16:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38cca54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5642b3959340 con 0x5642b39fe730 2024-04-26T20:16:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b39fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38c3400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f8bd0 0x5642b39f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b39fe0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43408/0 (socket says 172.21.15.131:43408) 2024-04-26T20:16:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.907+0000 7f38c2a00640 1 -- 172.21.15.131:0/3536507551 learned_addr learned my addr 172.21.15.131:0/3536507551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38c3400640 1 -- 172.21.15.131:0/3536507551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39fe730 msgr2=0x5642b3a00bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:16:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38c3400640 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39fe730 0x5642b3a00bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38c3400640 1 -- 172.21.15.131:0/3536507551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 msgr2=0x5642b39fe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38c3400640 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b39fe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38c3400640 1 -- 172.21.15.131:0/3536507551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b391b940 con 0x5642b39f8bd0 2024-04-26T20:16:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38c2a00640 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b39fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38c3400640 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f8bd0 0x5642b39f8ff0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009ee0 tx=0x7f38bc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83db2b5144bb59e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38bb400640 1 -- 172.21.15.131:0/2635512012 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f38b00037a0 con 0x7f38c4103b80 2024-04-26T20:16:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38afe00640 1 -- 172.21.15.131:0/3536507551 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc016070 con 0x5642b39f8bd0 2024-04-26T20:16:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38afe00640 1 -- 172.21.15.131:0/3536507551 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38bc01bb10 con 0x5642b39f8bd0 2024-04-26T20:16:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.908+0000 7f38afe00640 1 -- 172.21.15.131:0/3536507551 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc012be0 con 0x5642b39f8bd0 2024-04-26T20:16:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 -- 172.21.15.131:0/3536507551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f8bd0 msgr2=0x5642b39f8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f8bd0 0x5642b39f8ff0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009ee0 tx=0x7f38bc010740 comp rx=0 tx=0).stop 2024-04-26T20:16:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 -- 172.21.15.131:0/3536507551 shutdown_connections 2024-04-26T20:16:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39fe730 0x5642b3a00bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b39fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 --2- 172.21.15.131:0/3536507551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f8bd0 0x5642b39f8ff0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 -- 172.21.15.131:0/3536507551 >> 172.21.15.131:0/3536507551 conn(0x5642b39f4240 msgr2=0x5642b39f4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 -- 172.21.15.131:0/3536507551 shutdown_connections 2024-04-26T20:16:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.909+0000 7f38cca54740 1 -- 172.21.15.131:0/3536507551 wait complete. 2024-04-26T20:16:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.910+0000 7f38bb400640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38a80a3c00 0x7f38a80a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.910+0000 7f38bb400640 1 -- 172.21.15.131:0/2635512012 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f38b00d0cc0 con 0x7f38c4103b80 2024-04-26T20:16:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.910+0000 7f38cca54740 1 -- start start 2024-04-26T20:16:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.910+0000 7f38c2a00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38a80a3c00 0x7f38a80a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.910+0000 7f38cca54740 1 Processor -- start 2024-04-26T20:16:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38cca54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38cca54740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b3a033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38cca54740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a0a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38cca54740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5642b3a016e0 con 0x5642b39f95c0 2024-04-26T20:16:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38cca54740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5642b38e4c60 con 0x5642b39fe730 2024-04-26T20:16:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38cca54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5642b3a01860 con 0x5642b39f8bd0 2024-04-26T20:16:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38cca54740 10 client.0 _ll_register_callbacks cb 0x5642b3910e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:16:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38c2a00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38a80a3c00 0x7f38a80a60c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5642b39fa030 tx=0x7f38b4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38c2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b3a033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38c3400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a0a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38c2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b3a033c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43422/0 (socket says 172.21.15.131:43422) 2024-04-26T20:16:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53002/0 (socket says 172.21.15.131:53002) 2024-04-26T20:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.911+0000 7f38c2000640 1 -- 172.21.15.131:0/2591721477 learned_addr learned my addr 172.21.15.131:0/2591721477 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38c3400640 1 -- 172.21.15.131:0/2591721477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 msgr2=0x5642b3a02e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38c3400640 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38c3400640 1 -- 172.21.15.131:0/2591721477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 msgr2=0x5642b3a033c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38c3400640 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b3a033c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38c3400640 1 -- 172.21.15.131:0/2591721477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b38f3510 con 0x5642b39fe730 2024-04-26T20:16:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38c3400640 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a0a400 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009ee0 tx=0x7f38bc012680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ae41e2241763ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38acc00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:16:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38acc00640 1 -- 172.21.15.131:0/2591721477 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc016040 con 0x5642b39fe730 2024-04-26T20:16:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38acc00640 1 -- 172.21.15.131:0/2591721477 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f38bc004c40 con 0x5642b39fe730 2024-04-26T20:16:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.912+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38bc01bb10 con 0x5642b39fe730 2024-04-26T20:16:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38acc00640 1 -- 172.21.15.131:0/2591721477 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38bc031050 con 0x5642b39fe730 2024-04-26T20:16:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38cca54740 10 client.25670 fetch_fsmap learned FSMap version 27 2024-04-26T20:16:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642b366f210 con 0x5642b39fe730 2024-04-26T20:16:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38acc00640 1 -- 172.21.15.131:0/2591721477 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38bc01bb10 con 0x5642b39fe730 2024-04-26T20:16:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38acc00640 1 -- 172.21.15.131:0/2591721477 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f38bc02c030 con 0x5642b39fe730 2024-04-26T20:16:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38cca54740 10 client.25670 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:16:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38cca54740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:16:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.913+0000 7f38cca54740 20 client.25670 populate_metadata read hostname 'smithi131' 2024-04-26T20:16:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.914+0000 7f38cca54740 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a0e610 0x5642b3a10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.914+0000 7f38cca54740 4 client.25670 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:16:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.914+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642b396af90 con 0x5642b3a0e610 2024-04-26T20:16:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.914+0000 7f38c2a00640 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a0e610 0x5642b3a10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.915+0000 7f38acc00640 10 client.25670 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.915+0000 7f38c2a00640 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a0e610 0x5642b3a10a00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c135c403dcdbca2f server_cookie=8cc8da47f9c44cc4 in_seq=0 out_seq=0 2024-04-26T20:16:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38acc00640 1 -- 172.21.15.131:0/2591721477 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642b396af90 con 0x5642b3a0e610 2024-04-26T20:16:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38acc00640 10 client.25670 handle_command_reply: tid=0 2024-04-26T20:16:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 client.25670 shutdown 2024-04-26T20:16:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 msgr2=0x5642b3a0a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a0a400 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009ee0 tx=0x7f38bc012680 comp rx=0 tx=0).stop 2024-04-26T20:16:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 shutdown_connections 2024-04-26T20:16:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a0e610 0x5642b3a10a00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a0a400 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f95c0 0x5642b3a033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 --2- 172.21.15.131:0/2591721477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38acc00640 0 client.25670 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 >> 172.21.15.131:0/2591721477 conn(0x5642b39f4240 msgr2=0x5642b39f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 shutdown_connections 2024-04-26T20:16:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 1 -- 172.21.15.131:0/2591721477 wait complete. 2024-04-26T20:16:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.916+0000 7f38cca54740 20 client.25670 trim_cache size 0 max 16384 2024-04-26T20:16:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.932+0000 7f38cca54740 1 Processor -- start 2024-04-26T20:16:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.932+0000 7f38cca54740 1 -- start start 2024-04-26T20:16:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38cca54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38cca54740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f95c0 0x5642b3a0a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38cca54740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39fe730 0x5642b3a033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38cca54740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41b0170 con 0x5642b39fe730 2024-04-26T20:16:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38cca54740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41ada40 con 0x5642b39f95c0 2024-04-26T20:16:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38cca54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41ab890 con 0x5642b39f8bd0 2024-04-26T20:16:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39fe730 0x5642b3a033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53016/0 (socket says 172.21.15.131:53016) 2024-04-26T20:16:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 -- 172.21.15.131:0/3009819531 learned_addr learned my addr 172.21.15.131:0/3009819531 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2000640 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f95c0 0x5642b3a0a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 -- 172.21.15.131:0/3009819531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f95c0 msgr2=0x5642b3a0a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f95c0 0x5642b3a0a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 -- 172.21.15.131:0/3009819531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39fe730 msgr2=0x5642b3a033c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39fe730 0x5642b3a033c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.933+0000 7f38c2a00640 1 -- 172.21.15.131:0/3009819531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b373b620 con 0x5642b39f8bd0 2024-04-26T20:16:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.934+0000 7f38c2a00640 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f38b402a9e0 tx=0x7f38b4040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10125e500bc8a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.934+0000 7f38af400640 1 -- 172.21.15.131:0/3009819531 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4035040 con 0x5642b39f8bd0 2024-04-26T20:16:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.934+0000 7f38af400640 1 -- 172.21.15.131:0/3009819531 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b4058070 con 0x5642b39f8bd0 2024-04-26T20:16:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.934+0000 7f38af400640 1 -- 172.21.15.131:0/3009819531 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4031d80 con 0x5642b39f8bd0 2024-04-26T20:16:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 -- 172.21.15.131:0/3009819531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 msgr2=0x5642b3a02e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f38b402a9e0 tx=0x7f38b4040000 comp rx=0 tx=0).stop 2024-04-26T20:16:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 -- 172.21.15.131:0/3009819531 shutdown_connections 2024-04-26T20:16:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39fe730 0x5642b3a033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39f95c0 0x5642b3a0a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 --2- 172.21.15.131:0/3009819531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f8bd0 0x5642b3a02e80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 -- 172.21.15.131:0/3009819531 >> 172.21.15.131:0/3009819531 conn(0x5642b39f4240 msgr2=0x5642b39fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 -- 172.21.15.131:0/3009819531 shutdown_connections 2024-04-26T20:16:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 -- 172.21.15.131:0/3009819531 wait complete. 2024-04-26T20:16:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.935+0000 7f38cca54740 1 -- start start 2024-04-26T20:16:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 1 Processor -- start 2024-04-26T20:16:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f8bd0 0x5642b3a0a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f95c0 0x5642b3a06e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a07400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41abf00 con 0x5642b39f8bd0 2024-04-26T20:16:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41ad370 con 0x5642b39fe730 2024-04-26T20:16:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41ac5d0 con 0x5642b39f95c0 2024-04-26T20:16:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38cca54740 10 client.0 _ll_register_callbacks cb 0x7f38b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:16:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38c2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f95c0 0x5642b3a06e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38c2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f8bd0 0x5642b3a0a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38c3400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a07400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38c2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f95c0 0x5642b3a06e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53032/0 (socket says 172.21.15.131:53032) 2024-04-26T20:16:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.936+0000 7f38c2000640 1 -- 172.21.15.131:0/292227359 learned_addr learned my addr 172.21.15.131:0/292227359 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38c2000640 1 -- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 msgr2=0x5642b3a07400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38c2000640 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a07400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38c2000640 1 -- 172.21.15.131:0/292227359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f8bd0 msgr2=0x5642b3a0a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38c2000640 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f8bd0 0x5642b3a0a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38c2000640 1 -- 172.21.15.131:0/292227359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b3917cb0 con 0x5642b39f95c0 2024-04-26T20:16:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38c3400640 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a07400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38c2000640 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f95c0 0x5642b3a06e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38b000d010 tx=0x7f38b0003dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be018cf3b79b8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38acc00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:16:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38acc00640 1 -- 172.21.15.131:0/292227359 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00b5030 con 0x5642b39f95c0 2024-04-26T20:16:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38acc00640 1 -- 172.21.15.131:0/292227359 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f38b00ce4d0 con 0x5642b39f95c0 2024-04-26T20:16:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.937+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b4058070 con 0x5642b39f95c0 2024-04-26T20:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38acc00640 1 -- 172.21.15.131:0/292227359 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38b009d150 con 0x5642b39f95c0 2024-04-26T20:16:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 10 client.25669 fetch_fsmap learned FSMap version 27 2024-04-26T20:16:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642b3a73cc0 con 0x5642b39f95c0 2024-04-26T20:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38acc00640 1 -- 172.21.15.131:0/292227359 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b4058070 con 0x5642b39f95c0 2024-04-26T20:16:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38acc00640 1 -- 172.21.15.131:0/292227359 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f38b00990c0 con 0x5642b39f95c0 2024-04-26T20:16:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 10 client.25669 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:16:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 10 client.25669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:16:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 20 client.25669 populate_metadata read hostname 'smithi131' 2024-04-26T20:16:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a74620 0x5642b3a76a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 4 client.25669 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:16:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642b3964e00 con 0x5642b3a74620 2024-04-26T20:16:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.938+0000 7f38c2a00640 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a74620 0x5642b3a76a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.939+0000 7f38c2a00640 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a74620 0x5642b3a76a40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c6e5462cd148370 server_cookie=4f44ecfb2f324976 in_seq=0 out_seq=0 2024-04-26T20:16:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.939+0000 7f38acc00640 10 client.25669 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38acc00640 1 -- 172.21.15.131:0/292227359 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642b3964e00 con 0x5642b3a74620 2024-04-26T20:16:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38acc00640 10 client.25669 handle_command_reply: tid=0 2024-04-26T20:16:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 client.25669 shutdown 2024-04-26T20:16:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f95c0 msgr2=0x5642b3a06e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f95c0 0x5642b3a06e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38b000d010 tx=0x7f38b0003dd0 comp rx=0 tx=0).stop 2024-04-26T20:16:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 shutdown_connections 2024-04-26T20:16:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5642b3a74620 0x5642b3a76a40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5642b39fe730 0x5642b3a07400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b39f95c0 0x5642b3a06e40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 --2- 172.21.15.131:0/292227359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b39f8bd0 0x5642b3a0a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 >> 172.21.15.131:0/292227359 conn(0x5642b39f4240 msgr2=0x5642b3a02d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38acc00640 0 client.25669 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:16:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 shutdown_connections 2024-04-26T20:16:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 1 -- 172.21.15.131:0/292227359 wait complete. 2024-04-26T20:16:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.947+0000 7f38cca54740 20 client.25669 trim_cache size 0 max 16384 2024-04-26T20:16:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:16:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:16:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:16:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38a80a3c00 msgr2=0x7f38a80a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38a80a3c00 0x7f38a80a60c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5642b39fa030 tx=0x7f38b4005a20 comp rx=0 tx=0).stop 2024-04-26T20:16:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4103b80 msgr2=0x7f38c419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4103b80 0x7f38c419b2a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38b000ea60 tx=0x7f38b000ef20 comp rx=0 tx=0).stop 2024-04-26T20:16:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 shutdown_connections 2024-04-26T20:16:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c4104540 0x7f38c41a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f38a80a3c00 0x7f38a80a60c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4103b80 0x7f38c419b2a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 --2- 172.21.15.131:0/2635512012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c4102990 0x7f38c419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 >> 172.21.15.131:0/2635512012 conn(0x7f38c40fdf20 msgr2=0x7f38c4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 shutdown_connections 2024-04-26T20:16:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:38.948+0000 7f38c9c00640 1 -- 172.21.15.131:0/2635512012 wait complete. 2024-04-26T20:16:50.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:38 smithi131 ceph-mon[27919]: pgmap v579: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:16:50.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:38 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:16:50.837 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: pgmap v580: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:16:51.099 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:51.199 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:51.264 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:51.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:51.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:16:51.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:16:51.424 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:16:51.453 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:16:51.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:16:51.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:42 smithi131 ceph-mon[27919]: pgmap v581: 129 pgs: 129 active+clean; 662 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T20:16:51.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:44 smithi131 ceph-mon[27919]: pgmap v582: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T20:16:51.824 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:46 smithi131 ceph-mon[27919]: pgmap v583: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T20:16:51.835 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:48 smithi131 ceph-mon[27919]: pgmap v584: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T20:16:51.849 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:50 smithi131 ceph-mon[27919]: pgmap v585: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T20:16:51.858 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:37 smithi157 ceph-mon[32703]: pgmap v608: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:51.864 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:39 smithi157 ceph-mon[32703]: pgmap v609: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:52.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:40 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:16:52.147 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: pgmap v610: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:52.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.373 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.387 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.519 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.637 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.721 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:52.833 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:16:52.957 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:16:53.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:16:53.140 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:42 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:16:53.147 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:42 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:16:53.154 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:42 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:16:53.161 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:42 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:16:53.169 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:43 smithi157 ceph-mon[32703]: pgmap v611: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:53.298 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:45 smithi157 ceph-mon[32703]: pgmap v612: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:53.485 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:47 smithi157 ceph-mon[32703]: pgmap v613: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:16:53.589 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:49 smithi157 ceph-mon[32703]: pgmap v614: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:16:53.647 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:37 smithi148 ceph-mon[32281]: pgmap v608: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:53.760 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:39 smithi148 ceph-mon[32281]: pgmap v609: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:53.782 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:40 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:16:53.791 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: pgmap v610: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:53.907 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.057 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.211 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.483 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.592 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.730 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.807 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:16:54.814 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:16:54.820 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:16:54.994 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:16:55.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:42 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:16:55.113 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:42 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:16:55.124 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:42 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:16:55.216 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:42 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:16:55.224 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:43 smithi148 ceph-mon[32281]: pgmap v611: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:55.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:45 smithi148 ceph-mon[32281]: pgmap v612: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:16:55.314 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:47 smithi148 ceph-mon[32281]: pgmap v613: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:16:55.383 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:49 smithi148 ceph-mon[32281]: pgmap v614: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:16:55.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:51 smithi148 ceph-mon[32281]: pgmap v615: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:55.488 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:53 smithi148 ceph-mon[32281]: pgmap v616: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:55.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:51 smithi157 ceph-mon[32703]: pgmap v615: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:16:55.500 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:53 smithi157 ceph-mon[32703]: pgmap v616: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:16:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.091+0000 7effdfc00640 1 -- 172.21.15.131:0/1770442146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 msgr2=0x7effd810a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.091+0000 7effdfc00640 1 --2- 172.21.15.131:0/1770442146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 0x7effd810a630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:16:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.092+0000 7effdfc00640 1 -- 172.21.15.131:0/1770442146 shutdown_connections 2024-04-26T20:16:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.092+0000 7effdfc00640 1 --2- 172.21.15.131:0/1770442146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd810ab70 0x7effd8111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.092+0000 7effdfc00640 1 --2- 172.21.15.131:0/1770442146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 0x7effd810a630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.092+0000 7effdfc00640 1 --2- 172.21.15.131:0/1770442146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd8108fc0 0x7effd81093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.092+0000 7effdfc00640 1 -- 172.21.15.131:0/1770442146 >> 172.21.15.131:0/1770442146 conn(0x7effd8077f80 msgr2=0x7effd807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.092+0000 7effdfc00640 1 -- 172.21.15.131:0/1770442146 shutdown_connections 2024-04-26T20:16:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.092+0000 7effdfc00640 1 -- 172.21.15.131:0/1770442146 wait complete. 2024-04-26T20:16:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 Processor -- start 2024-04-26T20:16:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 -- start start 2024-04-26T20:16:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd8108fc0 0x7effd819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 0x7effd819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd810ab70 0x7effd81a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effd8114b50 con 0x7effd810ab70 2024-04-26T20:16:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7effd81149d0 con 0x7effd810a1b0 2024-04-26T20:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdfc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effd8114cd0 con 0x7effd8108fc0 2024-04-26T20:16:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdd200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd8108fc0 0x7effd819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdd200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd8108fc0 0x7effd819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52752/0 (socket says 172.21.15.131:52752) 2024-04-26T20:16:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effdd200640 1 -- 172.21.15.131:0/4008834015 learned_addr learned my addr 172.21.15.131:0/4008834015 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.093+0000 7effddc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd810ab70 0x7effd81a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effd7e00640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 0x7effd819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effdd200640 1 -- 172.21.15.131:0/4008834015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 msgr2=0x7effd819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effdd200640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 0x7effd819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effdd200640 1 -- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd810ab70 msgr2=0x7effd81a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effdd200640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd810ab70 0x7effd81a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effdd200640 1 -- 172.21.15.131:0/4008834015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81a6d40 con 0x7effd8108fc0 2024-04-26T20:16:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effdd200640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd8108fc0 0x7effd819f120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7effc800e9c0 tx=0x7effc800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.094+0000 7effd5600640 1 -- 172.21.15.131:0/4008834015 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc800cd80 con 0x7effd8108fc0 2024-04-26T20:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.095+0000 7effd5600640 1 -- 172.21.15.131:0/4008834015 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc80035f0 con 0x7effd8108fc0 2024-04-26T20:16:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.095+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd81a6fd0 con 0x7effd8108fc0 2024-04-26T20:16:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.095+0000 7effd5600640 1 -- 172.21.15.131:0/4008834015 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8010690 con 0x7effd8108fc0 2024-04-26T20:16:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.095+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a74e0 con 0x7effd8108fc0 2024-04-26T20:16:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 Processor -- start 2024-04-26T20:16:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effd5600640 1 -- 172.21.15.131:0/4008834015 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7effc80037a0 con 0x7effd8108fc0 2024-04-26T20:16:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 -- start start 2024-04-26T20:16:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b24bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c925a81380 con 0x55c925b1d5c0 2024-04-26T20:16:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c925a1dc30 con 0x55c925b1cbd0 2024-04-26T20:16:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.098+0000 7effe2a59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c925a7d340 con 0x55c925b22730 2024-04-26T20:16:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effdd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effdd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b220c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40370/0 (socket says 172.21.15.131:40370) 2024-04-26T20:16:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:45788/0 (socket says 172.21.15.131:45788) 2024-04-26T20:16:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effdd200640 1 -- 172.21.15.131:0/2484830296 learned_addr learned my addr 172.21.15.131:0/2484830296 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:16:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effd7e00640 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b24bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 -- 172.21.15.131:0/2484830296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 msgr2=0x55c925b24bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:59.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b24bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 -- 172.21.15.131:0/2484830296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 msgr2=0x55c925b220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 -- 172.21.15.131:0/2484830296 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c925a3f940 con 0x55c925b1cbd0 2024-04-26T20:16:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effdd200640 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:16:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.099+0000 7effddc00640 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1cff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7effd0009f00 tx=0x7effd0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f351b0a923a407 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effce000640 1 -- 172.21.15.131:0/2484830296 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0004d20 con 0x55c925b1cbd0 2024-04-26T20:17:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effce000640 1 -- 172.21.15.131:0/2484830296 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd0004030 con 0x55c925b1cbd0 2024-04-26T20:17:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effce000640 1 -- 172.21.15.131:0/2484830296 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0005280 con 0x55c925b1cbd0 2024-04-26T20:17:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effd5600640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effbc0a3d70 0x7effbc0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effd5600640 1 -- 172.21.15.131:0/4008834015 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7effc80d0c80 con 0x7effd8108fc0 2024-04-26T20:17:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effdd200640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effbc0a3d70 0x7effbc0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effe2a59740 1 -- 172.21.15.131:0/2484830296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 msgr2=0x55c925b1cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effe2a59740 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1cff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7effd0009f00 tx=0x7effd0011810 comp rx=0 tx=0).stop 2024-04-26T20:17:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effe2a59740 1 -- 172.21.15.131:0/2484830296 shutdown_connections 2024-04-26T20:17:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effe2a59740 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b24bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effe2a59740 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effe2a59740 1 --2- 172.21.15.131:0/2484830296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1cff0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.100+0000 7effe2a59740 1 -- 172.21.15.131:0/2484830296 >> 172.21.15.131:0/2484830296 conn(0x55c925b18240 msgr2=0x55c925b18640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.101+0000 7effe2a59740 1 -- 172.21.15.131:0/2484830296 shutdown_connections 2024-04-26T20:17:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.101+0000 7effe2a59740 1 -- 172.21.15.131:0/2484830296 wait complete. 2024-04-26T20:17:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.101+0000 7effdd200640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effbc0a3d70 0x7effbc0a6230 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7effc80058c0 tx=0x7effc80d47d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.101+0000 7effe2a59740 1 -- start start 2024-04-26T20:17:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.101+0000 7effe2a59740 1 Processor -- start 2024-04-26T20:17:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effe2a59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effe2a59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effe2a59740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effe2a59740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c925b256e0 con 0x55c925b22730 2024-04-26T20:17:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effe2a59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c925a08c60 con 0x55c925b1cbd0 2024-04-26T20:17:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effe2a59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c925b25860 con 0x55c925b1d5c0 2024-04-26T20:17:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effe2a59740 10 client.0 _ll_register_callbacks cb 0x55c925a34e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:17:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effd7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effddc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effd7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b272c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52780/0 (socket says 172.21.15.131:52780) 2024-04-26T20:17:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effd7e00640 1 -- 172.21.15.131:0/305466285 learned_addr learned my addr 172.21.15.131:0/305466285 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effddc00640 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effd7e00640 1 -- 172.21.15.131:0/305466285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 msgr2=0x55c925b26d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effd7e00640 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b26d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effd7e00640 1 -- 172.21.15.131:0/305466285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 msgr2=0x55c925b2e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.102+0000 7effd7e00640 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effd7e00640 1 -- 172.21.15.131:0/305466285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c925a17510 con 0x55c925b1d5c0 2024-04-26T20:17:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effddc00640 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:17:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effd7e00640 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b272c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effc4009230 tx=0x7effc4004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc782bf7193f7c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:17:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effc2a00640 1 -- 172.21.15.131:0/305466285 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035040 con 0x55c925b1d5c0 2024-04-26T20:17:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effd0004030 con 0x55c925b1d5c0 2024-04-26T20:17:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effc2a00640 1 -- 172.21.15.131:0/305466285 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7effc4003a00 con 0x55c925b1d5c0 2024-04-26T20:17:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effc2a00640 1 -- 172.21.15.131:0/305466285 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effc404e050 con 0x55c925b1d5c0 2024-04-26T20:17:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effe2a59740 10 client.25681 fetch_fsmap learned FSMap version 27 2024-04-26T20:17:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.103+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c925793210 con 0x55c925b1d5c0 2024-04-26T20:17:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effc2a00640 1 -- 172.21.15.131:0/305466285 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd0004030 con 0x55c925b1d5c0 2024-04-26T20:17:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effc2a00640 1 -- 172.21.15.131:0/305466285 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7effc403c440 con 0x55c925b1d5c0 2024-04-26T20:17:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effe2a59740 10 client.25681 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:17:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effe2a59740 10 client.25681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:17:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effe2a59740 20 client.25681 populate_metadata read hostname 'smithi131' 2024-04-26T20:17:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effe2a59740 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b32910 0x55c925b34d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effe2a59740 4 client.25681 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:17:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c925a8ef90 con 0x55c925b32910 2024-04-26T20:17:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.104+0000 7effddc00640 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b32910 0x55c925b34d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.105+0000 7effddc00640 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b32910 0x55c925b34d40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d089d296ce9d8b0c server_cookie=1119b68519d66cf in_seq=0 out_seq=0 2024-04-26T20:17:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.105+0000 7effc2a00640 10 client.25681 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effc2a00640 1 -- 172.21.15.131:0/305466285 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c925a8ef90 con 0x55c925b32910 2024-04-26T20:17:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effc2a00640 10 client.25681 handle_command_reply: tid=0 2024-04-26T20:17:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 client.25681 shutdown 2024-04-26T20:17:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 msgr2=0x55c925b272c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b272c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effc4009230 tx=0x7effc4004340 comp rx=0 tx=0).stop 2024-04-26T20:17:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 shutdown_connections 2024-04-26T20:17:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b32910 0x55c925b34d40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b272c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effc2a00640 0 client.25681 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 --2- 172.21.15.131:0/305466285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 >> 172.21.15.131:0/305466285 conn(0x55c925b18240 msgr2=0x55c925b17440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 shutdown_connections 2024-04-26T20:17:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 1 -- 172.21.15.131:0/305466285 wait complete. 2024-04-26T20:17:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.106+0000 7effe2a59740 20 client.25681 trim_cache size 0 max 16384 2024-04-26T20:17:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.122+0000 7effe2a59740 1 Processor -- start 2024-04-26T20:17:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.122+0000 7effe2a59740 1 -- start start 2024-04-26T20:17:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.122+0000 7effe2a59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effe2a59740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b2e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effe2a59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effe2a59740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b4470 con 0x55c925b1d5c0 2024-04-26T20:17:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effe2a59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b1d40 con 0x55c925b1cbd0 2024-04-26T20:17:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effe2a59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effd81afb90 con 0x55c925b22730 2024-04-26T20:17:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b2e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b2e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40384/0 (socket says 172.21.15.131:40384) 2024-04-26T20:17:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 -- 172.21.15.131:0/2402326261 learned_addr learned my addr 172.21.15.131:0/2402326261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effddc00640 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effddc00640 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 -- 172.21.15.131:0/2402326261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 msgr2=0x55c925b1b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b1b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 -- 172.21.15.131:0/2402326261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 msgr2=0x55c925b1b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.123+0000 7effd7e00640 1 -- 172.21.15.131:0/2402326261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92585f620 con 0x55c925b1d5c0 2024-04-26T20:17:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effd7e00640 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b2e2c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effc4004650 tx=0x7effc4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1804319d169a08ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effcd600640 1 -- 172.21.15.131:0/2402326261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035070 con 0x55c925b1d5c0 2024-04-26T20:17:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effcd600640 1 -- 172.21.15.131:0/2402326261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc4005850 con 0x55c925b1d5c0 2024-04-26T20:17:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effcd600640 1 -- 172.21.15.131:0/2402326261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc40315a0 con 0x55c925b1d5c0 2024-04-26T20:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effe2a59740 1 -- 172.21.15.131:0/2402326261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 msgr2=0x55c925b2e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effe2a59740 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b2e2c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effc4004650 tx=0x7effc4009520 comp rx=0 tx=0).stop 2024-04-26T20:17:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effe2a59740 1 -- 172.21.15.131:0/2402326261 shutdown_connections 2024-04-26T20:17:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effe2a59740 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b22730 0x55c925b1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effe2a59740 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b1d5c0 0x55c925b2e2c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effe2a59740 1 --2- 172.21.15.131:0/2402326261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b1b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.124+0000 7effe2a59740 1 -- 172.21.15.131:0/2402326261 >> 172.21.15.131:0/2402326261 conn(0x55c925b18240 msgr2=0x55c925b1e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.125+0000 7effe2a59740 1 -- 172.21.15.131:0/2402326261 shutdown_connections 2024-04-26T20:17:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.125+0000 7effe2a59740 1 -- 172.21.15.131:0/2402326261 wait complete. 2024-04-26T20:17:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.125+0000 7effe2a59740 1 -- start start 2024-04-26T20:17:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.125+0000 7effe2a59740 1 Processor -- start 2024-04-26T20:17:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effe2a59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b2ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effe2a59740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b2bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effe2a59740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effe2a59740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b0200 con 0x55c925b22730 2024-04-26T20:17:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effe2a59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b1670 con 0x55c925b1cbd0 2024-04-26T20:17:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effe2a59740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b08d0 con 0x55c925b1d5c0 2024-04-26T20:17:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effe2a59740 10 client.0 _ll_register_callbacks cb 0x7effc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:17:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b2bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b2bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52786/0 (socket says 172.21.15.131:52786) 2024-04-26T20:17:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 -- 172.21.15.131:0/319622757 learned_addr learned my addr 172.21.15.131:0/319622757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effddc00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b2ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effddc00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 -- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 msgr2=0x55c925b2ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b2ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 -- 172.21.15.131:0/319622757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 msgr2=0x55c925b2cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effd7e00640 1 -- 172.21.15.131:0/319622757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c925a3bcb0 con 0x55c925b1d5c0 2024-04-26T20:17:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.126+0000 7effddc00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b2ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:17:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effd7e00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b2bfd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7effc4009850 tx=0x7effc4004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7a1621889f95cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:17:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effc2a00640 1 -- 172.21.15.131:0/319622757 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035040 con 0x55c925b1d5c0 2024-04-26T20:17:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effc2a00640 1 -- 172.21.15.131:0/319622757 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7effc4003a00 con 0x55c925b1d5c0 2024-04-26T20:17:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc4005850 con 0x55c925b1d5c0 2024-04-26T20:17:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effc2a00640 1 -- 172.21.15.131:0/319622757 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effc4003f30 con 0x55c925b1d5c0 2024-04-26T20:17:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effe2a59740 10 client.25687 fetch_fsmap learned FSMap version 27 2024-04-26T20:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c925b98ec0 con 0x55c925b1d5c0 2024-04-26T20:17:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.127+0000 7effc2a00640 1 -- 172.21.15.131:0/319622757 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc4005850 con 0x55c925b1d5c0 2024-04-26T20:17:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effc2a00640 1 -- 172.21.15.131:0/319622757 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7effc4046550 con 0x55c925b1d5c0 2024-04-26T20:17:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effe2a59740 10 client.25687 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:17:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effe2a59740 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:17:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effe2a59740 20 client.25687 populate_metadata read hostname 'smithi131' 2024-04-26T20:17:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effe2a59740 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b99570 0x55c925b9b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effe2a59740 4 client.25687 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:17:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c925a88e00 con 0x55c925b99570 2024-04-26T20:17:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.128+0000 7effddc00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b99570 0x55c925b9b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.129+0000 7effddc00640 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b99570 0x55c925b9b990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdb684bd3cb6832 server_cookie=c4d92eba09b40a2c in_seq=0 out_seq=0 2024-04-26T20:17:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.129+0000 7effc2a00640 10 client.25687 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effc2a00640 1 -- 172.21.15.131:0/319622757 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c925a88e00 con 0x55c925b99570 2024-04-26T20:17:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effc2a00640 10 client.25687 handle_command_reply: tid=0 2024-04-26T20:17:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 client.25687 shutdown 2024-04-26T20:17:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 msgr2=0x55c925b2bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b2bfd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7effc4009850 tx=0x7effc4004470 comp rx=0 tx=0).stop 2024-04-26T20:17:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 shutdown_connections 2024-04-26T20:17:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c925b99570 0x55c925b9b990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c925b22730 0x55c925b2cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c925b1d5c0 0x55c925b2bfd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effc2a00640 0 client.25687 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 --2- 172.21.15.131:0/319622757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c925b1cbd0 0x55c925b2ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.132+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 >> 172.21.15.131:0/319622757 conn(0x55c925b18240 msgr2=0x55c925b1b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 shutdown_connections 2024-04-26T20:17:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effe2a59740 1 -- 172.21.15.131:0/319622757 wait complete. 2024-04-26T20:17:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effe2a59740 20 client.25687 trim_cache size 0 max 16384 2024-04-26T20:17:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:17:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:17:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:17:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:17:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effbc0a3d70 msgr2=0x7effbc0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effbc0a3d70 0x7effbc0a6230 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7effc80058c0 tx=0x7effc80d47d0 comp rx=0 tx=0).stop 2024-04-26T20:17:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd8108fc0 msgr2=0x7effd819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd8108fc0 0x7effd819f120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7effc800e9c0 tx=0x7effc800ee80 comp rx=0 tx=0).stop 2024-04-26T20:17:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 shutdown_connections 2024-04-26T20:17:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effd810ab70 0x7effd81a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd810a1b0 0x7effd819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7effbc0a3d70 0x7effbc0a6230 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 --2- 172.21.15.131:0/4008834015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effd8108fc0 0x7effd819f120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.133+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 >> 172.21.15.131:0/4008834015 conn(0x7effd8077f80 msgr2=0x7effd810d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.134+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 shutdown_connections 2024-04-26T20:17:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:15:52.134+0000 7effdfc00640 1 -- 172.21.15.131:0/4008834015 wait complete. 2024-04-26T20:17:11.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:52 smithi131 ceph-mon[27919]: pgmap v586: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T20:17:11.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:54 smithi131 ceph-mon[27919]: pgmap v587: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T20:17:11.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:56 smithi131 ceph-mon[27919]: pgmap v588: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:11.791 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:15:58 smithi131 ceph-mon[27919]: pgmap v589: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:11.804 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:55 smithi148 ceph-mon[32281]: pgmap v617: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:11.813 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:57 smithi148 ceph-mon[32281]: pgmap v618: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:11.953 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:16:59 smithi148 ceph-mon[32281]: pgmap v619: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:11.960 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:01 smithi148 ceph-mon[32281]: pgmap v620: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:11.999 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:03 smithi148 ceph-mon[32281]: pgmap v621: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:17:12.149 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:05 smithi148 ceph-mon[32281]: pgmap v622: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:12.286 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:07 smithi148 ceph-mon[32281]: pgmap v623: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:12.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:09 smithi148 ceph-mon[32281]: pgmap v624: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:12.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:55 smithi157 ceph-mon[32703]: pgmap v617: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:12.572 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:57 smithi157 ceph-mon[32703]: pgmap v618: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:12.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:16:59 smithi157 ceph-mon[32703]: pgmap v619: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:12.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:01 smithi157 ceph-mon[32703]: pgmap v620: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:12.674 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:03 smithi157 ceph-mon[32703]: pgmap v621: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:17:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:05 smithi157 ceph-mon[32703]: pgmap v622: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:12.845 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:07 smithi157 ceph-mon[32703]: pgmap v623: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:12.931 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:09 smithi157 ceph-mon[32703]: pgmap v624: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:12.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 -- 172.21.15.131:0/2540973518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff850108d90 msgr2=0x7ff8501091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 --2- 172.21.15.131:0/2540973518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff850108d90 0x7ff8501091b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff844009900 tx=0x7ff84402a410 comp rx=0 tx=0).stop 2024-04-26T20:17:13.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 -- 172.21.15.131:0/2540973518 shutdown_connections 2024-04-26T20:17:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 --2- 172.21.15.131:0/2540973518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff85010a940 0x7ff8501111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 --2- 172.21.15.131:0/2540973518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff850109f80 0x7ff85010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 --2- 172.21.15.131:0/2540973518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff850108d90 0x7ff8501091b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:13.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 -- 172.21.15.131:0/2540973518 >> 172.21.15.131:0/2540973518 conn(0x7ff850077d30 msgr2=0x7ff85007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:13.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 -- 172.21.15.131:0/2540973518 shutdown_connections 2024-04-26T20:17:13.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.165+0000 7ff856c00640 1 -- 172.21.15.131:0/2540973518 wait complete. 2024-04-26T20:17:13.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.166+0000 7ff856c00640 1 Processor -- start 2024-04-26T20:17:13.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.166+0000 7ff856c00640 1 -- start start 2024-04-26T20:17:13.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.166+0000 7ff856c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff850108d90 0x7ff85019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:13.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.166+0000 7ff856c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff850109f80 0x7ff85019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:13.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.166+0000 7ff856c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff85010a940 0x7ff8501a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:14.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.166+0000 7ff856c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501148e0 con 0x7ff85010a940 2024-04-26T20:17:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff856c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff850114760 con 0x7ff850108d90 2024-04-26T20:17:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff856c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff850114a60 con 0x7ff850109f80 2024-04-26T20:17:14.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff850108d90 0x7ff85019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:14.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff854e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff850109f80 0x7ff85019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:14.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff850108d90 0x7ff85019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:52192/0 (socket says 172.21.15.131:52192) 2024-04-26T20:17:14.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff856200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff85010a940 0x7ff8501a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:14.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 -- 172.21.15.131:0/1005451432 learned_addr learned my addr 172.21.15.131:0/1005451432 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:14.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 -- 172.21.15.131:0/1005451432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff850109f80 msgr2=0x7ff85019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:14.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff850109f80 0x7ff85019f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 -- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff85010a940 msgr2=0x7ff8501a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff85010a940 0x7ff8501a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:14.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.167+0000 7ff855800640 1 -- 172.21.15.131:0/1005451432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8501a6c00 con 0x7ff850108d90 2024-04-26T20:17:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.168+0000 7ff855800640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff850108d90 0x7ff85019efe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff844031500 tx=0x7ff844003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:14.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.168+0000 7ff84e000640 1 -- 172.21.15.131:0/1005451432 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84403f070 con 0x7ff850108d90 2024-04-26T20:17:14.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.168+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8501a6e90 con 0x7ff850108d90 2024-04-26T20:17:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.168+0000 7ff84e000640 1 -- 172.21.15.131:0/1005451432 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff844031c50 con 0x7ff850108d90 2024-04-26T20:17:15.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.168+0000 7ff84e000640 1 -- 172.21.15.131:0/1005451432 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844035030 con 0x7ff850108d90 2024-04-26T20:17:15.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.168+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8501a72f0 con 0x7ff850108d90 2024-04-26T20:17:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.169+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff850110180 con 0x7ff850108d90 2024-04-26T20:17:15.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.171+0000 7ff84e000640 1 -- 172.21.15.131:0/1005451432 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff844042070 con 0x7ff850108d90 2024-04-26T20:17:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.172+0000 7ff84e000640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff8380a3cc0 0x7ff8380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.173+0000 7ff84e000640 1 -- 172.21.15.131:0/1005451432 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff8440f0fe0 con 0x7ff850108d90 2024-04-26T20:17:15.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.173+0000 7ff854e00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff8380a3cc0 0x7ff8380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:15.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.173+0000 7ff854e00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff8380a3cc0 0x7ff8380a6180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff8400095f0 tx=0x7ff840005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:15.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.177+0000 7ff84e000640 1 -- 172.21.15.131:0/1005451432 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff8440bd450 con 0x7ff850108d90 2024-04-26T20:17:15.911 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:00 smithi131 ceph-mon[27919]: pgmap v590: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:16.018 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:11 smithi148 ceph-mon[32281]: pgmap v625: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:16.034 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:13 smithi148 ceph-mon[32281]: pgmap v626: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:16.154 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:11 smithi157 ceph-mon[32703]: pgmap v625: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:16.161 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:13 smithi157 ceph-mon[32703]: pgmap v626: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:16.287 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:15 smithi157 ceph-mon[32703]: pgmap v627: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:17:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.416+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff85002dad0 con 0x7ff850108d90 2024-04-26T20:17:16.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.418+0000 7ff84e000640 1 -- 172.21.15.131:0/1005451432 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7ff8440bcd50 con 0x7ff850108d90 2024-04-26T20:17:16.830 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:17:16.928 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:12:47.696526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24571,24557]},"id":1}]} 2024-04-26T20:17:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-04-26T20:17:17.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff8380a3cc0 msgr2=0x7ff8380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:17.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff8380a3cc0 0x7ff8380a6180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff8400095f0 tx=0x7ff840005f90 comp rx=0 tx=0).stop 2024-04-26T20:17:17.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff850108d90 msgr2=0x7ff85019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:17.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff850108d90 0x7ff85019efe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff844031500 tx=0x7ff844003c10 comp rx=0 tx=0).stop 2024-04-26T20:17:17.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 shutdown_connections 2024-04-26T20:17:17.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff85010a940 0x7ff8501a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff8380a3cc0 0x7ff8380a6180 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:17.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff850109f80 0x7ff85019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:17.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 --2- 172.21.15.131:0/1005451432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff850108d90 0x7ff85019efe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:17.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 >> 172.21.15.131:0/1005451432 conn(0x7ff850077d30 msgr2=0x7ff85010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:17.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 shutdown_connections 2024-04-26T20:17:17.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:16:01.420+0000 7ff856c00640 1 -- 172.21.15.131:0/1005451432 wait complete. 2024-04-26T20:17:17.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T20:17:17.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:01 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1005451432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:17:17.682 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:02 smithi131 ceph-mon[27919]: pgmap v591: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:17.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:04 smithi131 ceph-mon[27919]: pgmap v592: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:17.806 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:15 smithi148 ceph-mon[32281]: pgmap v627: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:17:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.273+0000 7fbab0800640 1 -- 172.21.15.131:0/2727115243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa810a1b0 msgr2=0x7fbaa810a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.273+0000 7fbab0800640 1 --2- 172.21.15.131:0/2727115243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa810a1b0 0x7fbaa810a630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba98009860 tx=0x7fba9802a340 comp rx=0 tx=0).stop 2024-04-26T20:17:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.274+0000 7fbab0800640 1 -- 172.21.15.131:0/2727115243 shutdown_connections 2024-04-26T20:17:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.274+0000 7fbab0800640 1 --2- 172.21.15.131:0/2727115243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaa810ab70 0x7fbaa8111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.274+0000 7fbab0800640 1 --2- 172.21.15.131:0/2727115243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa810a1b0 0x7fbaa810a630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.274+0000 7fbab0800640 1 --2- 172.21.15.131:0/2727115243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa8108fc0 0x7fbaa81093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.274+0000 7fbab0800640 1 -- 172.21.15.131:0/2727115243 >> 172.21.15.131:0/2727115243 conn(0x7fbaa8077f80 msgr2=0x7fbaa807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.274+0000 7fbab0800640 1 -- 172.21.15.131:0/2727115243 shutdown_connections 2024-04-26T20:17:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.274+0000 7fbab0800640 1 -- 172.21.15.131:0/2727115243 wait complete. 2024-04-26T20:17:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 Processor -- start 2024-04-26T20:17:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 -- start start 2024-04-26T20:17:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa8108fc0 0x7fbaa819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa810a1b0 0x7fbaa819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaa810ab70 0x7fbaa81a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8114b00 con 0x7fbaa810a1b0 2024-04-26T20:17:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8114980 con 0x7fbaa810ab70 2024-04-26T20:17:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbab0800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8114c80 con 0x7fbaa8108fc0 2024-04-26T20:17:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbaad400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa810a1b0 0x7fbaa819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbaad400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa810a1b0 0x7fbaa819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55124/0 (socket says 172.21.15.131:55124) 2024-04-26T20:17:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbaad400640 1 -- 172.21.15.131:0/631248766 learned_addr learned my addr 172.21.15.131:0/631248766 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbaade00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa8108fc0 0x7fbaa819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.275+0000 7fbaae800640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaa810ab70 0x7fbaa81a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaad400640 1 -- 172.21.15.131:0/631248766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa8108fc0 msgr2=0x7fbaa819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaad400640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa8108fc0 0x7fbaa819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaad400640 1 -- 172.21.15.131:0/631248766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaa810ab70 msgr2=0x7fbaa81a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaad400640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaa810ab70 0x7fbaa81a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaad400640 1 -- 172.21.15.131:0/631248766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa81a6d10 con 0x7fbaa810a1b0 2024-04-26T20:17:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaad400640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa810a1b0 0x7fbaa819f630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba980057c0 tx=0x7fba98032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaa2a00640 1 -- 172.21.15.131:0/631248766 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98037040 con 0x7fbaa810a1b0 2024-04-26T20:17:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa81a6fa0 con 0x7fbaa810a1b0 2024-04-26T20:17:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.276+0000 7fbaa2a00640 1 -- 172.21.15.131:0/631248766 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba98030bd0 con 0x7fbaa810a1b0 2024-04-26T20:17:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.277+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa81a73b0 con 0x7fbaa810a1b0 2024-04-26T20:17:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.277+0000 7fbaa2a00640 1 -- 172.21.15.131:0/631248766 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98005220 con 0x7fbaa810a1b0 2024-04-26T20:17:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 Processor -- start 2024-04-26T20:17:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 -- start start 2024-04-26T20:17:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbaa2a00640 1 -- 172.21.15.131:0/631248766 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fba98043020 con 0x7fbaa810a1b0 2024-04-26T20:17:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb52fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb5350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb535730 0x55dfdb537bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dfdb494380 con 0x55dfdb52fbd0 2024-04-26T20:17:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dfdb430c30 con 0x55dfdb5305c0 2024-04-26T20:17:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.280+0000 7fbab3611740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dfdb490340 con 0x55dfdb535730 2024-04-26T20:17:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb52fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb52fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55136/0 (socket says 172.21.15.131:55136) 2024-04-26T20:17:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 -- 172.21.15.131:0/2941243315 learned_addr learned my addr 172.21.15.131:0/2941243315 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaad400640 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb535730 0x55dfdb537bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaade00640 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb5350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 -- 172.21.15.131:0/2941243315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb535730 msgr2=0x55dfdb537bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb535730 0x55dfdb537bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 -- 172.21.15.131:0/2941243315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 msgr2=0x55dfdb5350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb5350c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 -- 172.21.15.131:0/2941243315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdb452940 con 0x55dfdb52fbd0 2024-04-26T20:17:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaad400640 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb535730 0x55dfdb537bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:17:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.281+0000 7fbaae800640 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb52fff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009fd0 tx=0x7fbaa40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ed40e120c9a22b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.282+0000 7fba97000640 1 -- 172.21.15.131:0/2941243315 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4004d20 con 0x55dfdb52fbd0 2024-04-26T20:17:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.282+0000 7fbaa2a00640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fba900a3cf0 0x7fba900a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.282+0000 7fba97000640 1 -- 172.21.15.131:0/2941243315 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa4004050 con 0x55dfdb52fbd0 2024-04-26T20:17:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.282+0000 7fba97000640 1 -- 172.21.15.131:0/2941243315 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4005280 con 0x55dfdb52fbd0 2024-04-26T20:17:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.282+0000 7fbaa2a00640 1 -- 172.21.15.131:0/631248766 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fba980eff00 con 0x7fbaa810a1b0 2024-04-26T20:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.282+0000 7fbaade00640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fba900a3cf0 0x7fba900a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 -- 172.21.15.131:0/2941243315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 msgr2=0x55dfdb52fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb52fff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009fd0 tx=0x7fbaa40118d0 comp rx=0 tx=0).stop 2024-04-26T20:17:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 -- 172.21.15.131:0/2941243315 shutdown_connections 2024-04-26T20:17:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb535730 0x55dfdb537bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb5350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 --2- 172.21.15.131:0/2941243315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb52fff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 -- 172.21.15.131:0/2941243315 >> 172.21.15.131:0/2941243315 conn(0x55dfdb52b240 msgr2=0x55dfdb52b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 -- 172.21.15.131:0/2941243315 shutdown_connections 2024-04-26T20:17:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbaade00640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fba900a3cf0 0x7fba900a61b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55dfdb5309c0 tx=0x7fba9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 -- 172.21.15.131:0/2941243315 wait complete. 2024-04-26T20:17:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.283+0000 7fbab3611740 1 -- start start 2024-04-26T20:17:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 1 Processor -- start 2024-04-26T20:17:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb5305c0 0x55dfdb53a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb5413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dfdb5386e0 con 0x55dfdb52fbd0 2024-04-26T20:17:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dfdb41bc60 con 0x55dfdb535730 2024-04-26T20:17:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dfdb538860 con 0x55dfdb5305c0 2024-04-26T20:17:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbab3611740 10 client.0 _ll_register_callbacks cb 0x55dfdb447e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:17:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbaade00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbaade00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55150/0 (socket says 172.21.15.131:55150) 2024-04-26T20:17:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbaade00640 1 -- 172.21.15.131:0/2635870224 learned_addr learned my addr 172.21.15.131:0/2635870224 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbaad400640 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb5305c0 0x55dfdb53a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.284+0000 7fbaae800640 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb5413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fbaade00640 1 -- 172.21.15.131:0/2635870224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb5305c0 msgr2=0x55dfdb53a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fbaade00640 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb5305c0 0x55dfdb53a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fbaade00640 1 -- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 msgr2=0x55dfdb5413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fbaade00640 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb5413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fbaade00640 1 -- 172.21.15.131:0/2635870224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdb42a510 con 0x55dfdb52fbd0 2024-04-26T20:17:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fbaade00640 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0045f0 tx=0x7fba9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0a5d5d9cc8953d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fba8b400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:17:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fba8b400640 1 -- 172.21.15.131:0/2635870224 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c02c060 con 0x55dfdb52fbd0 2024-04-26T20:17:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fba8b400640 1 -- 172.21.15.131:0/2635870224 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fba9c01d650 con 0x55dfdb52fbd0 2024-04-26T20:17:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.285+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa4004050 con 0x55dfdb52fbd0 2024-04-26T20:17:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fba8b400640 1 -- 172.21.15.131:0/2635870224 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba9c033050 con 0x55dfdb52fbd0 2024-04-26T20:17:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 10 client.15594 fetch_fsmap learned FSMap version 27 2024-04-26T20:17:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dfdb1a6210 con 0x55dfdb52fbd0 2024-04-26T20:17:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fba8b400640 1 -- 172.21.15.131:0/2635870224 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa4004050 con 0x55dfdb52fbd0 2024-04-26T20:17:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fba8b400640 1 -- 172.21.15.131:0/2635870224 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fba9c026030 con 0x55dfdb52fbd0 2024-04-26T20:17:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:17:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:17:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 20 client.15594 populate_metadata read hostname 'smithi131' 2024-04-26T20:17:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5459c0 0x55dfdb547dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 4 client.15594 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:17:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.286+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfdb4a1f90 con 0x55dfdb5459c0 2024-04-26T20:17:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.287+0000 7fbaad400640 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5459c0 0x55dfdb547dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.287+0000 7fbaad400640 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5459c0 0x55dfdb547dd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e421949b20c9e7e server_cookie=a6c74f44b32faf90 in_seq=0 out_seq=0 2024-04-26T20:17:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.287+0000 7fba8b400640 10 client.15594 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fba8b400640 1 -- 172.21.15.131:0/2635870224 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfdb4a1f90 con 0x55dfdb5459c0 2024-04-26T20:17:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fba8b400640 10 client.15594 handle_command_reply: tid=0 2024-04-26T20:17:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 client.15594 shutdown 2024-04-26T20:17:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 msgr2=0x55dfdb539e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0045f0 tx=0x7fba9c004730 comp rx=0 tx=0).stop 2024-04-26T20:17:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 shutdown_connections 2024-04-26T20:17:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5459c0 0x55dfdb547dd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb5413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fba8b400640 0 client.15594 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb5305c0 0x55dfdb53a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 --2- 172.21.15.131:0/2635870224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539e40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 >> 172.21.15.131:0/2635870224 conn(0x55dfdb52b240 msgr2=0x55dfdb52a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 shutdown_connections 2024-04-26T20:17:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.288+0000 7fbab3611740 1 -- 172.21.15.131:0/2635870224 wait complete. 2024-04-26T20:17:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.289+0000 7fbab3611740 20 client.15594 trim_cache size 0 max 16384 2024-04-26T20:17:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 Processor -- start 2024-04-26T20:17:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 -- start start 2024-04-26T20:17:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb541350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb535730 0x55dfdb539f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81b43c0 con 0x55dfdb535730 2024-04-26T20:17:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81b1c90 con 0x55dfdb5305c0 2024-04-26T20:17:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbab3611740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81afae0 con 0x55dfdb52fbd0 2024-04-26T20:17:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbaae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb535730 0x55dfdb539f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbaade00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbaae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb535730 0x55dfdb539f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55154/0 (socket says 172.21.15.131:55154) 2024-04-26T20:17:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbaae800640 1 -- 172.21.15.131:0/1819762869 learned_addr learned my addr 172.21.15.131:0/1819762869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.305+0000 7fbaad400640 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb541350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fbaad400640 1 -- 172.21.15.131:0/1819762869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 msgr2=0x55dfdb539a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fbaad400640 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fbaad400640 1 -- 172.21.15.131:0/1819762869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb535730 msgr2=0x55dfdb539f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fbaad400640 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb535730 0x55dfdb539f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fbaad400640 1 -- 172.21.15.131:0/1819762869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdb272620 con 0x55dfdb5305c0 2024-04-26T20:17:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fbaade00640 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:17:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fbaad400640 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb541350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fba980ed570 tx=0x7fba98005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe84653f52189a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fba96600640 1 -- 172.21.15.131:0/1819762869 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba980bb6a0 con 0x55dfdb5305c0 2024-04-26T20:17:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fba96600640 1 -- 172.21.15.131:0/1819762869 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba980bc440 con 0x55dfdb5305c0 2024-04-26T20:17:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.306+0000 7fba96600640 1 -- 172.21.15.131:0/1819762869 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba980053c0 con 0x55dfdb5305c0 2024-04-26T20:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 -- 172.21.15.131:0/1819762869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 msgr2=0x55dfdb541350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb541350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fba980ed570 tx=0x7fba98005f90 comp rx=0 tx=0).stop 2024-04-26T20:17:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 -- 172.21.15.131:0/1819762869 shutdown_connections 2024-04-26T20:17:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb535730 0x55dfdb539f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb5305c0 0x55dfdb541350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 --2- 172.21.15.131:0/1819762869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb539a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 -- 172.21.15.131:0/1819762869 >> 172.21.15.131:0/1819762869 conn(0x55dfdb52b240 msgr2=0x55dfdb52b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 -- 172.21.15.131:0/1819762869 shutdown_connections 2024-04-26T20:17:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 -- 172.21.15.131:0/1819762869 wait complete. 2024-04-26T20:17:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.307+0000 7fbab3611740 1 -- start start 2024-04-26T20:17:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 1 Processor -- start 2024-04-26T20:17:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb53e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb5305c0 0x55dfdb53edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb53fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81b0150 con 0x55dfdb5305c0 2024-04-26T20:17:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81b15c0 con 0x55dfdb535730 2024-04-26T20:17:28.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81b0820 con 0x55dfdb52fbd0 2024-04-26T20:17:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbab3611740 10 client.0 _ll_register_callbacks cb 0x7fba9c009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:17:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbaad400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb5305c0 0x55dfdb53edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbaade00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb53e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbaad400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb5305c0 0x55dfdb53edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55164/0 (socket says 172.21.15.131:55164) 2024-04-26T20:17:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbaad400640 1 -- 172.21.15.131:0/1369331148 learned_addr learned my addr 172.21.15.131:0/1369331148 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.308+0000 7fbaae800640 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb53fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbaad400640 1 -- 172.21.15.131:0/1369331148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 msgr2=0x55dfdb53e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbaad400640 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb53e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbaad400640 1 -- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 msgr2=0x55dfdb53fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbaad400640 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb53fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbaad400640 1 -- 172.21.15.131:0/1369331148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdb44ecb0 con 0x55dfdb5305c0 2024-04-26T20:17:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbaade00640 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb53e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:17:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbaad400640 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb5305c0 0x55dfdb53edc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fba980b8000 tx=0x7fba980d5240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9047745c4561735 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fba8b400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:17:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fba8b400640 1 -- 172.21.15.131:0/1369331148 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba980d9070 con 0x55dfdb5305c0 2024-04-26T20:17:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fba8b400640 1 -- 172.21.15.131:0/1369331148 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fba980b83b0 con 0x55dfdb5305c0 2024-04-26T20:17:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba980bc440 con 0x55dfdb5305c0 2024-04-26T20:17:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fba8b400640 1 -- 172.21.15.131:0/1369331148 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba98030df0 con 0x55dfdb5305c0 2024-04-26T20:17:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbab3611740 10 client.15600 fetch_fsmap learned FSMap version 27 2024-04-26T20:17:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.309+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dfdb541550 con 0x55dfdb5305c0 2024-04-26T20:17:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fba8b400640 1 -- 172.21.15.131:0/1369331148 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba980bc440 con 0x55dfdb5305c0 2024-04-26T20:17:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fba8b400640 1 -- 172.21.15.131:0/1369331148 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fba980f1040 con 0x55dfdb5305c0 2024-04-26T20:17:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fbab3611740 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:17:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fbab3611740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:17:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fbab3611740 20 client.15600 populate_metadata read hostname 'smithi131' 2024-04-26T20:17:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fbab3611740 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5ac610 0x55dfdb5aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fbab3611740 4 client.15600 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:17:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfdb49be00 con 0x55dfdb5ac610 2024-04-26T20:17:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.310+0000 7fbaade00640 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5ac610 0x55dfdb5aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.311+0000 7fbaade00640 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5ac610 0x55dfdb5aea30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea175b259e6adc2 server_cookie=630f3812ec195ab3 in_seq=0 out_seq=0 2024-04-26T20:17:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.311+0000 7fba8b400640 10 client.15600 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fba8b400640 1 -- 172.21.15.131:0/1369331148 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfdb49be00 con 0x55dfdb5ac610 2024-04-26T20:17:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fba8b400640 10 client.15600 handle_command_reply: tid=0 2024-04-26T20:17:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 client.15600 shutdown 2024-04-26T20:17:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb5305c0 msgr2=0x55dfdb53edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb5305c0 0x55dfdb53edc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fba980b8000 tx=0x7fba980d5240 comp rx=0 tx=0).stop 2024-04-26T20:17:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 shutdown_connections 2024-04-26T20:17:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55dfdb5ac610 0x55dfdb5aea30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:31.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfdb535730 0x55dfdb53fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfdb5305c0 0x55dfdb53edc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 --2- 172.21.15.131:0/1369331148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dfdb52fbd0 0x55dfdb53e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fba8b400640 0 client.15600 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 >> 172.21.15.131:0/1369331148 conn(0x55dfdb52b240 msgr2=0x55dfdb539a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 shutdown_connections 2024-04-26T20:17:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 1 -- 172.21.15.131:0/1369331148 wait complete. 2024-04-26T20:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.315+0000 7fbab3611740 20 client.15600 trim_cache size 0 max 16384 2024-04-26T20:17:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:17:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:17:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:17:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:17:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fba900a3cf0 msgr2=0x7fba900a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fba900a3cf0 0x7fba900a61b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55dfdb5309c0 tx=0x7fba9c009260 comp rx=0 tx=0).stop 2024-04-26T20:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa810a1b0 msgr2=0x7fbaa819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa810a1b0 0x7fbaa819f630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba980057c0 tx=0x7fba98032da0 comp rx=0 tx=0).stop 2024-04-26T20:17:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 shutdown_connections 2024-04-26T20:17:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaa810ab70 0x7fbaa81a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa810a1b0 0x7fbaa819f630 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fba900a3cf0 0x7fba900a61b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 --2- 172.21.15.131:0/631248766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaa8108fc0 0x7fbaa819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 >> 172.21.15.131:0/631248766 conn(0x7fbaa8077f80 msgr2=0x7fbaa810d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 shutdown_connections 2024-04-26T20:17:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:05.316+0000 7fbab0800640 1 -- 172.21.15.131:0/631248766 wait complete. 2024-04-26T20:17:33.363 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:06 smithi131 ceph-mon[27919]: pgmap v593: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:17:33.406 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:08 smithi131 ceph-mon[27919]: pgmap v594: 129 pgs: 129 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:17:33.457 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:10 smithi131 ceph-mon[27919]: pgmap v595: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:33.610 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:12 smithi131 ceph-mon[27919]: pgmap v596: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:33.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:14 smithi131 ceph-mon[27919]: pgmap v597: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:33.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:17 smithi131 ceph-mon[27919]: pgmap v598: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:33.787 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:17 smithi148 ceph-mon[32281]: pgmap v628: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:33.865 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:19 smithi148 ceph-mon[32281]: pgmap v629: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:33.871 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:21 smithi148 ceph-mon[32281]: pgmap v630: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:34.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:23 smithi148 ceph-mon[32281]: pgmap v631: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:34.108 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:25 smithi148 ceph-mon[32281]: pgmap v632: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:34.146 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:26 smithi148 ceph-mon[32281]: pgmap v633: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:17:34.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:28 smithi148 ceph-mon[32281]: pgmap v634: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:17:34.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:30 smithi148 ceph-mon[32281]: pgmap v635: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:34.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:32 smithi148 ceph-mon[32281]: pgmap v636: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:34.483 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:17 smithi157 ceph-mon[32703]: pgmap v628: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:34.574 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:19 smithi157 ceph-mon[32703]: pgmap v629: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:34.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:21 smithi157 ceph-mon[32703]: pgmap v630: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:34.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:23 smithi157 ceph-mon[32703]: pgmap v631: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:34.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:25 smithi157 ceph-mon[32703]: pgmap v632: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:34.854 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:26 smithi157 ceph-mon[32703]: pgmap v633: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:17:34.862 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:28 smithi157 ceph-mon[32703]: pgmap v634: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:17:34.868 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:30 smithi157 ceph-mon[32703]: pgmap v635: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:32 smithi157 ceph-mon[32703]: pgmap v636: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 -- 172.21.15.131:0/952957422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 msgr2=0x7f8cf0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/952957422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf0101d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009900 tx=0x7f8ce002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:17:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 -- 172.21.15.131:0/952957422 shutdown_connections 2024-04-26T20:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/952957422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf01034f0 0x7f8cf0109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/952957422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cf0102b30 0x7f8cf0102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/952957422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf0101d60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 -- 172.21.15.131:0/952957422 >> 172.21.15.131:0/952957422 conn(0x7f8cf00fceb0 msgr2=0x7f8cf00ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 -- 172.21.15.131:0/952957422 shutdown_connections 2024-04-26T20:17:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.460+0000 7f8cf7c00640 1 -- 172.21.15.131:0/952957422 wait complete. 2024-04-26T20:17:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 Processor -- start 2024-04-26T20:17:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 -- start start 2024-04-26T20:17:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf0102b30 0x7f8cf019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cf01034f0 0x7f8cf01a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010e560 con 0x7f8cf0101940 2024-04-26T20:17:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010e3e0 con 0x7f8cf01034f0 2024-04-26T20:17:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.461+0000 7f8cf7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010e6e0 con 0x7f8cf0102b30 2024-04-26T20:17:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43894/0 (socket says 172.21.15.131:43894) 2024-04-26T20:17:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5200640 1 -- 172.21.15.131:0/761983704 learned_addr learned my addr 172.21.15.131:0/761983704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cf01034f0 0x7f8cf01a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf0102b30 0x7f8cf019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5c00640 1 -- 172.21.15.131:0/761983704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf0102b30 msgr2=0x7f8cf019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf0102b30 0x7f8cf019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5c00640 1 -- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 msgr2=0x7f8cf019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5c00640 1 -- 172.21.15.131:0/761983704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01a2920 con 0x7f8cf01034f0 2024-04-26T20:17:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5200640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:17:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.462+0000 7f8cf5c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cf01034f0 0x7f8cf01a2280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009fd0 tx=0x7f8ce800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.463+0000 7f8cee000640 1 -- 172.21.15.131:0/761983704 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce800ce20 con 0x7f8cf01034f0 2024-04-26T20:17:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.463+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01a2c10 con 0x7f8cf01034f0 2024-04-26T20:17:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.463+0000 7f8cee000640 1 -- 172.21.15.131:0/761983704 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce8005020 con 0x7f8cf01034f0 2024-04-26T20:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.463+0000 7f8cee000640 1 -- 172.21.15.131:0/761983704 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8010690 con 0x7f8cf01034f0 2024-04-26T20:17:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.463+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a3110 con 0x7f8cf01034f0 2024-04-26T20:17:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.466+0000 7f8cfaa79740 1 Processor -- start 2024-04-26T20:17:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.466+0000 7f8cee000640 1 -- 172.21.15.131:0/761983704 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8ce80054d0 con 0x7f8cf01034f0 2024-04-26T20:17:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.466+0000 7f8cfaa79740 1 -- start start 2024-04-26T20:17:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.466+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd2bd0 0x55acf3cd2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd35c0 0x55acf3cd80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3cdabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55acf3c37380 con 0x55acf3cd35c0 2024-04-26T20:17:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acf3bd3c30 con 0x55acf3cd2bd0 2024-04-26T20:17:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55acf3c33340 con 0x55acf3cd8730 2024-04-26T20:17:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd35c0 0x55acf3cd80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd2bd0 0x55acf3cd2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3cdabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3cdabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40296/0 (socket says 172.21.15.131:40296) 2024-04-26T20:17:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 -- 172.21.15.131:0/2842816913 learned_addr learned my addr 172.21.15.131:0/2842816913 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd35c0 0x55acf3cd80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43898/0 (socket says 172.21.15.131:43898) 2024-04-26T20:17:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 -- 172.21.15.131:0/2842816913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd2bd0 msgr2=0x55acf3cd2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/2842816913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd2bd0 0x55acf3cd2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 -- 172.21.15.131:0/2842816913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd35c0 msgr2=0x55acf3cd80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/2842816913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd35c0 0x55acf3cd80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.467+0000 7f8ce7e00640 1 -- 172.21.15.131:0/2842816913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf3bf5940 con 0x55acf3cd8730 2024-04-26T20:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.468+0000 7f8cee000640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cd40a3cf0 0x7f8cd40a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.468+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/2842816913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3cdabc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8010960 tx=0x7f8cd8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82c9faa6a9a7f954 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.468+0000 7f8cee000640 1 -- 172.21.15.131:0/761983704 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8ce80d1150 con 0x7f8cf01034f0 2024-04-26T20:17:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.468+0000 7f8cf5200640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cd40a3cf0 0x7f8cd40a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.468+0000 7f8ce6000640 1 -- 172.21.15.131:0/2842816913 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8016070 con 0x55acf3cd8730 2024-04-26T20:17:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.468+0000 7f8ce6000640 1 -- 172.21.15.131:0/2842816913 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd801e3b0 con 0x55acf3cd8730 2024-04-26T20:17:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.468+0000 7f8ce6000640 1 -- 172.21.15.131:0/2842816913 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8019040 con 0x55acf3cd8730 2024-04-26T20:17:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 -- 172.21.15.131:0/2842816913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 msgr2=0x55acf3cdabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/2842816913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3cdabc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8010960 tx=0x7f8cd8005f90 comp rx=0 tx=0).stop 2024-04-26T20:17:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cf5200640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cd40a3cf0 0x7f8cd40a61b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0006010 tx=0x7f8ce0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 -- 172.21.15.131:0/2842816913 shutdown_connections 2024-04-26T20:17:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/2842816913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3cdabc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/2842816913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd35c0 0x55acf3cd80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/2842816913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd2bd0 0x55acf3cd2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 -- 172.21.15.131:0/2842816913 >> 172.21.15.131:0/2842816913 conn(0x55acf3cce240 msgr2=0x55acf3cce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 -- 172.21.15.131:0/2842816913 shutdown_connections 2024-04-26T20:17:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 -- 172.21.15.131:0/2842816913 wait complete. 2024-04-26T20:17:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.469+0000 7f8cfaa79740 1 -- start start 2024-04-26T20:17:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 1 Processor -- start 2024-04-26T20:17:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3cdd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd8730 0x55acf3ce42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55acf3cdb680 con 0x55acf3cd8730 2024-04-26T20:17:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acf3bbec60 con 0x55acf3cd35c0 2024-04-26T20:17:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55acf3cdb800 con 0x55acf3cd2bd0 2024-04-26T20:17:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cfaa79740 10 client.0 _ll_register_callbacks cb 0x55acf3beae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:17:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cf5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd8730 0x55acf3ce42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40304/0 (socket says 172.21.15.131:40304) 2024-04-26T20:17:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8cf5200640 1 -- 172.21.15.131:0/953508869 learned_addr learned my addr 172.21.15.131:0/953508869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.470+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3cdd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cf5200640 1 -- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 msgr2=0x55acf3cdd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cf5200640 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3cdd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cf5200640 1 -- 172.21.15.131:0/953508869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd8730 msgr2=0x55acf3ce42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cf5200640 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd8730 0x55acf3ce42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cf5200640 1 -- 172.21.15.131:0/953508869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf3bcd510 con 0x55acf3cd2bd0 2024-04-26T20:17:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cf5200640 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdcd20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005ab0 tx=0x7f8ce002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=824dd92880a2d069 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cdea00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:17:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cdea00640 1 -- 172.21.15.131:0/953508869 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0035050 con 0x55acf3cd2bd0 2024-04-26T20:17:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.471+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd801e3b0 con 0x55acf3cd2bd0 2024-04-26T20:17:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cdea00640 1 -- 172.21.15.131:0/953508869 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8ce00317b0 con 0x55acf3cd2bd0 2024-04-26T20:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cdea00640 1 -- 172.21.15.131:0/953508869 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ce00319b0 con 0x55acf3cd2bd0 2024-04-26T20:17:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 10 client.25723 fetch_fsmap learned FSMap version 27 2024-04-26T20:17:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acf3949210 con 0x55acf3cd2bd0 2024-04-26T20:17:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cdea00640 1 -- 172.21.15.131:0/953508869 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd801e3b0 con 0x55acf3cd2bd0 2024-04-26T20:17:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cdea00640 1 -- 172.21.15.131:0/953508869 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8ce0049430 con 0x55acf3cd2bd0 2024-04-26T20:17:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 10 client.25723 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:17:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 20 client.25723 populate_metadata read hostname 'smithi131' 2024-04-26T20:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3ce8860 0x55acf3ceacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 4 client.25723 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:17:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.472+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acf3c44f90 con 0x55acf3ce8860 2024-04-26T20:17:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.473+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3ce8860 0x55acf3ceacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.473+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3ce8860 0x55acf3ceacd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d126aa6b3e656b0 server_cookie=f371355c4d0b8e16 in_seq=0 out_seq=0 2024-04-26T20:17:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.473+0000 7f8cdea00640 10 client.25723 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.474+0000 7f8cdea00640 1 -- 172.21.15.131:0/953508869 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acf3c44f90 con 0x55acf3ce8860 2024-04-26T20:17:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.474+0000 7f8cdea00640 10 client.25723 handle_command_reply: tid=0 2024-04-26T20:17:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.474+0000 7f8cfaa79740 1 client.25723 shutdown 2024-04-26T20:17:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.474+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd2bd0 msgr2=0x55acf3cdcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.474+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdcd20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005ab0 tx=0x7f8ce002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:17:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 shutdown_connections 2024-04-26T20:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3ce8860 0x55acf3ceacd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd8730 0x55acf3ce42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3cdd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/953508869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdcd20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 >> 172.21.15.131:0/953508869 conn(0x55acf3cce240 msgr2=0x55acf3ccd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cdea00640 0 client.25723 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 shutdown_connections 2024-04-26T20:17:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 1 -- 172.21.15.131:0/953508869 wait complete. 2024-04-26T20:17:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.475+0000 7f8cfaa79740 20 client.25723 trim_cache size 0 max 16384 2024-04-26T20:17:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 Processor -- start 2024-04-26T20:17:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 -- start start 2024-04-26T20:17:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd35c0 0x55acf3ce4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd8730 0x55acf3cdd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01aff40 con 0x55acf3cd2bd0 2024-04-26T20:17:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01ad810 con 0x55acf3cd8730 2024-04-26T20:17:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.491+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01ab660 con 0x55acf3cd35c0 2024-04-26T20:17:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd35c0 0x55acf3ce4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd35c0 0x55acf3ce4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40314/0 (socket says 172.21.15.131:40314) 2024-04-26T20:17:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 -- 172.21.15.131:0/3003342046 learned_addr learned my addr 172.21.15.131:0/3003342046 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8cf5c00640 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd8730 0x55acf3cdd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 -- 172.21.15.131:0/3003342046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd8730 msgr2=0x55acf3cdd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd8730 0x55acf3cdd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 -- 172.21.15.131:0/3003342046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 msgr2=0x55acf3cdbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 -- 172.21.15.131:0/3003342046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf3a15620 con 0x55acf3cd35c0 2024-04-26T20:17:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd35c0 0x55acf3ce4260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8010930 tx=0x7f8cd8009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abfc42fc4d350e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.492+0000 7f8ce5600640 1 -- 172.21.15.131:0/3003342046 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd802e030 con 0x55acf3cd35c0 2024-04-26T20:17:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8ce5600640 1 -- 172.21.15.131:0/3003342046 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd80054f0 con 0x55acf3cd35c0 2024-04-26T20:17:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8ce5600640 1 -- 172.21.15.131:0/3003342046 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8016050 con 0x55acf3cd35c0 2024-04-26T20:17:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 -- 172.21.15.131:0/3003342046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd35c0 msgr2=0x55acf3ce4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd35c0 0x55acf3ce4260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8010930 tx=0x7f8cd8009370 comp rx=0 tx=0).stop 2024-04-26T20:17:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 -- 172.21.15.131:0/3003342046 shutdown_connections 2024-04-26T20:17:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd8730 0x55acf3cdd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd35c0 0x55acf3ce4260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/3003342046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3cdbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 -- 172.21.15.131:0/3003342046 >> 172.21.15.131:0/3003342046 conn(0x55acf3cce240 msgr2=0x55acf3cd7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 -- 172.21.15.131:0/3003342046 shutdown_connections 2024-04-26T20:17:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.493+0000 7f8cfaa79740 1 -- 172.21.15.131:0/3003342046 wait complete. 2024-04-26T20:17:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 -- start start 2024-04-26T20:17:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 Processor -- start 2024-04-26T20:17:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3ce1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3ce1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3ce2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01abcd0 con 0x55acf3cd2bd0 2024-04-26T20:17:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01ad140 con 0x55acf3cd35c0 2024-04-26T20:17:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01ac3a0 con 0x55acf3cd8730 2024-04-26T20:17:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.494+0000 7f8cfaa79740 10 client.0 _ll_register_callbacks cb 0x7f8ce0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:17:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3ce1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3ce1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43926/0 (socket says 172.21.15.131:43926) 2024-04-26T20:17:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 -- 172.21.15.131:0/38787334 learned_addr learned my addr 172.21.15.131:0/38787334 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5c00640 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3ce2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3ce1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 -- 172.21.15.131:0/38787334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 msgr2=0x55acf3ce2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3ce2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 -- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 msgr2=0x55acf3ce1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3ce1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 -- 172.21.15.131:0/38787334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf3bf1cb0 con 0x55acf3cd2bd0 2024-04-26T20:17:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cf5200640 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3ce1a60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ce002a9e0 tx=0x7f8ce0031e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea86c84e4379933f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.495+0000 7f8cdea00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:17:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cdea00640 1 -- 172.21.15.131:0/38787334 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0035040 con 0x55acf3cd2bd0 2024-04-26T20:17:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cdea00640 1 -- 172.21.15.131:0/38787334 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8ce00317b0 con 0x55acf3cd2bd0 2024-04-26T20:17:46.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd80054f0 con 0x55acf3cd2bd0 2024-04-26T20:17:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cdea00640 1 -- 172.21.15.131:0/38787334 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ce0003700 con 0x55acf3cd2bd0 2024-04-26T20:17:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 10 client.15618 fetch_fsmap learned FSMap version 27 2024-04-26T20:17:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acf3ce4600 con 0x55acf3cd2bd0 2024-04-26T20:17:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cdea00640 1 -- 172.21.15.131:0/38787334 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd80054f0 con 0x55acf3cd2bd0 2024-04-26T20:17:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cdea00640 1 -- 172.21.15.131:0/38787334 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8ce0068030 con 0x55acf3cd2bd0 2024-04-26T20:17:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:17:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:17:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 20 client.15618 populate_metadata read hostname 'smithi131' 2024-04-26T20:17:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3d4f680 0x55acf3d51aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 4 client.15618 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:17:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.496+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acf3c3ee00 con 0x55acf3d4f680 2024-04-26T20:17:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.497+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3d4f680 0x55acf3d51aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.497+0000 7f8ce7e00640 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3d4f680 0x55acf3d51aa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65cabc6eab08b419 server_cookie=8e1adc75d296796b in_seq=0 out_seq=0 2024-04-26T20:17:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.497+0000 7f8cdea00640 10 client.15618 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cdea00640 1 -- 172.21.15.131:0/38787334 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acf3c3ee00 con 0x55acf3d4f680 2024-04-26T20:17:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cdea00640 10 client.15618 handle_command_reply: tid=0 2024-04-26T20:17:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 client.15618 shutdown 2024-04-26T20:17:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 msgr2=0x55acf3ce1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3ce1a60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ce002a9e0 tx=0x7f8ce0031e00 comp rx=0 tx=0).stop 2024-04-26T20:17:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 shutdown_connections 2024-04-26T20:17:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55acf3d4f680 0x55acf3d51aa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55acf3cd8730 0x55acf3ce2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf3cd35c0 0x55acf3ce1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 --2- 172.21.15.131:0/38787334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55acf3cd2bd0 0x55acf3ce1a60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 >> 172.21.15.131:0/38787334 conn(0x55acf3cce240 msgr2=0x55acf3cd0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cdea00640 0 client.15618 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:17:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 shutdown_connections 2024-04-26T20:17:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 1 -- 172.21.15.131:0/38787334 wait complete. 2024-04-26T20:17:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.501+0000 7f8cfaa79740 20 client.15618 trim_cache size 0 max 16384 2024-04-26T20:17:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:17:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:17:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:17:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:17:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cd40a3cf0 msgr2=0x7f8cd40a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cd40a3cf0 0x7f8cd40a61b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0006010 tx=0x7f8ce0005a20 comp rx=0 tx=0).stop 2024-04-26T20:17:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cf01034f0 msgr2=0x7f8cf01a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cf01034f0 0x7f8cf01a2280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009fd0 tx=0x7f8ce800ef10 comp rx=0 tx=0).stop 2024-04-26T20:17:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 shutdown_connections 2024-04-26T20:17:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cf01034f0 0x7f8cf01a2280 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8cd40a3cf0 0x7f8cd40a61b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf0102b30 0x7f8cf019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 --2- 172.21.15.131:0/761983704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cf0101940 0x7f8cf019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 >> 172.21.15.131:0/761983704 conn(0x7f8cf00fceb0 msgr2=0x7f8cf010a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 shutdown_connections 2024-04-26T20:17:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:18.502+0000 7f8cf7c00640 1 -- 172.21.15.131:0/761983704 wait complete. 2024-04-26T20:17:50.059 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:19 smithi131 ceph-mon[27919]: pgmap v599: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:50.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:21 smithi131 ceph-mon[27919]: pgmap v600: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:50.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:23 smithi131 ceph-mon[27919]: pgmap v601: 129 pgs: 129 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:17:50.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:25 smithi131 ceph-mon[27919]: pgmap v602: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:50.450 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:27 smithi131 ceph-mon[27919]: pgmap v603: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:50.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:29 smithi131 ceph-mon[27919]: pgmap v604: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:50.712 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:34 smithi148 ceph-mon[32281]: pgmap v637: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:50.836 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:36 smithi148 ceph-mon[32281]: pgmap v638: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:50.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:38 smithi148 ceph-mon[32281]: pgmap v639: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:50.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:40 smithi148 ceph-mon[32281]: pgmap v640: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:17:50.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:41 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:17:50.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:42 smithi148 ceph-mon[32281]: pgmap v641: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:51.140 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:43 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:17:51.231 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:43 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:17:51.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:43 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:17:51.520 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:45 smithi148 ceph-mon[32281]: pgmap v642: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:51.616 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:47 smithi148 ceph-mon[32281]: pgmap v643: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:51.623 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:49 smithi148 ceph-mon[32281]: pgmap v644: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:51.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:34 smithi157 ceph-mon[32703]: pgmap v637: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:51.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:36 smithi157 ceph-mon[32703]: pgmap v638: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:51.872 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:38 smithi157 ceph-mon[32703]: pgmap v639: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:51.975 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:40 smithi157 ceph-mon[32703]: pgmap v640: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:17:51.983 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:41 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:17:51.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:42 smithi157 ceph-mon[32703]: pgmap v641: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:51.995 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:43 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:17:52.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:43 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:17:52.102 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:43 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:17:52.108 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:45 smithi157 ceph-mon[32703]: pgmap v642: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:17:52.114 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:47 smithi157 ceph-mon[32703]: pgmap v643: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:17:52.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:49 smithi157 ceph-mon[32703]: pgmap v644: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:52.131 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:51 smithi148 ceph-mon[32281]: pgmap v645: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:52.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:51 smithi157 ceph-mon[32703]: pgmap v645: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:17:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 -- 172.21.15.131:0/98004746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 msgr2=0x7f991c1093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 --2- 172.21.15.131:0/98004746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 0x7f991c1093e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f990c009900 tx=0x7f990c02a410 comp rx=0 tx=0).stop 2024-04-26T20:17:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 -- 172.21.15.131:0/98004746 shutdown_connections 2024-04-26T20:17:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 --2- 172.21.15.131:0/98004746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c10ab70 0x7f991c111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 --2- 172.21.15.131:0/98004746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f991c10a1b0 0x7f991c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 --2- 172.21.15.131:0/98004746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 0x7f991c1093e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 -- 172.21.15.131:0/98004746 >> 172.21.15.131:0/98004746 conn(0x7f991c077f80 msgr2=0x7f991c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 -- 172.21.15.131:0/98004746 shutdown_connections 2024-04-26T20:17:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.646+0000 7f9922400640 1 -- 172.21.15.131:0/98004746 wait complete. 2024-04-26T20:17:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.647+0000 7f9922400640 1 Processor -- start 2024-04-26T20:17:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.647+0000 7f9922400640 1 -- start start 2024-04-26T20:17:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f9922400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 0x7f991c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f9922400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c10a1b0 0x7f991c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f9922400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f991c10ab70 0x7f991c1a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f9922400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f991c114b00 con 0x7f991c108fc0 2024-04-26T20:17:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f9922400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f991c114980 con 0x7f991c10a1b0 2024-04-26T20:17:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f9922400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f991c114c80 con 0x7f991c10ab70 2024-04-26T20:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 0x7f991c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 0x7f991c19f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50382/0 (socket says 172.21.15.131:50382) 2024-04-26T20:17:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991b400640 1 -- 172.21.15.131:0/4031953164 learned_addr learned my addr 172.21.15.131:0/4031953164 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991be00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f991c10ab70 0x7f991c1a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f9913e00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c10a1b0 0x7f991c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991be00640 1 -- 172.21.15.131:0/4031953164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c10a1b0 msgr2=0x7f991c19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991be00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c10a1b0 0x7f991c19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991be00640 1 -- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 msgr2=0x7f991c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991be00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 0x7f991c19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.648+0000 7f991be00640 1 -- 172.21.15.131:0/4031953164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c1a6d70 con 0x7f991c10ab70 2024-04-26T20:17:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.649+0000 7f991be00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f991c10ab70 0x7f991c1a6700 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9914009fd0 tx=0x7f991400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.649+0000 7f9918c00640 1 -- 172.21.15.131:0/4031953164 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991400ce20 con 0x7f991c10ab70 2024-04-26T20:17:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.649+0000 7f9918c00640 1 -- 172.21.15.131:0/4031953164 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914005020 con 0x7f991c10ab70 2024-04-26T20:17:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.649+0000 7f9918c00640 1 -- 172.21.15.131:0/4031953164 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914010690 con 0x7f991c10ab70 2024-04-26T20:17:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.649+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c1a7060 con 0x7f991c10ab70 2024-04-26T20:17:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.649+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c1a7560 con 0x7f991c10ab70 2024-04-26T20:17:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.653+0000 7f992522c740 1 Processor -- start 2024-04-26T20:17:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f9918c00640 1 -- 172.21.15.131:0/4031953164 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f99140054d0 con 0x7f991c10ab70 2024-04-26T20:17:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f992522c740 1 -- start start 2024-04-26T20:17:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f992522c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbddfbd0 0x55cebbddfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f992522c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbde50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f992522c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbde7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f992522c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cebbd44380 con 0x55cebbde05c0 2024-04-26T20:17:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f992522c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cebbce0c30 con 0x55cebbddfbd0 2024-04-26T20:17:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f992522c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cebbd40340 con 0x55cebbde5730 2024-04-26T20:17:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.654+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbde50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f9913e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbde7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbde50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50396/0 (socket says 172.21.15.131:50396) 2024-04-26T20:17:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 -- 172.21.15.131:0/1490173696 learned_addr learned my addr 172.21.15.131:0/1490173696 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:17:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f9913e00640 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbddfbd0 0x55cebbddfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 -- 172.21.15.131:0/1490173696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 msgr2=0x55cebbde7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbde7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 -- 172.21.15.131:0/1490173696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbddfbd0 msgr2=0x55cebbddfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:17:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbddfbd0 0x55cebbddfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 -- 172.21.15.131:0/1490173696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebbd02940 con 0x55cebbde05c0 2024-04-26T20:17:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f9913e00640 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbde7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:17:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.655+0000 7f991b400640 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbde50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f990c005e20 tx=0x7f990c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be864a874af7b24a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f9910c00640 1 -- 172.21.15.131:0/1490173696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c035030 con 0x55cebbde05c0 2024-04-26T20:17:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f9910c00640 1 -- 172.21.15.131:0/1490173696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990c040070 con 0x55cebbde05c0 2024-04-26T20:17:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f9918c00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f98fc0a3cf0 0x7f98fc0a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:17:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f9910c00640 1 -- 172.21.15.131:0/1490173696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c003900 con 0x55cebbde05c0 2024-04-26T20:17:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f9918c00640 1 -- 172.21.15.131:0/4031953164 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f99140d10a0 con 0x7f991c10ab70 2024-04-26T20:17:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f9913e00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f98fc0a3cf0 0x7f98fc0a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:17:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f992522c740 1 -- 172.21.15.131:0/1490173696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 msgr2=0x55cebbde50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:17:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.656+0000 7f992522c740 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbde50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f990c005e20 tx=0x7f990c005e50 comp rx=0 tx=0).stop 2024-04-26T20:17:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 -- 172.21.15.131:0/1490173696 shutdown_connections 2024-04-26T20:17:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbde7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbde50c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 --2- 172.21.15.131:0/1490173696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbddfbd0 0x55cebbddfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:17:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 -- 172.21.15.131:0/1490173696 >> 172.21.15.131:0/1490173696 conn(0x55cebbddb240 msgr2=0x55cebbddb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:17:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 -- 172.21.15.131:0/1490173696 shutdown_connections 2024-04-26T20:17:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 -- 172.21.15.131:0/1490173696 wait complete. 2024-04-26T20:17:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f9913e00640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f98fc0a3cf0 0x7f98fc0a61b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55cebbde0e20 tx=0x7f9904004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:17:59.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 -- start start 2024-04-26T20:17:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.657+0000 7f992522c740 1 Processor -- start 2024-04-26T20:17:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f992522c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbddfbd0 0x55cebbde9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f992522c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbdea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f992522c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde5730 0x55cebbdf12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f992522c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cebbde8680 con 0x55cebbde05c0 2024-04-26T20:18:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f992522c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cebbccbc60 con 0x55cebbde5730 2024-04-26T20:18:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f992522c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cebbde8800 con 0x55cebbddfbd0 2024-04-26T20:18:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f992522c740 10 client.0 _ll_register_callbacks cb 0x55cebbcf7e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:18:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f9913e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbdea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f991b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbddfbd0 0x55cebbde9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f9913e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbdea2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50408/0 (socket says 172.21.15.131:50408) 2024-04-26T20:18:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f991b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbddfbd0 0x55cebbde9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50952/0 (socket says 172.21.15.131:50952) 2024-04-26T20:18:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f991be00640 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde5730 0x55cebbdf12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.658+0000 7f9913e00640 1 -- 172.21.15.131:0/914457660 learned_addr learned my addr 172.21.15.131:0/914457660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f991be00640 1 -- 172.21.15.131:0/914457660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbddfbd0 msgr2=0x55cebbde9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f991be00640 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbddfbd0 0x55cebbde9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f991be00640 1 -- 172.21.15.131:0/914457660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 msgr2=0x55cebbdea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f991be00640 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbdea2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f991be00640 1 -- 172.21.15.131:0/914457660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebbcda510 con 0x55cebbde5730 2024-04-26T20:18:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f991b400640 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbddfbd0 0x55cebbde9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f991be00640 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde5730 0x55cebbdf12e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99140ce600 tx=0x7f991409ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37e6af8734aca3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f9909600640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:18:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f9909600640 1 -- 172.21.15.131:0/914457660 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140a55b0 con 0x55cebbde5730 2024-04-26T20:18:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f990c040070 con 0x55cebbde5730 2024-04-26T20:18:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f9909600640 1 -- 172.21.15.131:0/914457660 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f99140a5750 con 0x55cebbde5730 2024-04-26T20:18:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f9909600640 1 -- 172.21.15.131:0/914457660 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99140b6200 con 0x55cebbde5730 2024-04-26T20:18:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f992522c740 10 client.25748 fetch_fsmap learned FSMap version 27 2024-04-26T20:18:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.659+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cebba56210 con 0x55cebbde5730 2024-04-26T20:18:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f9909600640 1 -- 172.21.15.131:0/914457660 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990c040070 con 0x55cebbde5730 2024-04-26T20:18:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f9909600640 1 -- 172.21.15.131:0/914457660 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f991409bc90 con 0x55cebbde5730 2024-04-26T20:18:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f992522c740 10 client.25748 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:18:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f992522c740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:18:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f992522c740 20 client.25748 populate_metadata read hostname 'smithi131' 2024-04-26T20:18:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f992522c740 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbdf58a0 0x55cebbdf7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f992522c740 4 client.25748 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:18:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cebbd51f90 con 0x55cebbdf58a0 2024-04-26T20:18:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.660+0000 7f991b400640 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbdf58a0 0x55cebbdf7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.661+0000 7f991b400640 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbdf58a0 0x55cebbdf7c90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd83200f619847 server_cookie=572114a31d20c016 in_seq=0 out_seq=0 2024-04-26T20:18:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.661+0000 7f9909600640 10 client.25748 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f9909600640 1 -- 172.21.15.131:0/914457660 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cebbd51f90 con 0x55cebbdf58a0 2024-04-26T20:18:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f9909600640 10 client.25748 handle_command_reply: tid=0 2024-04-26T20:18:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 client.25748 shutdown 2024-04-26T20:18:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde5730 msgr2=0x55cebbdf12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde5730 0x55cebbdf12e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99140ce600 tx=0x7f991409ce90 comp rx=0 tx=0).stop 2024-04-26T20:18:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 shutdown_connections 2024-04-26T20:18:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbdf58a0 0x55cebbdf7c90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde5730 0x55cebbdf12e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbde05c0 0x55cebbdea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f9909600640 0 client.25748 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 --2- 172.21.15.131:0/914457660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbddfbd0 0x55cebbde9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 >> 172.21.15.131:0/914457660 conn(0x55cebbddb240 msgr2=0x55cebbdda440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 shutdown_connections 2024-04-26T20:18:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 1 -- 172.21.15.131:0/914457660 wait complete. 2024-04-26T20:18:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.662+0000 7f992522c740 20 client.25748 trim_cache size 0 max 16384 2024-04-26T20:18:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.678+0000 7f992522c740 1 Processor -- start 2024-04-26T20:18:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.678+0000 7f992522c740 1 -- start start 2024-04-26T20:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f992522c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdde900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f992522c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdf12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f992522c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdfc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f992522c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1b44a0 con 0x55cebbddfbd0 2024-04-26T20:18:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f992522c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1b1d70 con 0x55cebbde05c0 2024-04-26T20:18:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f992522c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1afbc0 con 0x55cebbde5730 2024-04-26T20:18:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdde900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdde900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50420/0 (socket says 172.21.15.131:50420) 2024-04-26T20:18:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 -- 172.21.15.131:0/2806343453 learned_addr learned my addr 172.21.15.131:0/2806343453 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991be00640 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdfc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f9913e00640 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdf12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 -- 172.21.15.131:0/2806343453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 msgr2=0x55cebbdfc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdfc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 -- 172.21.15.131:0/2806343453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 msgr2=0x55cebbdf12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdf12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991b400640 1 -- 172.21.15.131:0/2806343453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebbdd6940 con 0x55cebbddfbd0 2024-04-26T20:18:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f991be00640 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdfc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.679+0000 7f9913e00640 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdf12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.680+0000 7f991b400640 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdde900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f990c005df0 tx=0x7f990c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a63e58e9f599998 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.680+0000 7f990be00640 1 -- 172.21.15.131:0/2806343453 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c035040 con 0x55cebbddfbd0 2024-04-26T20:18:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.680+0000 7f990be00640 1 -- 172.21.15.131:0/2806343453 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990c04b030 con 0x55cebbddfbd0 2024-04-26T20:18:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.680+0000 7f990be00640 1 -- 172.21.15.131:0/2806343453 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c0547f0 con 0x55cebbddfbd0 2024-04-26T20:18:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 -- 172.21.15.131:0/2806343453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 msgr2=0x55cebbdde900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdde900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f990c005df0 tx=0x7f990c02a410 comp rx=0 tx=0).stop 2024-04-26T20:18:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 -- 172.21.15.131:0/2806343453 shutdown_connections 2024-04-26T20:18:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdfc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdf12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 --2- 172.21.15.131:0/2806343453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdde900 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 -- 172.21.15.131:0/2806343453 >> 172.21.15.131:0/2806343453 conn(0x55cebbddb240 msgr2=0x55cebbdde1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 -- 172.21.15.131:0/2806343453 shutdown_connections 2024-04-26T20:18:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 -- 172.21.15.131:0/2806343453 wait complete. 2024-04-26T20:18:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 -- start start 2024-04-26T20:18:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.681+0000 7f992522c740 1 Processor -- start 2024-04-26T20:18:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f992522c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdeee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f992522c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f992522c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdf0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f992522c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1b0230 con 0x55cebbddfbd0 2024-04-26T20:18:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f992522c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1b16a0 con 0x55cebbde05c0 2024-04-26T20:18:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f992522c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1b0900 con 0x55cebbde5730 2024-04-26T20:18:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f992522c740 10 client.0 _ll_register_callbacks cb 0x7f9914099070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:18:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdeee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f991b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdeee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50424/0 (socket says 172.21.15.131:50424) 2024-04-26T20:18:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f991b400640 1 -- 172.21.15.131:0/1561588090 learned_addr learned my addr 172.21.15.131:0/1561588090 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:07.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f9913e00640 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f991be00640 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdf0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f991b400640 1 -- 172.21.15.131:0/1561588090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 msgr2=0x55cebbdf0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.682+0000 7f991b400640 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdf0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f991b400640 1 -- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 msgr2=0x55cebbdef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f991b400640 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f991b400640 1 -- 172.21.15.131:0/1561588090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebbb22620 con 0x55cebbddfbd0 2024-04-26T20:18:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f991b400640 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdeee70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f990c005ab0 tx=0x7f990c02aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef353be9630f9580 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f9909600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:18:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f9909600640 1 -- 172.21.15.131:0/1561588090 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c000e60 con 0x55cebbddfbd0 2024-04-26T20:18:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f9909600640 1 -- 172.21.15.131:0/1561588090 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f990c0545d0 con 0x55cebbddfbd0 2024-04-26T20:18:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f990c04b030 con 0x55cebbddfbd0 2024-04-26T20:18:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f9909600640 1 -- 172.21.15.131:0/1561588090 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f990c06c050 con 0x55cebbddfbd0 2024-04-26T20:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f992522c740 10 client.15636 fetch_fsmap learned FSMap version 27 2024-04-26T20:18:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.683+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cebbcfecb0 con 0x55cebbddfbd0 2024-04-26T20:18:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f9909600640 1 -- 172.21.15.131:0/1561588090 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990c04b030 con 0x55cebbddfbd0 2024-04-26T20:18:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f9909600640 1 -- 172.21.15.131:0/1561588090 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f990c045700 con 0x55cebbddfbd0 2024-04-26T20:18:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f992522c740 10 client.15636 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:18:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f992522c740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:18:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f992522c740 20 client.15636 populate_metadata read hostname 'smithi131' 2024-04-26T20:18:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f992522c740 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbe5c2d0 0x55cebbe5e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f992522c740 4 client.15636 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:18:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cebbd4be00 con 0x55cebbe5c2d0 2024-04-26T20:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.684+0000 7f9913e00640 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbe5c2d0 0x55cebbe5e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.685+0000 7f9913e00640 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbe5c2d0 0x55cebbe5e6c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8799560ea9991a3 server_cookie=b8b23247933fe9cf in_seq=0 out_seq=0 2024-04-26T20:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.685+0000 7f9909600640 10 client.15636 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f9909600640 1 -- 172.21.15.131:0/1561588090 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cebbd4be00 con 0x55cebbe5c2d0 2024-04-26T20:18:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f9909600640 10 client.15636 handle_command_reply: tid=0 2024-04-26T20:18:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 client.15636 shutdown 2024-04-26T20:18:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 msgr2=0x55cebbdeee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdeee70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f990c005ab0 tx=0x7f990c02aba0 comp rx=0 tx=0).stop 2024-04-26T20:18:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 shutdown_connections 2024-04-26T20:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55cebbe5c2d0 0x55cebbe5e6c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cebbde5730 0x55cebbdf0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cebbde05c0 0x55cebbdef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 --2- 172.21.15.131:0/1561588090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebbddfbd0 0x55cebbdeee70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 >> 172.21.15.131:0/1561588090 conn(0x55cebbddb240 msgr2=0x55cebbdd9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f9909600640 0 client.15636 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 shutdown_connections 2024-04-26T20:18:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 1 -- 172.21.15.131:0/1561588090 wait complete. 2024-04-26T20:18:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.689+0000 7f992522c740 20 client.15636 trim_cache size 0 max 16384 2024-04-26T20:18:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:18:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:18:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:18:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f98fc0a3cf0 msgr2=0x7f98fc0a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f98fc0a3cf0 0x7f98fc0a61b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55cebbde0e20 tx=0x7f9904004730 comp rx=0 tx=0).stop 2024-04-26T20:18:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f991c10ab70 msgr2=0x7f991c1a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f991c10ab70 0x7f991c1a6700 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9914009fd0 tx=0x7f991400ef10 comp rx=0 tx=0).stop 2024-04-26T20:18:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 shutdown_connections 2024-04-26T20:18:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f98fc0a3cf0 0x7f98fc0a61b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f991c10ab70 0x7f991c1a6700 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c10a1b0 0x7f991c19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 --2- 172.21.15.131:0/4031953164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f991c108fc0 0x7f991c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 >> 172.21.15.131:0/4031953164 conn(0x7f991c077f80 msgr2=0x7f991c10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 shutdown_connections 2024-04-26T20:18:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:31.690+0000 7f9922400640 1 -- 172.21.15.131:0/4031953164 wait complete. 2024-04-26T20:18:11.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:53 smithi148 ceph-mon[32281]: pgmap v646: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:11.991 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:55 smithi148 ceph-mon[32281]: pgmap v647: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:11.998 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:57 smithi148 ceph-mon[32281]: pgmap v648: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:12.082 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:17:59 smithi148 ceph-mon[32281]: pgmap v649: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:12.137 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:01 smithi148 ceph-mon[32281]: pgmap v650: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:12.282 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:03 smithi148 ceph-mon[32281]: pgmap v651: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:12.289 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:05 smithi148 ceph-mon[32281]: pgmap v652: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:12.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:07 smithi148 ceph-mon[32281]: pgmap v653: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:12.301 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:09 smithi148 ceph-mon[32281]: pgmap v654: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:18:12.453 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:53 smithi157 ceph-mon[32703]: pgmap v646: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:12.545 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:55 smithi157 ceph-mon[32703]: pgmap v647: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:12.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:57 smithi157 ceph-mon[32703]: pgmap v648: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:12.612 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:17:59 smithi157 ceph-mon[32703]: pgmap v649: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:12.668 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:01 smithi157 ceph-mon[32703]: pgmap v650: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:12.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:03 smithi157 ceph-mon[32703]: pgmap v651: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:12.809 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:05 smithi157 ceph-mon[32703]: pgmap v652: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:12.816 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:07 smithi157 ceph-mon[32703]: pgmap v653: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:12.825 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:09 smithi157 ceph-mon[32703]: pgmap v654: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:18:12.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:31 smithi131 ceph-mon[27919]: pgmap v605: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:13.055 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:33 smithi131 ceph-mon[27919]: pgmap v606: 129 pgs: 129 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:13.061 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:35 smithi131 ceph-mon[27919]: pgmap v607: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:18:13.112 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:37 smithi131 ceph-mon[27919]: pgmap v608: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:13.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:39 smithi131 ceph-mon[27919]: pgmap v609: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:13.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:40 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:18:13.316 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: pgmap v610: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:13.426 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:13.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:13.687 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:13.722 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:13.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:13.863 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:13.953 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:13.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:18:14.041 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:18:14.145 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:18:14.151 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:18:14.254 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:42 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:18:14.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:42 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:18:14.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:42 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:18:14.474 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:42 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:18:14.481 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:43 smithi131 ceph-mon[27919]: pgmap v611: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:14.490 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:11 smithi148 ceph-mon[32281]: pgmap v655: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:14.572 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:13 smithi148 ceph-mon[32281]: pgmap v656: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:14.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:11 smithi157 ceph-mon[32703]: pgmap v655: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:14.822 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:13 smithi157 ceph-mon[32703]: pgmap v656: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.830+0000 7f69a0c00640 1 -- 172.21.15.131:0/3297167685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c108fc0 msgr2=0x7f699c1093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.830+0000 7f69a0c00640 1 --2- 172.21.15.131:0/3297167685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c108fc0 0x7f699c1093e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f698c009900 tx=0x7f698c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:18:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.830+0000 7f69a0c00640 1 -- 172.21.15.131:0/3297167685 shutdown_connections 2024-04-26T20:18:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.830+0000 7f69a0c00640 1 --2- 172.21.15.131:0/3297167685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f699c10ab70 0x7f699c111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.831+0000 7f69a0c00640 1 --2- 172.21.15.131:0/3297167685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c10a1b0 0x7f699c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.831+0000 7f69a0c00640 1 --2- 172.21.15.131:0/3297167685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c108fc0 0x7f699c1093e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.831+0000 7f69a0c00640 1 -- 172.21.15.131:0/3297167685 >> 172.21.15.131:0/3297167685 conn(0x7f699c077f80 msgr2=0x7f699c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.831+0000 7f69a0c00640 1 -- 172.21.15.131:0/3297167685 shutdown_connections 2024-04-26T20:18:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.831+0000 7f69a0c00640 1 -- 172.21.15.131:0/3297167685 wait complete. 2024-04-26T20:18:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 Processor -- start 2024-04-26T20:18:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 -- start start 2024-04-26T20:18:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c108fc0 0x7f699c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c10a1b0 0x7f699c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f699c10ab70 0x7f699c1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114b50 con 0x7f699c10a1b0 2024-04-26T20:18:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1149d0 con 0x7f699c10ab70 2024-04-26T20:18:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f69a0c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114cd0 con 0x7f699c108fc0 2024-04-26T20:18:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f6993e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c10a1b0 0x7f699c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f6999e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c108fc0 0x7f699c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f6993e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c10a1b0 0x7f699c19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54638/0 (socket says 172.21.15.131:54638) 2024-04-26T20:18:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f6993e00640 1 -- 172.21.15.131:0/1478887939 learned_addr learned my addr 172.21.15.131:0/1478887939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f699a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f699c10ab70 0x7f699c1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.832+0000 7f6999e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c108fc0 0x7f699c19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47826/0 (socket says 172.21.15.131:47826) 2024-04-26T20:18:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.833+0000 7f6993e00640 1 -- 172.21.15.131:0/1478887939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c108fc0 msgr2=0x7f699c19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.833+0000 7f6993e00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c108fc0 0x7f699c19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.833+0000 7f6993e00640 1 -- 172.21.15.131:0/1478887939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f699c10ab70 msgr2=0x7f699c1a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.833+0000 7f6993e00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f699c10ab70 0x7f699c1a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.833+0000 7f6993e00640 1 -- 172.21.15.131:0/1478887939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1a6d90 con 0x7f699c10a1b0 2024-04-26T20:18:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.833+0000 7f6993e00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c10a1b0 0x7f699c19f6b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f698400e9f0 tx=0x7f698400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.833+0000 7f6992a00640 1 -- 172.21.15.131:0/1478887939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698400ccc0 con 0x7f699c10a1b0 2024-04-26T20:18:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.834+0000 7f6992a00640 1 -- 172.21.15.131:0/1478887939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698400ce60 con 0x7f699c10a1b0 2024-04-26T20:18:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.834+0000 7f6992a00640 1 -- 172.21.15.131:0/1478887939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984010430 con 0x7f699c10a1b0 2024-04-26T20:18:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.834+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1a7080 con 0x7f699c10a1b0 2024-04-26T20:18:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.834+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a75d0 con 0x7f699c10a1b0 2024-04-26T20:18:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 Processor -- start 2024-04-26T20:18:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 -- start start 2024-04-26T20:18:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891dd2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891dd80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891ddabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555891d37380 con 0x555891dd2bd0 2024-04-26T20:18:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555891cd3c30 con 0x555891dd35c0 2024-04-26T20:18:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f69a3985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555891d33340 con 0x555891dd8730 2024-04-26T20:18:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.838+0000 7f6992a00640 1 -- 172.21.15.131:0/1478887939 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f69840046c0 con 0x7f699c10a1b0 2024-04-26T20:18:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891dd2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891dd2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54644/0 (socket says 172.21.15.131:54644) 2024-04-26T20:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 -- 172.21.15.131:0/366333765 learned_addr learned my addr 172.21.15.131:0/366333765 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f6999e00640 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891dd80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f6993e00640 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891ddabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 -- 172.21.15.131:0/366333765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 msgr2=0x555891ddabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891ddabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 -- 172.21.15.131:0/366333765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 msgr2=0x555891dd80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891dd80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f699a800640 1 -- 172.21.15.131:0/366333765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555891cf5940 con 0x555891dd2bd0 2024-04-26T20:18:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.839+0000 7f6993e00640 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891ddabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.840+0000 7f699a800640 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891dd2ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6994011440 tx=0x7f6994011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796467f2f8392a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.840+0000 7f698b000640 1 -- 172.21.15.131:0/366333765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994003c30 con 0x555891dd2bd0 2024-04-26T20:18:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.840+0000 7f698b000640 1 -- 172.21.15.131:0/366333765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6994003dd0 con 0x555891dd2bd0 2024-04-26T20:18:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.840+0000 7f698b000640 1 -- 172.21.15.131:0/366333765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400fae0 con 0x555891dd2bd0 2024-04-26T20:18:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.840+0000 7f6992a00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69800a3d70 0x7f69800a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.840+0000 7f6992a00640 1 -- 172.21.15.131:0/1478887939 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f69840d0d20 con 0x7f699c10a1b0 2024-04-26T20:18:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.840+0000 7f6999e00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69800a3d70 0x7f69800a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 -- 172.21.15.131:0/366333765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 msgr2=0x555891dd2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891dd2ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6994011440 tx=0x7f6994011900 comp rx=0 tx=0).stop 2024-04-26T20:18:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 -- 172.21.15.131:0/366333765 shutdown_connections 2024-04-26T20:18:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891ddabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891dd80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 --2- 172.21.15.131:0/366333765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891dd2ff0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 -- 172.21.15.131:0/366333765 >> 172.21.15.131:0/366333765 conn(0x555891dce240 msgr2=0x555891dce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 -- 172.21.15.131:0/366333765 shutdown_connections 2024-04-26T20:18:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f69a3985740 1 -- 172.21.15.131:0/366333765 wait complete. 2024-04-26T20:18:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.841+0000 7f6999e00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69800a3d70 0x7f69800a6230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f698c005ab0 tx=0x7f698c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 -- start start 2024-04-26T20:18:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 Processor -- start 2024-04-26T20:18:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891ddce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891ddd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891de4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555891ddb6e0 con 0x555891dd2bd0 2024-04-26T20:18:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555891cbec60 con 0x555891dd35c0 2024-04-26T20:18:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555891ddb860 con 0x555891dd8730 2024-04-26T20:18:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.842+0000 7f69a3985740 10 client.0 _ll_register_callbacks cb 0x555891ceae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:18:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6999e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891ddce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6993e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891ddd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6999e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891ddce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54650/0 (socket says 172.21.15.131:54650) 2024-04-26T20:18:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6999e00640 1 -- 172.21.15.131:0/1671041346 learned_addr learned my addr 172.21.15.131:0/1671041346 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f699a800640 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891de4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6993e00640 1 -- 172.21.15.131:0/1671041346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 msgr2=0x555891de4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6993e00640 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891de4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6993e00640 1 -- 172.21.15.131:0/1671041346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 msgr2=0x555891ddce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6993e00640 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891ddce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6993e00640 1 -- 172.21.15.131:0/1671041346 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555891ccd510 con 0x555891dd35c0 2024-04-26T20:18:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f6993e00640 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891ddd350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f69840ce280 tx=0x7f698409c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6f153305f80731 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f697f400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:18:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.843+0000 7f697f400640 1 -- 172.21.15.131:0/1671041346 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840a0e00 con 0x555891dd35c0 2024-04-26T20:18:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6994003dd0 con 0x555891dd35c0 2024-04-26T20:18:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f697f400640 1 -- 172.21.15.131:0/1671041346 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f69840a5230 con 0x555891dd35c0 2024-04-26T20:18:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f697f400640 1 -- 172.21.15.131:0/1671041346 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69840a56c0 con 0x555891dd35c0 2024-04-26T20:18:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 10 client.25766 fetch_fsmap learned FSMap version 27 2024-04-26T20:18:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555891a49210 con 0x555891dd35c0 2024-04-26T20:18:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f697f400640 1 -- 172.21.15.131:0/1671041346 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6994003dd0 con 0x555891dd35c0 2024-04-26T20:18:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f697f400640 1 -- 172.21.15.131:0/1671041346 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f698409b910 con 0x555891dd35c0 2024-04-26T20:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 10 client.25766 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:18:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 10 client.25766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:18:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 20 client.25766 populate_metadata read hostname 'smithi131' 2024-04-26T20:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891de8880 0x555891deacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 4 client.25766 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:18:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.844+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555891d44f90 con 0x555891de8880 2024-04-26T20:18:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.845+0000 7f6999e00640 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891de8880 0x555891deacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.845+0000 7f6999e00640 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891de8880 0x555891deacb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e70b00ff6ad82b0 server_cookie=985e89e0ebe465e0 in_seq=0 out_seq=0 2024-04-26T20:18:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.845+0000 7f697f400640 10 client.25766 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.846+0000 7f697f400640 1 -- 172.21.15.131:0/1671041346 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555891d44f90 con 0x555891de8880 2024-04-26T20:18:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.846+0000 7f697f400640 10 client.25766 handle_command_reply: tid=0 2024-04-26T20:18:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.846+0000 7f69a3985740 1 client.25766 shutdown 2024-04-26T20:18:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.846+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 msgr2=0x555891ddd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891ddd350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f69840ce280 tx=0x7f698409c470 comp rx=0 tx=0).stop 2024-04-26T20:18:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 shutdown_connections 2024-04-26T20:18:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891de8880 0x555891deacb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd8730 0x555891de4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f697f400640 0 client.25766 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd35c0 0x555891ddd350 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 --2- 172.21.15.131:0/1671041346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd2bd0 0x555891ddce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 >> 172.21.15.131:0/1671041346 conn(0x555891dce240 msgr2=0x555891dcd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 shutdown_connections 2024-04-26T20:18:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 1 -- 172.21.15.131:0/1671041346 wait complete. 2024-04-26T20:18:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.847+0000 7f69a3985740 20 client.25766 trim_cache size 0 max 16384 2024-04-26T20:18:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.863+0000 7f69a3985740 1 Processor -- start 2024-04-26T20:18:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.863+0000 7f69a3985740 1 -- start start 2024-04-26T20:18:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f69a3985740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd2bd0 0x555891dd47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f69a3985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd35c0 0x555891dd4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f69a3985740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd8730 0x555891dd11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f69a3985740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b4450 con 0x555891dd8730 2024-04-26T20:18:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f69a3985740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b1d20 con 0x555891dd2bd0 2024-04-26T20:18:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f69a3985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1afb70 con 0x555891dd35c0 2024-04-26T20:18:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f699a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd8730 0x555891dd11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f6993e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd35c0 0x555891dd4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f699a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd8730 0x555891dd11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54652/0 (socket says 172.21.15.131:54652) 2024-04-26T20:18:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f699a800640 1 -- 172.21.15.131:0/4058629584 learned_addr learned my addr 172.21.15.131:0/4058629584 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.864+0000 7f6999e00640 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd2bd0 0x555891dd47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f6993e00640 1 -- 172.21.15.131:0/4058629584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd2bd0 msgr2=0x555891dd47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f6993e00640 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd2bd0 0x555891dd47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f6993e00640 1 -- 172.21.15.131:0/4058629584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd8730 msgr2=0x555891dd11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f6993e00640 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd8730 0x555891dd11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f6993e00640 1 -- 172.21.15.131:0/4058629584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555891b15620 con 0x555891dd35c0 2024-04-26T20:18:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f699a800640 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd8730 0x555891dd11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f6999e00640 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd2bd0 0x555891dd47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f6993e00640 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd35c0 0x555891dd4d20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f698409ad00 tx=0x7f698409cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32dbf29b40e8520 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f698a600640 1 -- 172.21.15.131:0/4058629584 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984098af0 con 0x555891dd35c0 2024-04-26T20:18:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f698a600640 1 -- 172.21.15.131:0/4058629584 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69840a1f90 con 0x555891dd35c0 2024-04-26T20:18:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.865+0000 7f698a600640 1 -- 172.21.15.131:0/4058629584 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984098900 con 0x555891dd35c0 2024-04-26T20:18:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 -- 172.21.15.131:0/4058629584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd35c0 msgr2=0x555891dd4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd35c0 0x555891dd4d20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f698409ad00 tx=0x7f698409cbd0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 -- 172.21.15.131:0/4058629584 shutdown_connections 2024-04-26T20:18:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd8730 0x555891dd11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd35c0 0x555891dd4d20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 --2- 172.21.15.131:0/4058629584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd2bd0 0x555891dd47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 -- 172.21.15.131:0/4058629584 >> 172.21.15.131:0/4058629584 conn(0x555891dce240 msgr2=0x555891dce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 -- 172.21.15.131:0/4058629584 shutdown_connections 2024-04-26T20:18:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 -- 172.21.15.131:0/4058629584 wait complete. 2024-04-26T20:18:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 -- start start 2024-04-26T20:18:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.866+0000 7f69a3985740 1 Processor -- start 2024-04-26T20:18:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f69a3985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd2bd0 0x555891de0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f69a3985740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd35c0 0x555891de13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f69a3985740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd8730 0x555891de23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f69a3985740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b01e0 con 0x555891dd35c0 2024-04-26T20:18:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f69a3985740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b1650 con 0x555891dd8730 2024-04-26T20:18:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f69a3985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b08b0 con 0x555891dd2bd0 2024-04-26T20:18:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f69a3985740 10 client.0 _ll_register_callbacks cb 0x7f6984009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:18:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f6993e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd35c0 0x555891de13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f6999e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd2bd0 0x555891de0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f6993e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd35c0 0x555891de13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54654/0 (socket says 172.21.15.131:54654) 2024-04-26T20:18:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f6993e00640 1 -- 172.21.15.131:0/1178281101 learned_addr learned my addr 172.21.15.131:0/1178281101 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.867+0000 7f699a800640 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd8730 0x555891de23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f6999e00640 1 -- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd8730 msgr2=0x555891de23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f6999e00640 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd8730 0x555891de23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f6999e00640 1 -- 172.21.15.131:0/1178281101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd35c0 msgr2=0x555891de13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f6999e00640 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd35c0 0x555891de13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f6999e00640 1 -- 172.21.15.131:0/1178281101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555891cf1cb0 con 0x555891dd2bd0 2024-04-26T20:18:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f6993e00640 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd35c0 0x555891de13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f6999e00640 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd2bd0 0x555891de0ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x555891deb560 tx=0x7f698c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95643b377948e953 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f697f400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:18:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f697f400640 1 -- 172.21.15.131:0/1178281101 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c035040 con 0x555891dd2bd0 2024-04-26T20:18:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69840a1f90 con 0x555891dd2bd0 2024-04-26T20:18:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f697f400640 1 -- 172.21.15.131:0/1178281101 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f698c0035f0 con 0x555891dd2bd0 2024-04-26T20:18:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.868+0000 7f697f400640 1 -- 172.21.15.131:0/1178281101 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f698c063050 con 0x555891dd2bd0 2024-04-26T20:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 10 client.25765 fetch_fsmap learned FSMap version 27 2024-04-26T20:18:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555891de3b90 con 0x555891dd2bd0 2024-04-26T20:18:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f697f400640 1 -- 172.21.15.131:0/1178281101 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69840a1f90 con 0x555891dd2bd0 2024-04-26T20:18:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f697f400640 1 -- 172.21.15.131:0/1178281101 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f698c044070 con 0x555891dd2bd0 2024-04-26T20:18:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 10 client.25765 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:18:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 20 client.25765 populate_metadata read hostname 'smithi131' 2024-04-26T20:18:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891e4f4e0 0x555891e51900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 4 client.25765 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:18:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555891d3ee00 con 0x555891e4f4e0 2024-04-26T20:18:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.869+0000 7f6993e00640 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891e4f4e0 0x555891e51900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.870+0000 7f6993e00640 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891e4f4e0 0x555891e51900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ada22246288f66f server_cookie=93e2a3f9396e75dc in_seq=0 out_seq=0 2024-04-26T20:18:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.870+0000 7f697f400640 10 client.25765 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f697f400640 1 -- 172.21.15.131:0/1178281101 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555891d3ee00 con 0x555891e4f4e0 2024-04-26T20:18:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f697f400640 10 client.25765 handle_command_reply: tid=0 2024-04-26T20:18:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 client.25765 shutdown 2024-04-26T20:18:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd2bd0 msgr2=0x555891de0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd2bd0 0x555891de0ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x555891deb560 tx=0x7f698c009260 comp rx=0 tx=0).stop 2024-04-26T20:18:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 shutdown_connections 2024-04-26T20:18:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555891e4f4e0 0x555891e51900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555891dd8730 0x555891de23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555891dd35c0 0x555891de13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 --2- 172.21.15.131:0/1178281101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555891dd2bd0 0x555891de0ea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f697f400640 0 client.25765 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 >> 172.21.15.131:0/1178281101 conn(0x555891dce240 msgr2=0x555891dce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 shutdown_connections 2024-04-26T20:18:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 1 -- 172.21.15.131:0/1178281101 wait complete. 2024-04-26T20:18:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.874+0000 7f69a3985740 20 client.25765 trim_cache size 0 max 16384 2024-04-26T20:18:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:18:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:18:29.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:18:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:18:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69800a3d70 msgr2=0x7f69800a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69800a3d70 0x7f69800a6230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f698c005ab0 tx=0x7f698c005a20 comp rx=0 tx=0).stop 2024-04-26T20:18:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c10a1b0 msgr2=0x7f699c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c10a1b0 0x7f699c19f6b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f698400e9f0 tx=0x7f698400edf0 comp rx=0 tx=0).stop 2024-04-26T20:18:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 shutdown_connections 2024-04-26T20:18:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f699c10ab70 0x7f699c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f699c10a1b0 0x7f699c19f6b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69800a3d70 0x7f69800a6230 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 --2- 172.21.15.131:0/1478887939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c108fc0 0x7f699c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 >> 172.21.15.131:0/1478887939 conn(0x7f699c077f80 msgr2=0x7f699c10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 shutdown_connections 2024-04-26T20:18:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:44.875+0000 7f69a0c00640 1 -- 172.21.15.131:0/1478887939 wait complete. 2024-04-26T20:18:30.878 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:45 smithi131 ceph-mon[27919]: pgmap v612: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:30.934 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:47 smithi131 ceph-mon[27919]: pgmap v613: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:31.047 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:49 smithi131 ceph-mon[27919]: pgmap v614: 129 pgs: 129 active+clean; 666 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:18:31.137 INFO:tasks.workunit.client.0.smithi131.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T20:18:31.247 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:51 smithi131 ceph-mon[27919]: pgmap v615: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:31.360 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:53 smithi131 ceph-mon[27919]: pgmap v616: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:31.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:55 smithi131 ceph-mon[27919]: pgmap v617: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:31.438 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:57 smithi131 ceph-mon[27919]: pgmap v618: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:31.449 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:15 smithi148 ceph-mon[32281]: pgmap v657: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:31.746 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:17 smithi148 ceph-mon[32281]: pgmap v658: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:31.920 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:19 smithi148 ceph-mon[32281]: pgmap v659: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:32.032 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:21 smithi148 ceph-mon[32281]: pgmap v660: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:18:32.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:23 smithi148 ceph-mon[32281]: pgmap v661: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:32.186 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:25 smithi148 ceph-mon[32281]: pgmap v662: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:32.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:27 smithi148 ceph-mon[32281]: pgmap v663: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:32.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:29 smithi148 ceph-mon[32281]: pgmap v664: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:32.373 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:15 smithi157 ceph-mon[32703]: pgmap v657: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:32.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:17 smithi157 ceph-mon[32703]: pgmap v658: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:32.386 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:19 smithi157 ceph-mon[32703]: pgmap v659: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:32.455 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:21 smithi157 ceph-mon[32703]: pgmap v660: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:18:32.462 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:23 smithi157 ceph-mon[32703]: pgmap v661: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:32.514 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:25 smithi157 ceph-mon[32703]: pgmap v662: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:32.605 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:27 smithi157 ceph-mon[32703]: pgmap v663: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:32.612 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:29 smithi157 ceph-mon[32703]: pgmap v664: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:32.618 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:31 smithi157 ceph-mon[32703]: pgmap v665: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:32.625 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:31 smithi148 ceph-mon[32281]: pgmap v665: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 -- 172.21.15.131:0/1752398119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168075390 msgr2=0x7f0168075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 --2- 172.21.15.131:0/1752398119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168075390 0x7f0168075810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0158009930 tx=0x7f015802a420 comp rx=0 tx=0).stop 2024-04-26T20:18:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 -- 172.21.15.131:0/1752398119 shutdown_connections 2024-04-26T20:18:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 --2- 172.21.15.131:0/1752398119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016810ed90 0x7f0168111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 --2- 172.21.15.131:0/1752398119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168075390 0x7f0168075810 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 --2- 172.21.15.131:0/1752398119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168076970 0x7f0168074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 -- 172.21.15.131:0/1752398119 >> 172.21.15.131:0/1752398119 conn(0x7f01680fdd30 msgr2=0x7f0168100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 -- 172.21.15.131:0/1752398119 shutdown_connections 2024-04-26T20:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.018+0000 7f016f200640 1 -- 172.21.15.131:0/1752398119 wait complete. 2024-04-26T20:18:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.019+0000 7f016f200640 1 Processor -- start 2024-04-26T20:18:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.019+0000 7f016f200640 1 -- start start 2024-04-26T20:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168075390 0x7f016819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168076970 0x7f016819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016810ed90 0x7f01681a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0168114930 con 0x7f0168076970 2024-04-26T20:18:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01681147b0 con 0x7f0168075390 2024-04-26T20:18:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0168114ab0 con 0x7f016810ed90 2024-04-26T20:18:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168075390 0x7f016819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168076970 0x7f016819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168075390 0x7f016819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36562/0 (socket says 172.21.15.131:36562) 2024-04-26T20:18:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016de00640 1 -- 172.21.15.131:0/1069653007 learned_addr learned my addr 172.21.15.131:0/1069653007 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.020+0000 7f016e800640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016810ed90 0x7f01681a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016de00640 1 -- 172.21.15.131:0/1069653007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016810ed90 msgr2=0x7f01681a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016de00640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016810ed90 0x7f01681a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016de00640 1 -- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168076970 msgr2=0x7f016819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016de00640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168076970 0x7f016819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016de00640 1 -- 172.21.15.131:0/1069653007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01681a6c30 con 0x7f0168075390 2024-04-26T20:18:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016de00640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168075390 0x7f016819f060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f016400ebc0 tx=0x7f016400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f015ea00640 1 -- 172.21.15.131:0/1069653007 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164010630 con 0x7f0168075390 2024-04-26T20:18:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f015ea00640 1 -- 172.21.15.131:0/1069653007 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0164010c80 con 0x7f0168075390 2024-04-26T20:18:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01681a6f20 con 0x7f0168075390 2024-04-26T20:18:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f015ea00640 1 -- 172.21.15.131:0/1069653007 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164018b60 con 0x7f0168075390 2024-04-26T20:18:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.021+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a7380 con 0x7f0168075390 2024-04-26T20:18:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 Processor -- start 2024-04-26T20:18:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 -- start start 2024-04-26T20:18:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f015ea00640 1 -- 172.21.15.131:0/1069653007 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0164018d00 con 0x7f0168075390 2024-04-26T20:18:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e26200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e2b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564808d8a380 con 0x564808e25de0 2024-04-26T20:18:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564808d26c30 con 0x564808e267d0 2024-04-26T20:18:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f0173563740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564808d86340 con 0x564808e2b940 2024-04-26T20:18:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e26200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e26200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58322/0 (socket says 172.21.15.131:58322) 2024-04-26T20:18:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f016d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.025+0000 7f016e800640 1 -- 172.21.15.131:0/3715339337 learned_addr learned my addr 172.21.15.131:0/3715339337 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016de00640 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e2b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016e800640 1 -- 172.21.15.131:0/3715339337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 msgr2=0x564808e2ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016e800640 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e2ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016e800640 1 -- 172.21.15.131:0/3715339337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 msgr2=0x564808e2b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016e800640 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e2b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016e800640 1 -- 172.21.15.131:0/3715339337 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564808d48940 con 0x564808e25de0 2024-04-26T20:18:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016d400640 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016de00640 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e2b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f016e800640 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e26200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0160009ca0 tx=0x7f0160011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507e88540457bd72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.026+0000 7f015ea00640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f01540a3c30 0x7f01540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f0153000640 1 -- 172.21.15.131:0/3715339337 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016001e070 con 0x564808e25de0 2024-04-26T20:18:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f0153000640 1 -- 172.21.15.131:0/3715339337 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0160009e50 con 0x564808e25de0 2024-04-26T20:18:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f0153000640 1 -- 172.21.15.131:0/3715339337 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016000ea50 con 0x564808e25de0 2024-04-26T20:18:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f015ea00640 1 -- 172.21.15.131:0/1069653007 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f01640d0f50 con 0x7f0168075390 2024-04-26T20:18:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f016de00640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f01540a3c30 0x7f01540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f0173563740 1 -- 172.21.15.131:0/3715339337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 msgr2=0x564808e26200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f0173563740 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e26200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0160009ca0 tx=0x7f0160011810 comp rx=0 tx=0).stop 2024-04-26T20:18:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.027+0000 7f016de00640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f01540a3c30 0x7f01540a60f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x564808e27240 tx=0x7f0164000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 -- 172.21.15.131:0/3715339337 shutdown_connections 2024-04-26T20:18:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e2b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 --2- 172.21.15.131:0/3715339337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e26200 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 -- 172.21.15.131:0/3715339337 >> 172.21.15.131:0/3715339337 conn(0x564808e21450 msgr2=0x564808e21850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 -- 172.21.15.131:0/3715339337 shutdown_connections 2024-04-26T20:18:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 -- 172.21.15.131:0/3715339337 wait complete. 2024-04-26T20:18:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 -- start start 2024-04-26T20:18:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.028+0000 7f0173563740 1 Processor -- start 2024-04-26T20:18:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f0173563740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f0173563740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f0173563740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e2b940 0x564808e37570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f0173563740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564808e2e8f0 con 0x564808e2b940 2024-04-26T20:18:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f0173563740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564808d11c60 con 0x564808e267d0 2024-04-26T20:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f0173563740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564808e2ea70 con 0x564808e25de0 2024-04-26T20:18:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f0173563740 10 client.0 _ll_register_callbacks cb 0x564808d3de50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:18:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e2b940 0x564808e37570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f016d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f016d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36586/0 (socket says 172.21.15.131:36586) 2024-04-26T20:18:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f016d400640 1 -- 172.21.15.131:0/2920819330 learned_addr learned my addr 172.21.15.131:0/2920819330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e2b940 0x564808e37570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58332/0 (socket says 172.21.15.131:58332) 2024-04-26T20:18:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.029+0000 7f016e800640 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f016d400640 1 -- 172.21.15.131:0/2920819330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e25de0 msgr2=0x564808e2fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f016d400640 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f016d400640 1 -- 172.21.15.131:0/2920819330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e2b940 msgr2=0x564808e37570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f016d400640 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e2b940 0x564808e37570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f016d400640 1 -- 172.21.15.131:0/2920819330 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564808d20510 con 0x564808e267d0 2024-04-26T20:18:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f016e800640 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e2b940 0x564808e37570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f016d400640 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f015802aa10 tx=0x7f01580043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c04c43b29cda7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f0147400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:18:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f0147400640 1 -- 172.21.15.131:0/2920819330 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158035040 con 0x564808e267d0 2024-04-26T20:18:39.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f0147400640 1 -- 172.21.15.131:0/2920819330 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0158003a00 con 0x564808e267d0 2024-04-26T20:18:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.030+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0160009e50 con 0x564808e267d0 2024-04-26T20:18:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0147400640 1 -- 172.21.15.131:0/2920819330 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f015804e050 con 0x564808e267d0 2024-04-26T20:18:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 10 client.25784 fetch_fsmap learned FSMap version 27 2024-04-26T20:18:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564808a9c210 con 0x564808e267d0 2024-04-26T20:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0147400640 1 -- 172.21.15.131:0/2920819330 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0160009e50 con 0x564808e267d0 2024-04-26T20:18:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0147400640 1 -- 172.21.15.131:0/2920819330 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f015803c440 con 0x564808e267d0 2024-04-26T20:18:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 10 client.25784 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:18:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:18:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 20 client.25784 populate_metadata read hostname 'smithi131' 2024-04-26T20:18:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808e3bb20 0x564808e3df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 4 client.25784 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:18:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.031+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564808d97f90 con 0x564808e3bb20 2024-04-26T20:18:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.032+0000 7f016e800640 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808e3bb20 0x564808e3df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.032+0000 7f016e800640 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808e3bb20 0x564808e3df10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90756a06d7349abf server_cookie=2ae42e2c29968c03 in_seq=0 out_seq=0 2024-04-26T20:18:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.032+0000 7f0147400640 10 client.25784 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.033+0000 7f0147400640 1 -- 172.21.15.131:0/2920819330 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564808d97f90 con 0x564808e3bb20 2024-04-26T20:18:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.033+0000 7f0147400640 10 client.25784 handle_command_reply: tid=0 2024-04-26T20:18:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.033+0000 7f0173563740 1 client.25784 shutdown 2024-04-26T20:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 msgr2=0x564808e30530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f015802aa10 tx=0x7f01580043b0 comp rx=0 tx=0).stop 2024-04-26T20:18:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 shutdown_connections 2024-04-26T20:18:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808e3bb20 0x564808e3df10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e2b940 0x564808e37570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 --2- 172.21.15.131:0/2920819330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0147400640 0 client.25784 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 >> 172.21.15.131:0/2920819330 conn(0x564808e21450 msgr2=0x564808e20650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 shutdown_connections 2024-04-26T20:18:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 1 -- 172.21.15.131:0/2920819330 wait complete. 2024-04-26T20:18:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.034+0000 7f0173563740 20 client.25784 trim_cache size 0 max 16384 2024-04-26T20:18:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 Processor -- start 2024-04-26T20:18:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 -- start start 2024-04-26T20:18:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e37530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b4340 con 0x564808e25de0 2024-04-26T20:18:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b1c10 con 0x564808e267d0 2024-04-26T20:18:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f0173563740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01681afa60 con 0x564808e2b940 2024-04-26T20:18:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58334/0 (socket says 172.21.15.131:58334) 2024-04-26T20:18:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.050+0000 7f016e800640 1 -- 172.21.15.131:0/3546172206 learned_addr learned my addr 172.21.15.131:0/3546172206 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016d400640 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016e800640 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e37530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016e800640 1 -- 172.21.15.131:0/3546172206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 msgr2=0x564808e37530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016e800640 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e37530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016e800640 1 -- 172.21.15.131:0/3546172206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 msgr2=0x564808e30530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016e800640 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016e800640 1 -- 172.21.15.131:0/3546172206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564808e3e280 con 0x564808e25de0 2024-04-26T20:18:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016d400640 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f016e800640 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e2fff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0160009800 tx=0x7f016000e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9af757bf5d93580 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.051+0000 7f0152600640 1 -- 172.21.15.131:0/3546172206 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016001e040 con 0x564808e25de0 2024-04-26T20:18:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0152600640 1 -- 172.21.15.131:0/3546172206 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01600054f0 con 0x564808e25de0 2024-04-26T20:18:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0152600640 1 -- 172.21.15.131:0/3546172206 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160019430 con 0x564808e25de0 2024-04-26T20:18:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 -- 172.21.15.131:0/3546172206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 msgr2=0x564808e2fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e2fff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0160009800 tx=0x7f016000e430 comp rx=0 tx=0).stop 2024-04-26T20:18:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 -- 172.21.15.131:0/3546172206 shutdown_connections 2024-04-26T20:18:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e2b940 0x564808e37530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e267d0 0x564808e30530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 --2- 172.21.15.131:0/3546172206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e2fff0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 -- 172.21.15.131:0/3546172206 >> 172.21.15.131:0/3546172206 conn(0x564808e21450 msgr2=0x564808e27a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 -- 172.21.15.131:0/3546172206 shutdown_connections 2024-04-26T20:18:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.052+0000 7f0173563740 1 -- 172.21.15.131:0/3546172206 wait complete. 2024-04-26T20:18:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 -- start start 2024-04-26T20:18:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 Processor -- start 2024-04-26T20:18:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e34710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e267d0 0x564808e34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e2b940 0x564808e35bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b00d0 con 0x564808e25de0 2024-04-26T20:18:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b1540 con 0x564808e2b940 2024-04-26T20:18:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b07a0 con 0x564808e267d0 2024-04-26T20:18:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.053+0000 7f0173563740 10 client.0 _ll_register_callbacks cb 0x7f0158009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:18:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e34710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e34710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58340/0 (socket says 172.21.15.131:58340) 2024-04-26T20:18:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016e800640 1 -- 172.21.15.131:0/2687104322 learned_addr learned my addr 172.21.15.131:0/2687104322 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016d400640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e267d0 0x564808e34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016e800640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e2b940 0x564808e35bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016d400640 1 -- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e2b940 msgr2=0x564808e35bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016d400640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e2b940 0x564808e35bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016d400640 1 -- 172.21.15.131:0/2687104322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 msgr2=0x564808e34710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016d400640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e34710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016d400640 1 -- 172.21.15.131:0/2687104322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564808b68620 con 0x564808e267d0 2024-04-26T20:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016e800640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e2b940 0x564808e35bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f016d400640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e267d0 0x564808e34c50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0158005ab0 tx=0x7f015803c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215fb3851e8798fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.054+0000 7f0147400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:18:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0147400640 1 -- 172.21.15.131:0/2687104322 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158057060 con 0x564808e267d0 2024-04-26T20:18:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0147400640 1 -- 172.21.15.131:0/2687104322 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0158003a00 con 0x564808e267d0 2024-04-26T20:18:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01600054f0 con 0x564808e267d0 2024-04-26T20:18:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0147400640 1 -- 172.21.15.131:0/2687104322 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f015805e050 con 0x564808e267d0 2024-04-26T20:18:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0173563740 10 client.25783 fetch_fsmap learned FSMap version 27 2024-04-26T20:18:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564808d44cb0 con 0x564808e267d0 2024-04-26T20:18:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0147400640 1 -- 172.21.15.131:0/2687104322 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01600054f0 con 0x564808e267d0 2024-04-26T20:18:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.055+0000 7f0147400640 1 -- 172.21.15.131:0/2687104322 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f01580318a0 con 0x564808e267d0 2024-04-26T20:18:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f0173563740 10 client.25783 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:18:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f0173563740 10 client.25783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:18:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f0173563740 20 client.25783 populate_metadata read hostname 'smithi131' 2024-04-26T20:18:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f0173563740 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808ea2100 0x564808ea4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f0173563740 4 client.25783 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:18:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564808d91e00 con 0x564808ea2100 2024-04-26T20:18:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f016e800640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808ea2100 0x564808ea4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f016e800640 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808ea2100 0x564808ea4520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64cb1f4b93bc5df5 server_cookie=2d050eeb1a194268 in_seq=0 out_seq=0 2024-04-26T20:18:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.056+0000 7f0147400640 10 client.25783 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0147400640 1 -- 172.21.15.131:0/2687104322 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564808d91e00 con 0x564808ea2100 2024-04-26T20:18:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0147400640 10 client.25783 handle_command_reply: tid=0 2024-04-26T20:18:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 client.25783 shutdown 2024-04-26T20:18:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e267d0 msgr2=0x564808e34c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e267d0 0x564808e34c50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0158005ab0 tx=0x7f015803c6f0 comp rx=0 tx=0).stop 2024-04-26T20:18:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 shutdown_connections 2024-04-26T20:18:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564808ea2100 0x564808ea4520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564808e2b940 0x564808e35bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564808e267d0 0x564808e34c50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0147400640 0 client.25783 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:18:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 --2- 172.21.15.131:0/2687104322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564808e25de0 0x564808e34710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 >> 172.21.15.131:0/2687104322 conn(0x564808e21450 msgr2=0x564808e2fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 shutdown_connections 2024-04-26T20:18:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.060+0000 7f0173563740 1 -- 172.21.15.131:0/2687104322 wait complete. 2024-04-26T20:18:49.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f0173563740 20 client.25783 trim_cache size 0 max 16384 2024-04-26T20:18:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:18:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:18:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f01540a3c30 msgr2=0x7f01540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f01540a3c30 0x7f01540a60f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x564808e27240 tx=0x7f0164000f40 comp rx=0 tx=0).stop 2024-04-26T20:18:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168075390 msgr2=0x7f016819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168075390 0x7f016819f060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f016400ebc0 tx=0x7f016400c6a0 comp rx=0 tx=0).stop 2024-04-26T20:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 shutdown_connections 2024-04-26T20:18:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f01540a3c30 0x7f01540a60f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016810ed90 0x7f01681a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0168076970 0x7f016819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 --2- 172.21.15.131:0/1069653007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0168075390 0x7f016819f060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 >> 172.21.15.131:0/1069653007 conn(0x7f01680fdd30 msgr2=0x7f0168102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 shutdown_connections 2024-04-26T20:18:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:16:58.061+0000 7f016f200640 1 -- 172.21.15.131:0/1069653007 wait complete. 2024-04-26T20:18:50.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:16:59 smithi131 ceph-mon[27919]: pgmap v619: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:50.812 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:01 smithi131 ceph-mon[27919]: pgmap v620: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:50.915 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:03 smithi131 ceph-mon[27919]: pgmap v621: 129 pgs: 129 active+clean; 667 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:50.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:05 smithi131 ceph-mon[27919]: pgmap v622: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:51.031 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:07 smithi131 ceph-mon[27919]: pgmap v623: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:51.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:09 smithi131 ceph-mon[27919]: pgmap v624: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:51.080 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:33 smithi148 ceph-mon[32281]: pgmap v666: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:51.160 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:35 smithi148 ceph-mon[32281]: pgmap v667: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:51.310 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:37 smithi148 ceph-mon[32281]: pgmap v668: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:51.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:39 smithi148 ceph-mon[32281]: pgmap v669: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:51.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:41 smithi148 ceph-mon[32281]: pgmap v670: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:51.448 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:43 smithi148 ceph-mon[32281]: pgmap v671: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:51.494 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:43 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:18:51.565 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:18:51.584 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:18:51.592 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:44 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:18:51.597 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:45 smithi148 ceph-mon[32281]: pgmap v672: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:51.603 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:47 smithi148 ceph-mon[32281]: pgmap v673: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:51.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:49 smithi148 ceph-mon[32281]: pgmap v674: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:18:51.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:33 smithi157 ceph-mon[32703]: pgmap v666: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:51.831 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:35 smithi157 ceph-mon[32703]: pgmap v667: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:51.914 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:37 smithi157 ceph-mon[32703]: pgmap v668: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:51.922 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:39 smithi157 ceph-mon[32703]: pgmap v669: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:51.983 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:41 smithi157 ceph-mon[32703]: pgmap v670: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:52.159 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:43 smithi157 ceph-mon[32703]: pgmap v671: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:18:52.280 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:43 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:18:52.341 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:18:52.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:18:52.514 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:44 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:18:52.631 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:45 smithi157 ceph-mon[32703]: pgmap v672: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:18:52.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:47 smithi157 ceph-mon[32703]: pgmap v673: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:18:52.765 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:49 smithi157 ceph-mon[32703]: pgmap v674: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:18:52.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:51 smithi148 ceph-mon[32281]: pgmap v675: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:52.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:51 smithi157 ceph-mon[32703]: pgmap v675: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:18:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.203+0000 7f3bece00640 1 -- 172.21.15.131:0/3329827321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be810a990 msgr2=0x7f3be8111220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.203+0000 7f3bece00640 1 --2- 172.21.15.131:0/3329827321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be810a990 0x7f3be8111220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009ca0 tx=0x7f3be002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:18:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.204+0000 7f3bece00640 1 -- 172.21.15.131:0/3329827321 shutdown_connections 2024-04-26T20:18:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.204+0000 7f3bece00640 1 --2- 172.21.15.131:0/3329827321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be810a990 0x7f3be8111220 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.204+0000 7f3bece00640 1 --2- 172.21.15.131:0/3329827321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be8109fd0 0x7f3be810a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.204+0000 7f3bece00640 1 --2- 172.21.15.131:0/3329827321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be8108de0 0x7f3be8109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.204+0000 7f3bece00640 1 -- 172.21.15.131:0/3329827321 >> 172.21.15.131:0/3329827321 conn(0x7f3be8077dc0 msgr2=0x7f3be807a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.204+0000 7f3bece00640 1 -- 172.21.15.131:0/3329827321 shutdown_connections 2024-04-26T20:18:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.204+0000 7f3bece00640 1 -- 172.21.15.131:0/3329827321 wait complete. 2024-04-26T20:18:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 Processor -- start 2024-04-26T20:18:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 -- start start 2024-04-26T20:18:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be8108de0 0x7f3be819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be8109fd0 0x7f3be819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be810a990 0x7f3be81a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8114940 con 0x7f3be8108de0 2024-04-26T20:18:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81147c0 con 0x7f3be8109fd0 2024-04-26T20:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3bece00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8114ac0 con 0x7f3be810a990 2024-04-26T20:18:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3be7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be8108de0 0x7f3be819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3be7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be8108de0 0x7f3be819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53938/0 (socket says 172.21.15.131:53938) 2024-04-26T20:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.205+0000 7f3be7400640 1 -- 172.21.15.131:0/3111257291 learned_addr learned my addr 172.21.15.131:0/3111257291 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be8109fd0 0x7f3be819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3be7e00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be810a990 0x7f3be81a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3be7400640 1 -- 172.21.15.131:0/3111257291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be810a990 msgr2=0x7f3be81a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3be7400640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be810a990 0x7f3be81a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3be7400640 1 -- 172.21.15.131:0/3111257291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be8109fd0 msgr2=0x7f3be819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3be7400640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be8109fd0 0x7f3be819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3be7400640 1 -- 172.21.15.131:0/3111257291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be81a6ba0 con 0x7f3be8108de0 2024-04-26T20:18:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.206+0000 7f3be7400640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be8108de0 0x7f3be819ef80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800e9c0 tx=0x7f3bd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.207+0000 7f3be4c00640 1 -- 172.21.15.131:0/3111257291 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800cd80 con 0x7f3be8108de0 2024-04-26T20:18:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.207+0000 7f3be4c00640 1 -- 172.21.15.131:0/3111257291 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd80035f0 con 0x7f3be8108de0 2024-04-26T20:18:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.207+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be81a6e90 con 0x7f3be8108de0 2024-04-26T20:18:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.207+0000 7f3be4c00640 1 -- 172.21.15.131:0/3111257291 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8010690 con 0x7f3be8108de0 2024-04-26T20:18:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.207+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be81a72f0 con 0x7f3be8108de0 2024-04-26T20:18:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 Processor -- start 2024-04-26T20:18:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 -- start start 2024-04-26T20:18:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e736de0 0x55db4e737200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3be4c00640 1 -- 172.21.15.131:0/3111257291 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3bd80037a0 con 0x7f3be8108de0 2024-04-26T20:18:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e73c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e73c940 0x55db4e73edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db4e69b380 con 0x55db4e7377d0 2024-04-26T20:18:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55db4e637c30 con 0x55db4e736de0 2024-04-26T20:18:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.211+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db4e697340 con 0x55db4e73c940 2024-04-26T20:18:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e736de0 0x55db4e737200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e73c940 0x55db4e73edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e73c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e736de0 0x55db4e737200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:47234/0 (socket says 172.21.15.131:47234) 2024-04-26T20:18:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e73c940 0x55db4e73edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48486/0 (socket says 172.21.15.131:48486) 2024-04-26T20:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 -- 172.21.15.131:0/267969453 learned_addr learned my addr 172.21.15.131:0/267969453 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 -- 172.21.15.131:0/267969453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e73c940 msgr2=0x55db4e73edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e73c940 0x55db4e73edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 -- 172.21.15.131:0/267969453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 msgr2=0x55db4e73c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e73c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3be7e00640 1 -- 172.21.15.131:0/267969453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db4e659940 con 0x55db4e736de0 2024-04-26T20:18:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.212+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e73c940 0x55db4e73edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.213+0000 7f3be7e00640 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e736de0 0x55db4e737200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009ca0 tx=0x7f3be00093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb72cf79cf95ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.213+0000 7f3bdcc00640 1 -- 172.21.15.131:0/267969453 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0031650 con 0x55db4e736de0 2024-04-26T20:18:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.213+0000 7f3bdcc00640 1 -- 172.21.15.131:0/267969453 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be0036030 con 0x55db4e736de0 2024-04-26T20:18:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.213+0000 7f3bdcc00640 1 -- 172.21.15.131:0/267969453 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0004660 con 0x55db4e736de0 2024-04-26T20:18:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.213+0000 7f3be4c00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3bcc0a3c30 0x7f3bcc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.213+0000 7f3be4c00640 1 -- 172.21.15.131:0/3111257291 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3bd80d0d40 con 0x7f3be8108de0 2024-04-26T20:18:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.213+0000 7f3be7400640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3bcc0a3c30 0x7f3bcc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 -- 172.21.15.131:0/267969453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e736de0 msgr2=0x55db4e737200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e736de0 0x55db4e737200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009ca0 tx=0x7f3be00093c0 comp rx=0 tx=0).stop 2024-04-26T20:18:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 -- 172.21.15.131:0/267969453 shutdown_connections 2024-04-26T20:18:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e73c940 0x55db4e73edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e73c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/267969453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e736de0 0x55db4e737200 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 -- 172.21.15.131:0/267969453 >> 172.21.15.131:0/267969453 conn(0x55db4e732450 msgr2=0x55db4e732850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 -- 172.21.15.131:0/267969453 shutdown_connections 2024-04-26T20:18:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 -- 172.21.15.131:0/267969453 wait complete. 2024-04-26T20:18:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3be7400640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3bcc0a3c30 0x7f3bcc0a60f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55db4e737bd0 tx=0x7f3bd80d47f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.214+0000 7f3bf13aa740 1 -- start start 2024-04-26T20:18:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 1 Processor -- start 2024-04-26T20:18:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e736de0 0x55db4e740e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e7377d0 0x55db4e741360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e7483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db4e73f710 con 0x55db4e736de0 2024-04-26T20:18:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55db4e622c60 con 0x55db4e73c940 2024-04-26T20:18:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db4e73f890 con 0x55db4e7377d0 2024-04-26T20:18:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bf13aa740 10 client.0 _ll_register_callbacks cb 0x55db4e64ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:18:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e736de0 0x55db4e740e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e736de0 0x55db4e740e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53956/0 (socket says 172.21.15.131:53956) 2024-04-26T20:18:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3be7e00640 1 -- 172.21.15.131:0/1252529634 learned_addr learned my addr 172.21.15.131:0/1252529634 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:18:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.215+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e7377d0 0x55db4e741360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3be7e00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e7483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3be7e00640 1 -- 172.21.15.131:0/1252529634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e7377d0 msgr2=0x55db4e741360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3be7e00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e7377d0 0x55db4e741360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3be7e00640 1 -- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 msgr2=0x55db4e7483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3be7e00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e7483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3be7e00640 1 -- 172.21.15.131:0/1252529634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4e631510 con 0x55db4e736de0 2024-04-26T20:18:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e7377d0 0x55db4e741360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3be7e00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e736de0 0x55db4e740e20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3be0005c40 tx=0x7f3be0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd61f54f1f05d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3bd5600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:18:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3bd5600640 1 -- 172.21.15.131:0/1252529634 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0058060 con 0x55db4e736de0 2024-04-26T20:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3bd5600640 1 -- 172.21.15.131:0/1252529634 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3be0003b70 con 0x55db4e736de0 2024-04-26T20:18:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.216+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be0036030 con 0x55db4e736de0 2024-04-26T20:18:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bd5600640 1 -- 172.21.15.131:0/1252529634 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3be0003d70 con 0x55db4e736de0 2024-04-26T20:18:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 10 client.15684 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db4e3ad210 con 0x55db4e736de0 2024-04-26T20:19:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bd5600640 1 -- 172.21.15.131:0/1252529634 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be0036030 con 0x55db4e736de0 2024-04-26T20:19:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bd5600640 1 -- 172.21.15.131:0/1252529634 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3be0043070 con 0x55db4e736de0 2024-04-26T20:19:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 20 client.15684 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e74c5c0 0x55db4e74e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 4 client.15684 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:19:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.217+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db4e6a8f90 con 0x55db4e74c5c0 2024-04-26T20:19:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.218+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e74c5c0 0x55db4e74e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.218+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e74c5c0 0x55db4e74e9b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391960997dc536bd server_cookie=1c089b4301f088c6 in_seq=0 out_seq=0 2024-04-26T20:19:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.218+0000 7f3bd5600640 10 client.15684 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bd5600640 1 -- 172.21.15.131:0/1252529634 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db4e6a8f90 con 0x55db4e74c5c0 2024-04-26T20:19:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bd5600640 10 client.15684 handle_command_reply: tid=0 2024-04-26T20:19:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 client.15684 shutdown 2024-04-26T20:19:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e736de0 msgr2=0x55db4e740e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e736de0 0x55db4e740e20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3be0005c40 tx=0x7f3be0005bd0 comp rx=0 tx=0).stop 2024-04-26T20:19:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 shutdown_connections 2024-04-26T20:19:01.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e74c5c0 0x55db4e74e9b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e7483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e7377d0 0x55db4e741360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bd5600640 0 client.15684 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1252529634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e736de0 0x55db4e740e20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.219+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 >> 172.21.15.131:0/1252529634 conn(0x55db4e732450 msgr2=0x55db4e731650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.220+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 shutdown_connections 2024-04-26T20:19:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.220+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1252529634 wait complete. 2024-04-26T20:19:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.220+0000 7f3bf13aa740 20 client.15684 trim_cache size 0 max 16384 2024-04-26T20:19:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 Processor -- start 2024-04-26T20:19:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 -- start start 2024-04-26T20:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e731dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e748360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e735c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b42d0 con 0x55db4e7377d0 2024-04-26T20:19:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b1ba0 con 0x55db4e73c940 2024-04-26T20:19:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.236+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81af9f0 con 0x55db4e736de0 2024-04-26T20:19:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e731dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e748360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e731dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48508/0 (socket says 172.21.15.131:48508) 2024-04-26T20:19:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3be7e00640 1 -- 172.21.15.131:0/1840284278 learned_addr learned my addr 172.21.15.131:0/1840284278 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3be7e00640 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e735c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bdfe00640 1 -- 172.21.15.131:0/1840284278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 msgr2=0x55db4e731dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e731dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bdfe00640 1 -- 172.21.15.131:0/1840284278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 msgr2=0x55db4e735c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e735c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bdfe00640 1 -- 172.21.15.131:0/1840284278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db4e479620 con 0x55db4e7377d0 2024-04-26T20:19:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3be7e00640 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e731dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e748360 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55db4e74f1f0 tx=0x7f3bd0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e36a59d528c2abb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.237+0000 7f3bd7e00640 1 -- 172.21.15.131:0/1840284278 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0004250 con 0x55db4e7377d0 2024-04-26T20:19:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bd7e00640 1 -- 172.21.15.131:0/1840284278 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd00043f0 con 0x55db4e7377d0 2024-04-26T20:19:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bd7e00640 1 -- 172.21.15.131:0/1840284278 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0003b00 con 0x55db4e7377d0 2024-04-26T20:19:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1840284278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 msgr2=0x55db4e748360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e748360 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55db4e74f1f0 tx=0x7f3bd0005f90 comp rx=0 tx=0).stop 2024-04-26T20:19:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1840284278 shutdown_connections 2024-04-26T20:19:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e73c940 0x55db4e735c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e7377d0 0x55db4e748360 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/1840284278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e731dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1840284278 >> 172.21.15.131:0/1840284278 conn(0x55db4e732450 msgr2=0x55db4e7410a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1840284278 shutdown_connections 2024-04-26T20:19:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.238+0000 7f3bf13aa740 1 -- 172.21.15.131:0/1840284278 wait complete. 2024-04-26T20:19:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 -- start start 2024-04-26T20:19:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 Processor -- start 2024-04-26T20:19:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e747e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e7377d0 0x55db4e746450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e73c940 0x55db4e746990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b0060 con 0x55db4e73c940 2024-04-26T20:19:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b14d0 con 0x55db4e7377d0 2024-04-26T20:19:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b0730 con 0x55db4e736de0 2024-04-26T20:19:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.239+0000 7f3bf13aa740 10 client.0 _ll_register_callbacks cb 0x7f3bd80d46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:19:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e7377d0 0x55db4e746450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e73c940 0x55db4e746990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e7377d0 0x55db4e746450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:47274/0 (socket says 172.21.15.131:47274) 2024-04-26T20:19:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 -- 172.21.15.131:0/2673202057 learned_addr learned my addr 172.21.15.131:0/2673202057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3be7e00640 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e747e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 -- 172.21.15.131:0/2673202057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 msgr2=0x55db4e747e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e747e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 -- 172.21.15.131:0/2673202057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e73c940 msgr2=0x55db4e746990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e73c940 0x55db4e746990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 -- 172.21.15.131:0/2673202057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4e655cb0 con 0x55db4e7377d0 2024-04-26T20:19:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3be7e00640 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e73c940 0x55db4e746990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bdfe00640 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e7377d0 0x55db4e746450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0014010 tx=0x7f3bd0004220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58fca94703e4a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.240+0000 7f3bd5600640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:19:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.241+0000 7f3bd5600640 1 -- 172.21.15.131:0/2673202057 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00050c0 con 0x55db4e7377d0 2024-04-26T20:19:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.241+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd00043f0 con 0x55db4e7377d0 2024-04-26T20:19:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.241+0000 7f3bd5600640 1 -- 172.21.15.131:0/2673202057 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3bd0005260 con 0x55db4e7377d0 2024-04-26T20:19:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.241+0000 7f3bd5600640 1 -- 172.21.15.131:0/2673202057 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3bd0038050 con 0x55db4e7377d0 2024-04-26T20:19:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.241+0000 7f3bf13aa740 10 client.25808 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.241+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db4e7b3600 con 0x55db4e7377d0 2024-04-26T20:19:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bd5600640 1 -- 172.21.15.131:0/2673202057 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd00043f0 con 0x55db4e7377d0 2024-04-26T20:19:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bd5600640 1 -- 172.21.15.131:0/2673202057 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3bd00201f0 con 0x55db4e7377d0 2024-04-26T20:19:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bf13aa740 10 client.25808 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bf13aa740 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bf13aa740 20 client.25808 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e7b3fb0 0x55db4e7b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bf13aa740 4 client.25808 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:19:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db4e6a2e00 con 0x55db4e7b3fb0 2024-04-26T20:19:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.242+0000 7f3be7e00640 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e7b3fb0 0x55db4e7b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.243+0000 7f3be7e00640 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e7b3fb0 0x55db4e7b63d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c292c1c9d79978d1 server_cookie=ae0c24ce065acd44 in_seq=0 out_seq=0 2024-04-26T20:19:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.243+0000 7f3bd5600640 10 client.25808 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bd5600640 1 -- 172.21.15.131:0/2673202057 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db4e6a2e00 con 0x55db4e7b3fb0 2024-04-26T20:19:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bd5600640 10 client.25808 handle_command_reply: tid=0 2024-04-26T20:19:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 client.25808 shutdown 2024-04-26T20:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e7377d0 msgr2=0x55db4e746450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e7377d0 0x55db4e746450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0014010 tx=0x7f3bd0004220 comp rx=0 tx=0).stop 2024-04-26T20:19:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 shutdown_connections 2024-04-26T20:19:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55db4e7b3fb0 0x55db4e7b63d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e73c940 0x55db4e746990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db4e7377d0 0x55db4e746450 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 --2- 172.21.15.131:0/2673202057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e736de0 0x55db4e747e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bd5600640 0 client.25808 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 >> 172.21.15.131:0/2673202057 conn(0x55db4e732450 msgr2=0x55db4e730dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 shutdown_connections 2024-04-26T20:19:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 1 -- 172.21.15.131:0/2673202057 wait complete. 2024-04-26T20:19:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.247+0000 7f3bf13aa740 20 client.25808 trim_cache size 0 max 16384 2024-04-26T20:19:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:19:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:19:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:19:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:19:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3bcc0a3c30 msgr2=0x7f3bcc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3bcc0a3c30 0x7f3bcc0a60f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55db4e737bd0 tx=0x7f3bd80d47f0 comp rx=0 tx=0).stop 2024-04-26T20:19:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be8108de0 msgr2=0x7f3be819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be8108de0 0x7f3be819ef80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800e9c0 tx=0x7f3bd800ee80 comp rx=0 tx=0).stop 2024-04-26T20:19:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 shutdown_connections 2024-04-26T20:19:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3bcc0a3c30 0x7f3bcc0a60f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be810a990 0x7f3be81a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be8109fd0 0x7f3be819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 --2- 172.21.15.131:0/3111257291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be8108de0 0x7f3be819ef80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 >> 172.21.15.131:0/3111257291 conn(0x7f3be8077dc0 msgr2=0x7f3be810d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 shutdown_connections 2024-04-26T20:19:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:11.248+0000 7f3bece00640 1 -- 172.21.15.131:0/3111257291 wait complete. 2024-04-26T20:19:08.026 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:11 smithi131 ceph-mon[27919]: pgmap v625: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:08.105 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:13 smithi131 ceph-mon[27919]: pgmap v626: 129 pgs: 129 active+clean; 668 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:08.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:15 smithi131 ceph-mon[27919]: pgmap v627: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:19:08.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:17 smithi131 ceph-mon[27919]: pgmap v628: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:08.481 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:19 smithi131 ceph-mon[27919]: pgmap v629: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:08.541 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:21 smithi131 ceph-mon[27919]: pgmap v630: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:08.596 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:23 smithi131 ceph-mon[27919]: pgmap v631: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:08.606 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:53 smithi148 ceph-mon[32281]: pgmap v676: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:08.696 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:55 smithi148 ceph-mon[32281]: pgmap v677: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:08.814 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:57 smithi148 ceph-mon[32281]: pgmap v678: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:08.821 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:18:59 smithi148 ceph-mon[32281]: pgmap v679: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:08.935 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:01 smithi148 ceph-mon[32281]: pgmap v680: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:19:08.983 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:03 smithi148 ceph-mon[32281]: pgmap v681: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:09.077 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:05 smithi148 ceph-mon[32281]: pgmap v682: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:09.195 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:07 smithi148 ceph-mon[32281]: pgmap v683: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:09.270 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:53 smithi157 ceph-mon[32703]: pgmap v676: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:09.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:55 smithi157 ceph-mon[32703]: pgmap v677: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:09.486 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:57 smithi157 ceph-mon[32703]: pgmap v678: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:09.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:18:59 smithi157 ceph-mon[32703]: pgmap v679: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:09.583 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:01 smithi157 ceph-mon[32703]: pgmap v680: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:19:09.590 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:03 smithi157 ceph-mon[32703]: pgmap v681: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:09.667 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:05 smithi157 ceph-mon[32703]: pgmap v682: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:09.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:07 smithi157 ceph-mon[32703]: pgmap v683: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 -- 172.21.15.131:0/2498845153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc105bc0 msgr2=0x7efdcc106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 --2- 172.21.15.131:0/2498845153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc105bc0 0x7efdcc106040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009900 tx=0x7efdb402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:19:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 -- 172.21.15.131:0/2498845153 shutdown_connections 2024-04-26T20:19:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 --2- 172.21.15.131:0/2498845153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdcc106580 0x7efdcc10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 --2- 172.21.15.131:0/2498845153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc105bc0 0x7efdcc106040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 --2- 172.21.15.131:0/2498845153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdcc1049d0 0x7efdcc104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 -- 172.21.15.131:0/2498845153 >> 172.21.15.131:0/2498845153 conn(0x7efdcc0fff40 msgr2=0x7efdcc102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 -- 172.21.15.131:0/2498845153 shutdown_connections 2024-04-26T20:19:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.392+0000 7efdd1600640 1 -- 172.21.15.131:0/2498845153 wait complete. 2024-04-26T20:19:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 Processor -- start 2024-04-26T20:19:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 -- start start 2024-04-26T20:19:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc1049d0 0x7efdcc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdcc105bc0 0x7efdcc19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdcc106580 0x7efdcc1a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc110500 con 0x7efdcc105bc0 2024-04-26T20:19:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc110380 con 0x7efdcc106580 2024-04-26T20:19:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.394+0000 7efdd1600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc110680 con 0x7efdcc1049d0 2024-04-26T20:19:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdcbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc1049d0 0x7efdcc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdcc105bc0 0x7efdcc19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdcbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc1049d0 0x7efdcc19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54446/0 (socket says 172.21.15.131:54446) 2024-04-26T20:19:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdcbe00640 1 -- 172.21.15.131:0/1336725732 learned_addr learned my addr 172.21.15.131:0/1336725732 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdd0c00640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdcc106580 0x7efdcc1a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdc3e00640 1 -- 172.21.15.131:0/1336725732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc1049d0 msgr2=0x7efdcc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdc3e00640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc1049d0 0x7efdcc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdc3e00640 1 -- 172.21.15.131:0/1336725732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdcc106580 msgr2=0x7efdcc1a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdc3e00640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdcc106580 0x7efdcc1a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdc3e00640 1 -- 172.21.15.131:0/1336725732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc1a27d0 con 0x7efdcc105bc0 2024-04-26T20:19:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.395+0000 7efdc3e00640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdcc105bc0 0x7efdcc19b0f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006010 tx=0x7efdb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.396+0000 7efdc9600640 1 -- 172.21.15.131:0/1336725732 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb403f030 con 0x7efdcc105bc0 2024-04-26T20:19:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.396+0000 7efdc9600640 1 -- 172.21.15.131:0/1336725732 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb4031910 con 0x7efdcc105bc0 2024-04-26T20:19:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.396+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdcc1a2a60 con 0x7efdcc105bc0 2024-04-26T20:19:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.396+0000 7efdc9600640 1 -- 172.21.15.131:0/1336725732 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4035070 con 0x7efdcc105bc0 2024-04-26T20:19:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.396+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdcc1a2f50 con 0x7efdcc105bc0 2024-04-26T20:19:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.400+0000 7efdd5bc2740 1 Processor -- start 2024-04-26T20:19:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd5bc2740 1 -- start start 2024-04-26T20:19:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdc9600640 1 -- 172.21.15.131:0/1336725732 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efdb4042030 con 0x7efdcc105bc0 2024-04-26T20:19:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fefd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910ff02940 0x55910ff04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55910fe61380 con 0x55910fefd7d0 2024-04-26T20:19:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55910fdfdc30 con 0x55910fefcde0 2024-04-26T20:19:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55910fe5d340 con 0x55910ff02940 2024-04-26T20:19:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdd0c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fefd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910ff02940 0x55910ff04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50768/0 (socket says 172.21.15.131:50768) 2024-04-26T20:19:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.401+0000 7efdcbe00640 1 -- 172.21.15.131:0/1663832335 learned_addr learned my addr 172.21.15.131:0/1663832335 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.402+0000 7efdd0c00640 1 -- 172.21.15.131:0/1663832335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910ff02940 msgr2=0x55910ff04dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.402+0000 7efdd0c00640 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910ff02940 0x55910ff04dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.402+0000 7efdd0c00640 1 -- 172.21.15.131:0/1663832335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 msgr2=0x55910ff022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.402+0000 7efdd0c00640 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.402+0000 7efdd0c00640 1 -- 172.21.15.131:0/1663832335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55910fe1f940 con 0x55910fefcde0 2024-04-26T20:19:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.402+0000 7efdcbe00640 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.402+0000 7efdd0c00640 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fefd200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009f00 tx=0x7efdc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7dc67b16ed0491 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdc1200640 1 -- 172.21.15.131:0/1663832335 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4004d20 con 0x55910fefcde0 2024-04-26T20:19:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdc1200640 1 -- 172.21.15.131:0/1663832335 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc4004050 con 0x55910fefcde0 2024-04-26T20:19:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdc1200640 1 -- 172.21.15.131:0/1663832335 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4005260 con 0x55910fefcde0 2024-04-26T20:19:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdc9600640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efdb00a3c30 0x7efdb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdc9600640 1 -- 172.21.15.131:0/1336725732 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7efdb40f0ef0 con 0x7efdcc105bc0 2024-04-26T20:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdcbe00640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efdb00a3c30 0x7efdb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1663832335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 msgr2=0x55910fefd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.403+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fefd200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009f00 tx=0x7efdc4011810 comp rx=0 tx=0).stop 2024-04-26T20:19:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1663832335 shutdown_connections 2024-04-26T20:19:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910ff02940 0x55910ff04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1663832335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fefd200 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1663832335 >> 172.21.15.131:0/1663832335 conn(0x55910fef8450 msgr2=0x55910fef8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1663832335 shutdown_connections 2024-04-26T20:19:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdcbe00640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efdb00a3c30 0x7efdb00a60f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55910fefdbd0 tx=0x7efdbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1663832335 wait complete. 2024-04-26T20:19:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 -- start start 2024-04-26T20:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.404+0000 7efdd5bc2740 1 Processor -- start 2024-04-26T20:19:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff06fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff07510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55910ff058f0 con 0x55910fefd7d0 2024-04-26T20:19:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55910fde8c60 con 0x55910ff02940 2024-04-26T20:19:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55910ff05a70 con 0x55910fefcde0 2024-04-26T20:19:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdcbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff06fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff07510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd5bc2740 10 client.0 _ll_register_callbacks cb 0x55910fe14e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:19:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff07510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50770/0 (socket says 172.21.15.131:50770) 2024-04-26T20:19:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdcbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff06fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54458/0 (socket says 172.21.15.131:54458) 2024-04-26T20:19:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdc3e00640 1 -- 172.21.15.131:0/2694084638 learned_addr learned my addr 172.21.15.131:0/2694084638 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.405+0000 7efdd0c00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdc3e00640 1 -- 172.21.15.131:0/2694084638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 msgr2=0x55910ff06fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdc3e00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff06fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdc3e00640 1 -- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 msgr2=0x55910ff0e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdc3e00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdc3e00640 1 -- 172.21.15.131:0/2694084638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910fdf7510 con 0x55910fefd7d0 2024-04-26T20:19:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdcbe00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff06fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdd0c00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdc3e00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff07510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efdb40b96a0 tx=0x7efdb4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4560776eeb01a0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdb9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:19:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdb9600640 1 -- 172.21.15.131:0/2694084638 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4049050 con 0x55910fefd7d0 2024-04-26T20:19:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdb9600640 1 -- 172.21.15.131:0/2694084638 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7efdb40dd0b0 con 0x55910fefd7d0 2024-04-26T20:19:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc4004050 con 0x55910fefd7d0 2024-04-26T20:19:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.406+0000 7efdb9600640 1 -- 172.21.15.131:0/2694084638 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdb40bcaa0 con 0x55910fefd7d0 2024-04-26T20:19:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 10 client.15702 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55910fb73210 con 0x55910fefd7d0 2024-04-26T20:19:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdb9600640 1 -- 172.21.15.131:0/2694084638 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc4004050 con 0x55910fefd7d0 2024-04-26T20:19:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdb9600640 1 -- 172.21.15.131:0/2694084638 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efdb40c5400 con 0x55910fefd7d0 2024-04-26T20:19:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 10 client.15702 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 20 client.15702 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff12a20 0x55910ff14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 4 client.15702 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:19:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.407+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55910fe6ef90 con 0x55910ff12a20 2024-04-26T20:19:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.408+0000 7efdcbe00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff12a20 0x55910ff14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.408+0000 7efdb9600640 10 client.15702 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.408+0000 7efdcbe00640 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff12a20 0x55910ff14e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528dd2d1efe1c1fb server_cookie=46a92a01fd6679a8 in_seq=0 out_seq=0 2024-04-26T20:19:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdb9600640 1 -- 172.21.15.131:0/2694084638 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55910fe6ef90 con 0x55910ff12a20 2024-04-26T20:19:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdb9600640 10 client.15702 handle_command_reply: tid=0 2024-04-26T20:19:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 client.15702 shutdown 2024-04-26T20:19:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 msgr2=0x55910ff07510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff07510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efdb40b96a0 tx=0x7efdb4009560 comp rx=0 tx=0).stop 2024-04-26T20:19:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 shutdown_connections 2024-04-26T20:19:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff12a20 0x55910ff14e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff07510 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdb9600640 0 client.15702 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/2694084638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff06fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.409+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 >> 172.21.15.131:0/2694084638 conn(0x55910fef8450 msgr2=0x55910fef7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.410+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 shutdown_connections 2024-04-26T20:19:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.410+0000 7efdd5bc2740 1 -- 172.21.15.131:0/2694084638 wait complete. 2024-04-26T20:19:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.410+0000 7efdd5bc2740 20 client.15702 trim_cache size 0 max 16384 2024-04-26T20:19:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 Processor -- start 2024-04-26T20:19:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 -- start start 2024-04-26T20:19:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fef32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefd7d0 0x55910ff0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910ff02940 0x55910fefbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1afe80 con 0x55910ff02940 2024-04-26T20:19:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1ad750 con 0x55910fefcde0 2024-04-26T20:19:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.426+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1ab5a0 con 0x55910fefd7d0 2024-04-26T20:19:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910ff02940 0x55910fefbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910ff02940 0x55910fefbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50772/0 (socket says 172.21.15.131:50772) 2024-04-26T20:19:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 -- 172.21.15.131:0/1824106092 learned_addr learned my addr 172.21.15.131:0/1824106092 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdcbe00640 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fef32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdc3e00640 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefd7d0 0x55910ff0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 -- 172.21.15.131:0/1824106092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefd7d0 msgr2=0x55910ff0e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefd7d0 0x55910ff0e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 -- 172.21.15.131:0/1824106092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 msgr2=0x55910fef32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fef32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 -- 172.21.15.131:0/1824106092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55910fc3f620 con 0x55910ff02940 2024-04-26T20:19:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.427+0000 7efdd0c00640 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910ff02940 0x55910fefbb10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009ed0 tx=0x7efdc4009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b552721a1a3b10b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdbbe00640 1 -- 172.21.15.131:0/1824106092 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4004510 con 0x55910ff02940 2024-04-26T20:19:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdbbe00640 1 -- 172.21.15.131:0/1824106092 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc40046b0 con 0x55910ff02940 2024-04-26T20:19:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdbbe00640 1 -- 172.21.15.131:0/1824106092 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40052a0 con 0x55910ff02940 2024-04-26T20:19:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1824106092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910ff02940 msgr2=0x55910fefbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910ff02940 0x55910fefbb10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009ed0 tx=0x7efdc4009ab0 comp rx=0 tx=0).stop 2024-04-26T20:19:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1824106092 shutdown_connections 2024-04-26T20:19:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910ff02940 0x55910fefbb10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefd7d0 0x55910ff0e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/1824106092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910fefcde0 0x55910fef32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1824106092 >> 172.21.15.131:0/1824106092 conn(0x55910fef8450 msgr2=0x55910fefb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1824106092 shutdown_connections 2024-04-26T20:19:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.428+0000 7efdd5bc2740 1 -- 172.21.15.131:0/1824106092 wait complete. 2024-04-26T20:19:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 -- start start 2024-04-26T20:19:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 Processor -- start 2024-04-26T20:19:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff0c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff0c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1abc10 con 0x55910fefd7d0 2024-04-26T20:19:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1ad080 con 0x55910ff02940 2024-04-26T20:19:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1ac2e0 con 0x55910fefcde0 2024-04-26T20:19:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.429+0000 7efdd5bc2740 10 client.0 _ll_register_callbacks cb 0x7efdb40f4690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:19:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff0c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff0c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50788/0 (socket says 172.21.15.131:50788) 2024-04-26T20:19:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 -- 172.21.15.131:0/3453105603 learned_addr learned my addr 172.21.15.131:0/3453105603 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdd0c00640 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdcbe00640 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff0c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 -- 172.21.15.131:0/3453105603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 msgr2=0x55910ff0c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff0c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 -- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 msgr2=0x55910ff0d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 -- 172.21.15.131:0/3453105603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910fe1bcb0 con 0x55910fefd7d0 2024-04-26T20:19:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdc3e00640 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff0c9a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efdb40bc290 tx=0x7efdb40c5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78330132d89b029 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdb9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:19:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.430+0000 7efdb9600640 1 -- 172.21.15.131:0/3453105603 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4049040 con 0x55910fefd7d0 2024-04-26T20:19:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdb9600640 1 -- 172.21.15.131:0/3453105603 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7efdb40b8a20 con 0x55910fefd7d0 2024-04-26T20:19:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55910fe68e00 con 0x55910fefd7d0 2024-04-26T20:19:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdb9600640 1 -- 172.21.15.131:0/3453105603 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdb40bc630 con 0x55910fefd7d0 2024-04-26T20:19:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 10 client.15708 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55910ff79610 con 0x55910fefd7d0 2024-04-26T20:19:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdb9600640 1 -- 172.21.15.131:0/3453105603 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55910fe68e00 con 0x55910fefd7d0 2024-04-26T20:19:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdb9600640 1 -- 172.21.15.131:0/3453105603 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efdb40b8c20 con 0x55910fefd7d0 2024-04-26T20:19:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 20 client.15708 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff79dc0 0x55910ff7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 4 client.15708 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:19:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.431+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55910fde44b0 con 0x55910ff79dc0 2024-04-26T20:19:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.432+0000 7efdcbe00640 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff79dc0 0x55910ff7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.432+0000 7efdcbe00640 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff79dc0 0x55910ff7c1e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4233ad5194dcd21 server_cookie=bdf922bd66d128d in_seq=0 out_seq=0 2024-04-26T20:19:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.432+0000 7efdb9600640 10 client.15708 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdb9600640 1 -- 172.21.15.131:0/3453105603 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55910fde44b0 con 0x55910ff79dc0 2024-04-26T20:19:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdb9600640 10 client.15708 handle_command_reply: tid=0 2024-04-26T20:19:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 client.15708 shutdown 2024-04-26T20:19:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 msgr2=0x55910ff0c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff0c9a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efdb40bc290 tx=0x7efdb40c5630 comp rx=0 tx=0).stop 2024-04-26T20:19:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 shutdown_connections 2024-04-26T20:19:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55910ff79dc0 0x55910ff7c1e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55910ff02940 0x55910ff0d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55910fefd7d0 0x55910ff0c9a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdb9600640 0 client.15708 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 --2- 172.21.15.131:0/3453105603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55910fefcde0 0x55910ff0c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 >> 172.21.15.131:0/3453105603 conn(0x55910fef8450 msgr2=0x55910fef6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 shutdown_connections 2024-04-26T20:19:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 1 -- 172.21.15.131:0/3453105603 wait complete. 2024-04-26T20:19:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.436+0000 7efdd5bc2740 20 client.15708 trim_cache size 0 max 16384 2024-04-26T20:19:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:19:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:19:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:19:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:19:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efdb00a3c30 msgr2=0x7efdb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efdb00a3c30 0x7efdb00a60f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55910fefdbd0 tx=0x7efdbc005f90 comp rx=0 tx=0).stop 2024-04-26T20:19:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdcc105bc0 msgr2=0x7efdcc19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdcc105bc0 0x7efdcc19b0f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006010 tx=0x7efdb40042f0 comp rx=0 tx=0).stop 2024-04-26T20:19:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 shutdown_connections 2024-04-26T20:19:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdcc106580 0x7efdcc1a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdcc105bc0 0x7efdcc19b0f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efdb00a3c30 0x7efdb00a60f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 --2- 172.21.15.131:0/1336725732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc1049d0 0x7efdcc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 >> 172.21.15.131:0/1336725732 conn(0x7efdcc0fff40 msgr2=0x7efdcc108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 shutdown_connections 2024-04-26T20:19:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:24.437+0000 7efdd1600640 1 -- 172.21.15.131:0/1336725732 wait complete. 2024-04-26T20:19:26.056 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:25 smithi131 ceph-mon[27919]: pgmap v632: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:26.062 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:26 smithi131 ceph-mon[27919]: pgmap v633: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:19:26.069 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:28 smithi131 ceph-mon[27919]: pgmap v634: 129 pgs: 129 active+clean; 669 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:19:26.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:30 smithi131 ceph-mon[27919]: pgmap v635: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:26.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:32 smithi131 ceph-mon[27919]: pgmap v636: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:26.232 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:34 smithi131 ceph-mon[27919]: pgmap v637: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:26.352 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:36 smithi131 ceph-mon[27919]: pgmap v638: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:26.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:09 smithi157 ceph-mon[32703]: pgmap v684: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:26.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:11 smithi157 ceph-mon[32703]: pgmap v685: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:26.576 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:12 smithi157 ceph-mon[32703]: pgmap v686: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:19:26.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:14 smithi157 ceph-mon[32703]: pgmap v687: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:26.587 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:16 smithi157 ceph-mon[32703]: pgmap v688: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:26.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:19 smithi157 ceph-mon[32703]: pgmap v689: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:26.758 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:21 smithi157 ceph-mon[32703]: pgmap v690: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:26.867 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:23 smithi157 ceph-mon[32703]: pgmap v691: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:26.995 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:25 smithi157 ceph-mon[32703]: pgmap v692: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:27.121 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:09 smithi148 ceph-mon[32281]: pgmap v684: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:27.260 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:11 smithi148 ceph-mon[32281]: pgmap v685: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:27.282 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:12 smithi148 ceph-mon[32281]: pgmap v686: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:19:27.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:14 smithi148 ceph-mon[32281]: pgmap v687: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:27.384 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:16 smithi148 ceph-mon[32281]: pgmap v688: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:27.484 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:19 smithi148 ceph-mon[32281]: pgmap v689: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:27.693 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:21 smithi148 ceph-mon[32281]: pgmap v690: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:27.779 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:23 smithi148 ceph-mon[32281]: pgmap v691: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:27.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:25 smithi148 ceph-mon[32281]: pgmap v692: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:28.047 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:27 smithi157 ceph-mon[32703]: pgmap v693: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:19:28.162 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:27 smithi148 ceph-mon[32281]: pgmap v693: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 -- 172.21.15.131:0/4036617427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078530 msgr2=0x7fc864078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 --2- 172.21.15.131:0/4036617427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078530 0x7fc864078990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc850009900 tx=0x7fc85002a410 comp rx=0 tx=0).stop 2024-04-26T20:19:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 -- 172.21.15.131:0/4036617427 shutdown_connections 2024-04-26T20:19:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 --2- 172.21.15.131:0/4036617427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc864078ed0 0x7fc864079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 --2- 172.21.15.131:0/4036617427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078530 0x7fc864078990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 --2- 172.21.15.131:0/4036617427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 0x7fc864077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 -- 172.21.15.131:0/4036617427 >> 172.21.15.131:0/4036617427 conn(0x7fc864100150 msgr2=0x7fc864102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 -- 172.21.15.131:0/4036617427 shutdown_connections 2024-04-26T20:19:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.578+0000 7fc869200640 1 -- 172.21.15.131:0/4036617427 wait complete. 2024-04-26T20:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.579+0000 7fc869200640 1 Processor -- start 2024-04-26T20:19:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.579+0000 7fc869200640 1 -- start start 2024-04-26T20:19:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc869200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 0x7fc86419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc869200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc864078530 0x7fc86419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc869200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078ed0 0x7fc8641a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc869200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc864114b40 con 0x7fc864078ed0 2024-04-26T20:19:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc869200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641149c0 con 0x7fc8640772f0 2024-04-26T20:19:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc869200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc864114cc0 con 0x7fc864078530 2024-04-26T20:19:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc861e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 0x7fc86419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:29.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc861e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 0x7fc86419f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50524/0 (socket says 172.21.15.131:50524) 2024-04-26T20:19:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc862800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078ed0 0x7fc8641a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc861e00640 1 -- 172.21.15.131:0/2517583529 learned_addr learned my addr 172.21.15.131:0/2517583529 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc861400640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc864078530 0x7fc86419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc862800640 1 -- 172.21.15.131:0/2517583529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc864078530 msgr2=0x7fc86419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc862800640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc864078530 0x7fc86419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc862800640 1 -- 172.21.15.131:0/2517583529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 msgr2=0x7fc86419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc862800640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 0x7fc86419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc862800640 1 -- 172.21.15.131:0/2517583529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8641a6d30 con 0x7fc864078ed0 2024-04-26T20:19:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.580+0000 7fc861e00640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 0x7fc86419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.581+0000 7fc862800640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078ed0 0x7fc8641a66c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e940 tx=0x7fc85c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.581+0000 7fc85aa00640 1 -- 172.21.15.131:0/2517583529 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c004d20 con 0x7fc864078ed0 2024-04-26T20:19:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.581+0000 7fc85aa00640 1 -- 172.21.15.131:0/2517583529 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c005020 con 0x7fc864078ed0 2024-04-26T20:19:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.581+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8641a6fc0 con 0x7fc864078ed0 2024-04-26T20:19:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.581+0000 7fc85aa00640 1 -- 172.21.15.131:0/2517583529 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0116d0 con 0x7fc864078ed0 2024-04-26T20:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.581+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8641a7490 con 0x7fc864078ed0 2024-04-26T20:19:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 Processor -- start 2024-04-26T20:19:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc85aa00640 1 -- 172.21.15.131:0/2517583529 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc85c005460 con 0x7fc864078ed0 2024-04-26T20:19:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 -- start start 2024-04-26T20:19:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f712ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f7180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f71abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555d0f677380 con 0x555d0f7135c0 2024-04-26T20:19:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555d0f613c30 con 0x555d0f718730 2024-04-26T20:19:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.585+0000 7fc86bff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555d0f673340 con 0x555d0f712bd0 2024-04-26T20:19:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f7180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f7180c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57342/0 (socket says 172.21.15.131:57342) 2024-04-26T20:19:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 -- 172.21.15.131:0/2838149791 learned_addr learned my addr 172.21.15.131:0/2838149791 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861400640 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f712ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861400640 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f71abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 -- 172.21.15.131:0/2838149791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 msgr2=0x555d0f712ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f712ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 -- 172.21.15.131:0/2838149791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 msgr2=0x555d0f71abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f71abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861e00640 1 -- 172.21.15.131:0/2838149791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0f635940 con 0x555d0f7135c0 2024-04-26T20:19:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.586+0000 7fc861400640 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f71abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.587+0000 7fc85aa00640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc8480a3cf0 0x7fc8480a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.587+0000 7fc861e00640 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f7180c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc854011650 tx=0x7fc854009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8990207084454aab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.587+0000 7fc85aa00640 1 -- 172.21.15.131:0/2517583529 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc85c0d0a50 con 0x7fc864078ed0 2024-04-26T20:19:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.587+0000 7fc84f400640 1 -- 172.21.15.131:0/2838149791 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85401e070 con 0x555d0f7135c0 2024-04-26T20:19:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.587+0000 7fc861400640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc8480a3cf0 0x7fc8480a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.587+0000 7fc84f400640 1 -- 172.21.15.131:0/2838149791 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85400fb70 con 0x555d0f7135c0 2024-04-26T20:19:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.587+0000 7fc84f400640 1 -- 172.21.15.131:0/2838149791 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854019bf0 con 0x555d0f7135c0 2024-04-26T20:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 -- 172.21.15.131:0/2838149791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 msgr2=0x555d0f7180c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc861400640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc8480a3cf0 0x7fc8480a61b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc850031500 tx=0x7fc850005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f7180c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc854011650 tx=0x7fc854009f90 comp rx=0 tx=0).stop 2024-04-26T20:19:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 -- 172.21.15.131:0/2838149791 shutdown_connections 2024-04-26T20:19:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f71abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f7180c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 --2- 172.21.15.131:0/2838149791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f712ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 -- 172.21.15.131:0/2838149791 >> 172.21.15.131:0/2838149791 conn(0x555d0f70e240 msgr2=0x555d0f70e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 -- 172.21.15.131:0/2838149791 shutdown_connections 2024-04-26T20:19:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.588+0000 7fc86bff9740 1 -- 172.21.15.131:0/2838149791 wait complete. 2024-04-26T20:19:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 -- start start 2024-04-26T20:19:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 Processor -- start 2024-04-26T20:19:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f712bd0 0x555d0f71ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f7135c0 0x555d0f71d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f724390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555d0f71b6e0 con 0x555d0f712bd0 2024-04-26T20:19:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555d0f5fec60 con 0x555d0f718730 2024-04-26T20:19:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555d0f71b860 con 0x555d0f7135c0 2024-04-26T20:19:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.589+0000 7fc86bff9740 10 client.0 _ll_register_callbacks cb 0x555d0f62ae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:19:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f712bd0 0x555d0f71ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f7135c0 0x555d0f71d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f7135c0 0x555d0f71d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53082/0 (socket says 172.21.15.131:53082) 2024-04-26T20:19:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 -- 172.21.15.131:0/4149456778 learned_addr learned my addr 172.21.15.131:0/4149456778 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc862800640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f724390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 -- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 msgr2=0x555d0f724390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f724390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 -- 172.21.15.131:0/4149456778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f712bd0 msgr2=0x555d0f71ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f712bd0 0x555d0f71ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861400640 1 -- 172.21.15.131:0/4149456778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0f60d510 con 0x555d0f7135c0 2024-04-26T20:19:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc862800640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f724390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.590+0000 7fc861e00640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f712bd0 0x555d0f71ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc861400640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f7135c0 0x555d0f71d350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc850033000 tx=0x7fc850009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1653f245693df125 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc843e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc843e00640 1 -- 172.21.15.131:0/4149456778 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850035040 con 0x555d0f7135c0 2024-04-26T20:19:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85400fb70 con 0x555d0f7135c0 2024-04-26T20:19:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc843e00640 1 -- 172.21.15.131:0/4149456778 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc8500317b0 con 0x555d0f7135c0 2024-04-26T20:19:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc843e00640 1 -- 172.21.15.131:0/4149456778 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8500319b0 con 0x555d0f7135c0 2024-04-26T20:19:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc86bff9740 10 client.25831 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.591+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d0f389210 con 0x555d0f7135c0 2024-04-26T20:19:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc843e00640 1 -- 172.21.15.131:0/4149456778 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85400fb70 con 0x555d0f7135c0 2024-04-26T20:19:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc843e00640 1 -- 172.21.15.131:0/4149456778 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc850054a50 con 0x555d0f7135c0 2024-04-26T20:19:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc86bff9740 10 client.25831 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc86bff9740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc86bff9740 20 client.25831 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc86bff9740 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f7288e0 0x555d0f72ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc86bff9740 4 client.25831 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:19:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d0f684f90 con 0x555d0f7288e0 2024-04-26T20:19:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.592+0000 7fc861e00640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f7288e0 0x555d0f72ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.593+0000 7fc861e00640 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f7288e0 0x555d0f72ad30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51307953464cab5 server_cookie=7d810501ccdd418f in_seq=0 out_seq=0 2024-04-26T20:19:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.593+0000 7fc843e00640 10 client.25831 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc843e00640 1 -- 172.21.15.131:0/4149456778 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d0f684f90 con 0x555d0f7288e0 2024-04-26T20:19:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc843e00640 10 client.25831 handle_command_reply: tid=0 2024-04-26T20:19:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 client.25831 shutdown 2024-04-26T20:19:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f7135c0 msgr2=0x555d0f71d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f7135c0 0x555d0f71d350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc850033000 tx=0x7fc850009260 comp rx=0 tx=0).stop 2024-04-26T20:19:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 shutdown_connections 2024-04-26T20:19:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f7288e0 0x555d0f72ad30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f718730 0x555d0f724390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f7135c0 0x555d0f71d350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc843e00640 0 client.25831 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 --2- 172.21.15.131:0/4149456778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f712bd0 0x555d0f71ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 >> 172.21.15.131:0/4149456778 conn(0x555d0f70e240 msgr2=0x555d0f70d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 shutdown_connections 2024-04-26T20:19:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 1 -- 172.21.15.131:0/4149456778 wait complete. 2024-04-26T20:19:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.594+0000 7fc86bff9740 20 client.25831 trim_cache size 0 max 16384 2024-04-26T20:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.610+0000 7fc86bff9740 1 Processor -- start 2024-04-26T20:19:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.610+0000 7fc86bff9740 1 -- start start 2024-04-26T20:19:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f712bd0 0x555d0f7111f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f724350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f718730 0x555d0f711730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc86bff9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641b44e0 con 0x555d0f7135c0 2024-04-26T20:19:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc86bff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641b1db0 con 0x555d0f712bd0 2024-04-26T20:19:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc86bff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641afc00 con 0x555d0f718730 2024-04-26T20:19:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f712bd0 0x555d0f7111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc862800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f718730 0x555d0f711730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f712bd0 0x555d0f7111f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50560/0 (socket says 172.21.15.131:50560) 2024-04-26T20:19:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 -- 172.21.15.131:0/1353369723 learned_addr learned my addr 172.21.15.131:0/1353369723 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861400640 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f724350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 -- 172.21.15.131:0/1353369723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f718730 msgr2=0x555d0f711730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f718730 0x555d0f711730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 -- 172.21.15.131:0/1353369723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 msgr2=0x555d0f724350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f724350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.611+0000 7fc861e00640 1 -- 172.21.15.131:0/1353369723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0f455620 con 0x555d0f712bd0 2024-04-26T20:19:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.612+0000 7fc861e00640 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f712bd0 0x555d0f7111f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc85401c040 tx=0x7fc854005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b161520c15694e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.612+0000 7fc84ea00640 1 -- 172.21.15.131:0/1353369723 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854029070 con 0x555d0f712bd0 2024-04-26T20:19:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.612+0000 7fc84ea00640 1 -- 172.21.15.131:0/1353369723 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8540054f0 con 0x555d0f712bd0 2024-04-26T20:19:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.612+0000 7fc84ea00640 1 -- 172.21.15.131:0/1353369723 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85401e040 con 0x555d0f712bd0 2024-04-26T20:19:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 -- 172.21.15.131:0/1353369723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f712bd0 msgr2=0x555d0f7111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f712bd0 0x555d0f7111f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc85401c040 tx=0x7fc854005c70 comp rx=0 tx=0).stop 2024-04-26T20:19:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 -- 172.21.15.131:0/1353369723 shutdown_connections 2024-04-26T20:19:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f718730 0x555d0f711730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f7135c0 0x555d0f724350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 --2- 172.21.15.131:0/1353369723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f712bd0 0x555d0f7111f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 -- 172.21.15.131:0/1353369723 >> 172.21.15.131:0/1353369723 conn(0x555d0f70e240 msgr2=0x555d0f7147e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 -- 172.21.15.131:0/1353369723 shutdown_connections 2024-04-26T20:19:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 -- 172.21.15.131:0/1353369723 wait complete. 2024-04-26T20:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.613+0000 7fc86bff9740 1 -- start start 2024-04-26T20:19:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 1 Processor -- start 2024-04-26T20:19:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f721140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f7135c0 0x555d0f721680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f718730 0x555d0f78cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641b0270 con 0x555d0f718730 2024-04-26T20:19:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641b16e0 con 0x555d0f7135c0 2024-04-26T20:19:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641b0940 con 0x555d0f712bd0 2024-04-26T20:19:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc86bff9740 10 client.0 _ll_register_callbacks cb 0x7fc850003800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:19:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc861e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f721140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc862800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f718730 0x555d0f78cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc861e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f721140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53102/0 (socket says 172.21.15.131:53102) 2024-04-26T20:19:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc861e00640 1 -- 172.21.15.131:0/134831783 learned_addr learned my addr 172.21.15.131:0/134831783 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.614+0000 7fc861400640 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f7135c0 0x555d0f721680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc861e00640 1 -- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f7135c0 msgr2=0x555d0f721680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc861e00640 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f7135c0 0x555d0f721680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc861e00640 1 -- 172.21.15.131:0/134831783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f718730 msgr2=0x555d0f78cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc861e00640 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f718730 0x555d0f78cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc861e00640 1 -- 172.21.15.131:0/134831783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0f631cb0 con 0x555d0f712bd0 2024-04-26T20:19:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc861e00640 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f721140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc854009c90 tx=0x7fc854019560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9f1d44c7f9d777 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc843e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc843e00640 1 -- 172.21.15.131:0/134831783 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85401e070 con 0x555d0f712bd0 2024-04-26T20:19:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc843e00640 1 -- 172.21.15.131:0/134831783 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc8540196c0 con 0x555d0f712bd0 2024-04-26T20:19:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8540054f0 con 0x555d0f712bd0 2024-04-26T20:19:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc843e00640 1 -- 172.21.15.131:0/134831783 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc85403c050 con 0x555d0f712bd0 2024-04-26T20:19:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc86bff9740 10 client.25837 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.615+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d0f78e390 con 0x555d0f712bd0 2024-04-26T20:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc843e00640 1 -- 172.21.15.131:0/134831783 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8540054f0 con 0x555d0f712bd0 2024-04-26T20:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc843e00640 1 -- 172.21.15.131:0/134831783 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc8540050a0 con 0x555d0f712bd0 2024-04-26T20:19:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc86bff9740 10 client.25837 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc86bff9740 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc86bff9740 20 client.25837 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc86bff9740 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f78eaa0 0x555d0f790ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc86bff9740 4 client.25837 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:19:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d0f67ee00 con 0x555d0f78eaa0 2024-04-26T20:19:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.616+0000 7fc861400640 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f78eaa0 0x555d0f790ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.617+0000 7fc861400640 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f78eaa0 0x555d0f790ec0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a457d55a73ba6a6 server_cookie=f6c8a27c81287cc3 in_seq=0 out_seq=0 2024-04-26T20:19:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.617+0000 7fc843e00640 10 client.25837 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.620+0000 7fc843e00640 1 -- 172.21.15.131:0/134831783 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d0f67ee00 con 0x555d0f78eaa0 2024-04-26T20:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.620+0000 7fc843e00640 10 client.25837 handle_command_reply: tid=0 2024-04-26T20:19:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.620+0000 7fc86bff9740 1 client.25837 shutdown 2024-04-26T20:19:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 msgr2=0x555d0f721140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f721140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc854009c90 tx=0x7fc854019560 comp rx=0 tx=0).stop 2024-04-26T20:19:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 shutdown_connections 2024-04-26T20:19:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555d0f78eaa0 0x555d0f790ec0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d0f718730 0x555d0f78cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d0f7135c0 0x555d0f721680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 --2- 172.21.15.131:0/134831783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d0f712bd0 0x555d0f721140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc843e00640 0 client.25837 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 >> 172.21.15.131:0/134831783 conn(0x555d0f70e240 msgr2=0x555d0f7111f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 shutdown_connections 2024-04-26T20:19:42.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 1 -- 172.21.15.131:0/134831783 wait complete. 2024-04-26T20:19:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.621+0000 7fc86bff9740 20 client.25837 trim_cache size 0 max 16384 2024-04-26T20:19:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:19:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:19:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:19:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:19:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc8480a3cf0 msgr2=0x7fc8480a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc8480a3cf0 0x7fc8480a61b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc850031500 tx=0x7fc850005a20 comp rx=0 tx=0).stop 2024-04-26T20:19:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078ed0 msgr2=0x7fc8641a66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078ed0 0x7fc8641a66c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e940 tx=0x7fc85c00ee00 comp rx=0 tx=0).stop 2024-04-26T20:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 shutdown_connections 2024-04-26T20:19:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc864078ed0 0x7fc8641a66c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc8480a3cf0 0x7fc8480a61b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc864078530 0x7fc86419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 --2- 172.21.15.131:0/2517583529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8640772f0 0x7fc86419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 >> 172.21.15.131:0/2517583529 conn(0x7fc864100150 msgr2=0x7fc864101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 shutdown_connections 2024-04-26T20:19:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:37.622+0000 7fc869200640 1 -- 172.21.15.131:0/2517583529 wait complete. 2024-04-26T20:19:43.082 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:29 smithi148 ceph-mon[32281]: pgmap v694: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:19:43.158 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:31 smithi148 ceph-mon[32281]: pgmap v695: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:43.283 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:33 smithi148 ceph-mon[32281]: pgmap v696: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:43.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:35 smithi148 ceph-mon[32281]: pgmap v697: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:43.523 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:37 smithi148 ceph-mon[32281]: pgmap v698: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:43.622 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:39 smithi148 ceph-mon[32281]: pgmap v699: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:43.630 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:41 smithi148 ceph-mon[32281]: pgmap v700: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:19:43.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:29 smithi157 ceph-mon[32703]: pgmap v694: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:19:43.832 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:31 smithi157 ceph-mon[32703]: pgmap v695: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:43.865 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:33 smithi157 ceph-mon[32703]: pgmap v696: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:43.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:35 smithi157 ceph-mon[32703]: pgmap v697: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:43.971 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:37 smithi157 ceph-mon[32703]: pgmap v698: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:43.978 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:39 smithi157 ceph-mon[32703]: pgmap v699: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:44.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:41 smithi157 ceph-mon[32703]: pgmap v700: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:19:44.182 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:43 smithi148 ceph-mon[32281]: pgmap v701: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:44.310 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:43 smithi157 ceph-mon[32703]: pgmap v701: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:44.403 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:38 smithi131 ceph-mon[27919]: pgmap v639: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:44.474 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:40 smithi131 ceph-mon[27919]: pgmap v640: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:19:44.539 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:41 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:19:44.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:42 smithi131 ceph-mon[27919]: pgmap v641: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:44.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:43 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:19:44.831 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:43 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:19:44.916 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:43 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:19:44.922 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:45 smithi131 ceph-mon[27919]: pgmap v642: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:19:45.042 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:47 smithi131 ceph-mon[27919]: pgmap v643: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:19:45.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:49 smithi131 ceph-mon[27919]: pgmap v644: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:19:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 -- 172.21.15.131:0/674633678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 msgr2=0x7f7cb0074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 --2- 172.21.15.131:0/674633678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 0x7f7cb0074d60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009860 tx=0x7f7ca002a3d0 comp rx=0 tx=0).stop 2024-04-26T20:19:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 -- 172.21.15.131:0/674633678 shutdown_connections 2024-04-26T20:19:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 --2- 172.21.15.131:0/674633678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb010a680 0x7f7cb010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 --2- 172.21.15.131:0/674633678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb0075330 0x7f7cb0075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 --2- 172.21.15.131:0/674633678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 0x7f7cb0074d60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 -- 172.21.15.131:0/674633678 >> 172.21.15.131:0/674633678 conn(0x7f7cb00fdd60 msgr2=0x7f7cb01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.773+0000 7f7cafe00640 1 -- 172.21.15.131:0/674633678 shutdown_connections 2024-04-26T20:19:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.774+0000 7f7cafe00640 1 -- 172.21.15.131:0/674633678 wait complete. 2024-04-26T20:19:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 Processor -- start 2024-04-26T20:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 -- start start 2024-04-26T20:19:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb0075330 0x7f7cb019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 0x7f7cb019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb010a680 0x7f7cb01a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0110560 con 0x7f7cb0076910 2024-04-26T20:19:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01103e0 con 0x7f7cb010a680 2024-04-26T20:19:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.775+0000 7f7cafe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01106e0 con 0x7f7cb0075330 2024-04-26T20:19:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb0075330 0x7f7cb019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb0075330 0x7f7cb019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44860/0 (socket says 172.21.15.131:44860) 2024-04-26T20:19:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 -- 172.21.15.131:0/204393986 learned_addr learned my addr 172.21.15.131:0/204393986 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caf400640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb010a680 0x7f7cb01a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7cae000640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 0x7f7cb019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 -- 172.21.15.131:0/204393986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb010a680 msgr2=0x7f7cb01a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb010a680 0x7f7cb01a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 -- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 msgr2=0x7f7cb019b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 0x7f7cb019b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.776+0000 7f7caea00640 1 -- 172.21.15.131:0/204393986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb01a27e0 con 0x7f7cb0075330 2024-04-26T20:19:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.777+0000 7f7caea00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb0075330 0x7f7cb019abc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00308b0 tx=0x7f7ca00050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.777+0000 7f7cb4a00640 1 -- 172.21.15.131:0/204393986 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0004050 con 0x7f7cb0075330 2024-04-26T20:19:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.777+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb01a2a70 con 0x7f7cb0075330 2024-04-26T20:19:46.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.777+0000 7f7cb4a00640 1 -- 172.21.15.131:0/204393986 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca00318b0 con 0x7f7cb0075330 2024-04-26T20:19:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.777+0000 7f7cb4a00640 1 -- 172.21.15.131:0/204393986 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0004a20 con 0x7f7cb0075330 2024-04-26T20:19:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.777+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb01a2f60 con 0x7f7cb0075330 2024-04-26T20:19:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.780+0000 7f7cb8c35740 1 Processor -- start 2024-04-26T20:19:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.781+0000 7f7cb8c35740 1 -- start start 2024-04-26T20:19:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.781+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bcf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.781+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 0x559873bd42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.781+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bd6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.781+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559873b33380 con 0x559873bd4940 2024-04-26T20:19:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.781+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559873acfc30 con 0x559873bcede0 2024-04-26T20:19:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.781+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559873b2f340 con 0x559873bcf7d0 2024-04-26T20:19:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7cae000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bd6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7caf400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bcf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7cae000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bd6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34244/0 (socket says 172.21.15.131:34244) 2024-04-26T20:19:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7cae000640 1 -- 172.21.15.131:0/2228052053 learned_addr learned my addr 172.21.15.131:0/2228052053 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7cb4a00640 1 -- 172.21.15.131:0/204393986 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7ca0039040 con 0x7f7cb0075330 2024-04-26T20:19:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7caf400640 1 -- 172.21.15.131:0/2228052053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 msgr2=0x559873bd42d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:19:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7caf400640 1 --2- 172.21.15.131:0/2228052053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 0x559873bd42d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7caf400640 1 -- 172.21.15.131:0/2228052053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 msgr2=0x559873bd6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7caf400640 1 --2- 172.21.15.131:0/2228052053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bd6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7caf400640 1 -- 172.21.15.131:0/2228052053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559873af1940 con 0x559873bcede0 2024-04-26T20:19:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.782+0000 7f7caf400640 1 --2- 172.21.15.131:0/2228052053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bcf200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009f30 tx=0x7f7ca8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d318c80cb38ba222 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.783+0000 7f7c9be00640 1 -- 172.21.15.131:0/2228052053 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8016070 con 0x559873bcede0 2024-04-26T20:19:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.783+0000 7f7c9be00640 1 -- 172.21.15.131:0/2228052053 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca8004d20 con 0x559873bcede0 2024-04-26T20:19:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.783+0000 7f7c9be00640 1 -- 172.21.15.131:0/2228052053 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80127d0 con 0x559873bcede0 2024-04-26T20:19:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 -- 172.21.15.131:0/2228052053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 msgr2=0x559873bcf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/2228052053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bcf200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009f30 tx=0x7f7ca8010800 comp rx=0 tx=0).stop 2024-04-26T20:19:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 -- 172.21.15.131:0/2228052053 shutdown_connections 2024-04-26T20:19:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/2228052053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bd6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/2228052053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 0x559873bd42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/2228052053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bcf200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 -- 172.21.15.131:0/2228052053 >> 172.21.15.131:0/2228052053 conn(0x559873bca450 msgr2=0x559873bca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb4a00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c940a3c30 0x7f7c940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 -- 172.21.15.131:0/2228052053 shutdown_connections 2024-04-26T20:19:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb4a00640 1 -- 172.21.15.131:0/204393986 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7ca00f1c70 con 0x7f7cb0075330 2024-04-26T20:19:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.784+0000 7f7cb8c35740 1 -- 172.21.15.131:0/2228052053 wait complete. 2024-04-26T20:19:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.785+0000 7f7cb8c35740 1 -- start start 2024-04-26T20:19:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.785+0000 7f7cae000640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c940a3c30 0x7f7c940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 1 Processor -- start 2024-04-26T20:19:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cae000640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c940a3c30 0x7f7c940a60f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x559873bcb3c0 tx=0x7f7c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873bd95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873be05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559873bd78f0 con 0x559873bcf7d0 2024-04-26T20:19:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559873abac60 con 0x559873bd4940 2024-04-26T20:19:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559873bd7a70 con 0x559873bcede0 2024-04-26T20:19:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.786+0000 7f7cb8c35740 10 client.0 _ll_register_callbacks cb 0x559873ae6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:19:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44872/0 (socket says 172.21.15.131:44872) 2024-04-26T20:19:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 -- 172.21.15.131:0/46788439 learned_addr learned my addr 172.21.15.131:0/46788439 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caf400640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873be05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7cae000640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873bd95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 -- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 msgr2=0x559873be05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873be05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 -- 172.21.15.131:0/46788439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 msgr2=0x559873bd95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873bd95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 -- 172.21.15.131:0/46788439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559873ac9510 con 0x559873bcede0 2024-04-26T20:19:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caf400640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873be05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7caea00640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd9090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0005e00 tx=0x7f7ca00bbeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93be4f459795ddae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.787+0000 7f7c98c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:19:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.788+0000 7f7c98c00640 1 -- 172.21.15.131:0/46788439 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00bd9a0 con 0x559873bcede0 2024-04-26T20:19:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.788+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca8004d20 con 0x559873bcede0 2024-04-26T20:19:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.788+0000 7f7c98c00640 1 -- 172.21.15.131:0/46788439 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7ca00bdc00 con 0x559873bcede0 2024-04-26T20:19:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.788+0000 7f7c98c00640 1 -- 172.21.15.131:0/46788439 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca00c1d50 con 0x559873bcede0 2024-04-26T20:19:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.788+0000 7f7cb8c35740 10 client.25846 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.788+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559873845210 con 0x559873bcede0 2024-04-26T20:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7c98c00640 1 -- 172.21.15.131:0/46788439 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca8004d20 con 0x559873bcede0 2024-04-26T20:19:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7c98c00640 1 -- 172.21.15.131:0/46788439 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7ca0004bc0 con 0x559873bcede0 2024-04-26T20:19:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7cb8c35740 10 client.25846 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7cb8c35740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7cb8c35740 20 client.25846 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873be4a40 0x559873be6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7cb8c35740 4 client.25846 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:19:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559873b40f90 con 0x559873be4a40 2024-04-26T20:19:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.789+0000 7f7cae000640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873be4a40 0x559873be6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.790+0000 7f7c98c00640 10 client.25846 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.790+0000 7f7cae000640 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873be4a40 0x559873be6e80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70eaee195bc19bf1 server_cookie=a2a7b22854a6ab98 in_seq=0 out_seq=0 2024-04-26T20:19:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.791+0000 7f7c98c00640 1 -- 172.21.15.131:0/46788439 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559873b40f90 con 0x559873be4a40 2024-04-26T20:19:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.791+0000 7f7c98c00640 10 client.25846 handle_command_reply: tid=0 2024-04-26T20:19:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.791+0000 7f7cb8c35740 1 client.25846 shutdown 2024-04-26T20:19:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 msgr2=0x559873bd9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd9090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0005e00 tx=0x7f7ca00bbeb0 comp rx=0 tx=0).stop 2024-04-26T20:19:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 shutdown_connections 2024-04-26T20:19:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873be4a40 0x559873be6e80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873be05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873bd95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/46788439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd9090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 >> 172.21.15.131:0/46788439 conn(0x559873bca450 msgr2=0x559873bc9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7c98c00640 0 client.25846 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 shutdown_connections 2024-04-26T20:19:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 1 -- 172.21.15.131:0/46788439 wait complete. 2024-04-26T20:19:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.792+0000 7f7cb8c35740 20 client.25846 trim_cache size 0 max 16384 2024-04-26T20:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.811+0000 7f7cb8c35740 1 Processor -- start 2024-04-26T20:19:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cb8c35740 1 -- start start 2024-04-26T20:19:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873be0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873bcdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01afda0 con 0x559873bcf7d0 2024-04-26T20:19:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01ad670 con 0x559873bd4940 2024-04-26T20:19:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01ab4c0 con 0x559873bcede0 2024-04-26T20:19:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cae000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873be0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cae000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873be0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34262/0 (socket says 172.21.15.131:34262) 2024-04-26T20:19:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7cae000640 1 -- 172.21.15.131:0/1653545956 learned_addr learned my addr 172.21.15.131:0/1653545956 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7caea00640 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.812+0000 7f7caf400640 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873bcdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.813+0000 7f7cae000640 1 -- 172.21.15.131:0/1653545956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 msgr2=0x559873bd42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.813+0000 7f7cae000640 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.813+0000 7f7cae000640 1 -- 172.21.15.131:0/1653545956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 msgr2=0x559873bcdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.813+0000 7f7cae000640 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873bcdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.813+0000 7f7cae000640 1 -- 172.21.15.131:0/1653545956 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559873911620 con 0x559873bcf7d0 2024-04-26T20:19:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.813+0000 7f7caea00640 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.813+0000 7f7cae000640 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873be0530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005fd0 tx=0x7f7c9c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af4df9c5937c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.814+0000 7f7c9b400640 1 -- 172.21.15.131:0/1653545956 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c01cdc0 con 0x559873bcf7d0 2024-04-26T20:19:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.814+0000 7f7c9b400640 1 -- 172.21.15.131:0/1653545956 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c9c02f070 con 0x559873bcf7d0 2024-04-26T20:19:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.814+0000 7f7c9b400640 1 -- 172.21.15.131:0/1653545956 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00fc30 con 0x559873bcf7d0 2024-04-26T20:19:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.814+0000 7f7cb8c35740 1 -- 172.21.15.131:0/1653545956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 msgr2=0x559873be0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.814+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873be0530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005fd0 tx=0x7f7c9c018000 comp rx=0 tx=0).stop 2024-04-26T20:19:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 -- 172.21.15.131:0/1653545956 shutdown_connections 2024-04-26T20:19:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bd4940 0x559873bcdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bcf7d0 0x559873be0530 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/1653545956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcede0 0x559873bd42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 -- 172.21.15.131:0/1653545956 >> 172.21.15.131:0/1653545956 conn(0x559873bca450 msgr2=0x559873bcd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 -- 172.21.15.131:0/1653545956 shutdown_connections 2024-04-26T20:19:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 -- 172.21.15.131:0/1653545956 wait complete. 2024-04-26T20:19:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.815+0000 7f7cb8c35740 1 -- start start 2024-04-26T20:19:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 1 Processor -- start 2024-04-26T20:19:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bdd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 0x559873bddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bdec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01abb30 con 0x559873bd4940 2024-04-26T20:19:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01acfa0 con 0x559873bcede0 2024-04-26T20:19:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01ac200 con 0x559873bcf7d0 2024-04-26T20:19:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7cb8c35740 10 client.0 _ll_register_callbacks cb 0x7f7ca00bd840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:19:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7caea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bdd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7caea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bdd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:49740/0 (socket says 172.21.15.131:49740) 2024-04-26T20:19:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.816+0000 7f7caea00640 1 -- 172.21.15.131:0/674400711 learned_addr learned my addr 172.21.15.131:0/674400711 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:19:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7caf400640 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bdec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7cae000640 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 0x559873bddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7caea00640 1 -- 172.21.15.131:0/674400711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 msgr2=0x559873bddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7caea00640 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 0x559873bddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7caea00640 1 -- 172.21.15.131:0/674400711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 msgr2=0x559873bdec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7caea00640 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bdec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7caea00640 1 -- 172.21.15.131:0/674400711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559873aedcb0 con 0x559873bcede0 2024-04-26T20:19:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7caea00640 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bdd7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00bbc50 tx=0x7f7ca00ef650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8804b6343a9b9e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7c98c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:19:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7c98c00640 1 -- 172.21.15.131:0/674400711 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00bd9a0 con 0x559873bcede0 2024-04-26T20:19:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c9c02f070 con 0x559873bcede0 2024-04-26T20:19:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.817+0000 7f7c98c00640 1 -- 172.21.15.131:0/674400711 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7ca00bdc00 con 0x559873bcede0 2024-04-26T20:19:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7c98c00640 1 -- 172.21.15.131:0/674400711 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca00c6180 con 0x559873bcede0 2024-04-26T20:19:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 10 client.25862 fetch_fsmap learned FSMap version 27 2024-04-26T20:19:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559873be03b0 con 0x559873bcede0 2024-04-26T20:19:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7c98c00640 1 -- 172.21.15.131:0/674400711 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c9c02f070 con 0x559873bcede0 2024-04-26T20:19:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7c98c00640 1 -- 172.21.15.131:0/674400711 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7ca00bcdf0 con 0x559873bcede0 2024-04-26T20:19:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 10 client.25862 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:19:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:19:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 20 client.25862 populate_metadata read hostname 'smithi131' 2024-04-26T20:19:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873c4b0c0 0x559873c4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 4 client.25862 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:19:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.818+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559873b3ae00 con 0x559873c4b0c0 2024-04-26T20:19:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.819+0000 7f7cae000640 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873c4b0c0 0x559873c4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.819+0000 7f7cae000640 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873c4b0c0 0x559873c4d4b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8693dec091bd3abb server_cookie=a436c44e7e4b9fd3 in_seq=0 out_seq=0 2024-04-26T20:19:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.819+0000 7f7c98c00640 10 client.25862 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7c98c00640 1 -- 172.21.15.131:0/674400711 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559873b3ae00 con 0x559873c4b0c0 2024-04-26T20:19:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7c98c00640 10 client.25862 handle_command_reply: tid=0 2024-04-26T20:19:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 client.25862 shutdown 2024-04-26T20:19:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 msgr2=0x559873bdd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bdd7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00bbc50 tx=0x7f7ca00ef650 comp rx=0 tx=0).stop 2024-04-26T20:19:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 shutdown_connections 2024-04-26T20:19:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559873c4b0c0 0x559873c4d4b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559873bd4940 0x559873bdec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559873bcf7d0 0x559873bddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 --2- 172.21.15.131:0/674400711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559873bcede0 0x559873bdd7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 >> 172.21.15.131:0/674400711 conn(0x559873bca450 msgr2=0x559873bd8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7c98c00640 0 client.25862 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:19:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.823+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 shutdown_connections 2024-04-26T20:19:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cb8c35740 1 -- 172.21.15.131:0/674400711 wait complete. 2024-04-26T20:19:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cb8c35740 20 client.25862 trim_cache size 0 max 16384 2024-04-26T20:19:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:19:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:19:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:19:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:19:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c940a3c30 msgr2=0x7f7c940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c940a3c30 0x7f7c940a60f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x559873bcb3c0 tx=0x7f7c9c009260 comp rx=0 tx=0).stop 2024-04-26T20:20:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb0075330 msgr2=0x7f7cb019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb0075330 0x7f7cb019abc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00308b0 tx=0x7f7ca00050c0 comp rx=0 tx=0).stop 2024-04-26T20:20:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 shutdown_connections 2024-04-26T20:20:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb010a680 0x7f7cb01a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cb0076910 0x7f7cb019b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f7c940a3c30 0x7f7c940a60f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 --2- 172.21.15.131:0/204393986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cb0075330 0x7f7cb019abc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.824+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 >> 172.21.15.131:0/204393986 conn(0x7f7cb00fdd60 msgr2=0x7f7cb0106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.825+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 shutdown_connections 2024-04-26T20:20:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:17:50.825+0000 7f7cafe00640 1 -- 172.21.15.131:0/204393986 wait complete. 2024-04-26T20:20:00.659 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:51 smithi131 ceph-mon[27919]: pgmap v645: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:00.665 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:53 smithi131 ceph-mon[27919]: pgmap v646: 129 pgs: 129 active+clean; 670 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:00.689 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:55 smithi131 ceph-mon[27919]: pgmap v647: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:00.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:57 smithi131 ceph-mon[27919]: pgmap v648: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:00.982 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:17:59 smithi131 ceph-mon[27919]: pgmap v649: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:00.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:01 smithi131 ceph-mon[27919]: pgmap v650: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:01.047 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:03 smithi131 ceph-mon[27919]: pgmap v651: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:01.152 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:45 smithi148 ceph-mon[32281]: pgmap v702: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:01.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:45 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:20:01.256 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.270 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:46 smithi148 ceph-mon[32281]: pgmap v703: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.281 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:46 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.481 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:49 smithi148 ceph-mon[32281]: pgmap v704: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:01.586 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.604 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:49 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:01.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:20:01.731 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:20:01.902 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:50 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:02.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:51 smithi148 ceph-mon[32281]: pgmap v705: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:02.029 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:53 smithi148 ceph-mon[32281]: pgmap v706: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:02.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:55 smithi148 ceph-mon[32281]: pgmap v707: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:02.142 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:57 smithi148 ceph-mon[32281]: pgmap v708: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:02.148 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:19:59 smithi148 ceph-mon[32281]: pgmap v709: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:02.271 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:00 smithi148 ceph-mon[32281]: overall HEALTH_OK 2024-04-26T20:20:02.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:45 smithi157 ceph-mon[32703]: pgmap v702: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:02.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:45 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:20:02.466 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:02.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:02.533 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:46 smithi157 ceph-mon[32703]: pgmap v703: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:02.623 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:02.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:02.808 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:02.901 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:46 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:03.099 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:49 smithi157 ceph-mon[32703]: pgmap v704: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:03.203 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:03.213 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:49 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:03.264 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:20:03.333 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:20:03.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:50 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:03.347 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:51 smithi157 ceph-mon[32703]: pgmap v705: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:03.402 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:53 smithi157 ceph-mon[32703]: pgmap v706: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:03.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:55 smithi157 ceph-mon[32703]: pgmap v707: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:03.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:57 smithi157 ceph-mon[32703]: pgmap v708: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:03.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:19:59 smithi157 ceph-mon[32703]: pgmap v709: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:03.485 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:00 smithi157 ceph-mon[32703]: overall HEALTH_OK 2024-04-26T20:20:03.515 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:01 smithi157 ceph-mon[32703]: pgmap v710: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:03.572 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:01 smithi148 ceph-mon[32281]: pgmap v710: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 -- 172.21.15.131:0/1181536072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 msgr2=0x7f2e34074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 --2- 172.21.15.131:0/1181536072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e34074d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009900 tx=0x7f2e2402a480 comp rx=0 tx=0).stop 2024-04-26T20:20:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 -- 172.21.15.131:0/1181536072 shutdown_connections 2024-04-26T20:20:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 --2- 172.21.15.131:0/1181536072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e3410a6a0 0x7f2e3410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 --2- 172.21.15.131:0/1181536072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34075310 0x7f2e34075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 --2- 172.21.15.131:0/1181536072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e34074d40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 -- 172.21.15.131:0/1181536072 >> 172.21.15.131:0/1181536072 conn(0x7f2e340fdd60 msgr2=0x7f2e341001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 -- 172.21.15.131:0/1181536072 shutdown_connections 2024-04-26T20:20:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.015+0000 7f2e33200640 1 -- 172.21.15.131:0/1181536072 wait complete. 2024-04-26T20:20:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 Processor -- start 2024-04-26T20:20:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 -- start start 2024-04-26T20:20:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34075310 0x7f2e3419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e3419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e3410a6a0 0x7f2e341a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34110550 con 0x7f2e340768f0 2024-04-26T20:20:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341103d0 con 0x7f2e34075310 2024-04-26T20:20:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.017+0000 7f2e33200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341106d0 con 0x7f2e3410a6a0 2024-04-26T20:20:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e31400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e3419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e31400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e3419b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43786/0 (socket says 172.21.15.131:43786) 2024-04-26T20:20:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e31400640 1 -- 172.21.15.131:0/1793110296 learned_addr learned my addr 172.21.15.131:0/1793110296 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e31e00640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34075310 0x7f2e3419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e32800640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e3410a6a0 0x7f2e341a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e32800640 1 -- 172.21.15.131:0/1793110296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34075310 msgr2=0x7f2e3419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e32800640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34075310 0x7f2e3419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e32800640 1 -- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 msgr2=0x7f2e3419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e32800640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e3419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e32800640 1 -- 172.21.15.131:0/1793110296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e341a2800 con 0x7f2e3410a6a0 2024-04-26T20:20:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e31400640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e3419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.018+0000 7f2e31e00640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34075310 0x7f2e3419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:20:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.019+0000 7f2e32800640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e3410a6a0 0x7f2e341a2190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009fd0 tx=0x7f2e2c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.019+0000 7f2e2aa00640 1 -- 172.21.15.131:0/1793110296 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00ce20 con 0x7f2e3410a6a0 2024-04-26T20:20:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.019+0000 7f2e2aa00640 1 -- 172.21.15.131:0/1793110296 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c005020 con 0x7f2e3410a6a0 2024-04-26T20:20:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.019+0000 7f2e2aa00640 1 -- 172.21.15.131:0/1793110296 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c010690 con 0x7f2e3410a6a0 2024-04-26T20:20:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.019+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e341a2af0 con 0x7f2e3410a6a0 2024-04-26T20:20:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.019+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341a2ff0 con 0x7f2e3410a6a0 2024-04-26T20:20:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.024+0000 7f2e3be28740 1 Processor -- start 2024-04-26T20:20:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.024+0000 7f2e2aa00640 1 -- 172.21.15.131:0/1793110296 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2e2c0054d0 con 0x7f2e3410a6a0 2024-04-26T20:20:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e3be28740 1 -- start start 2024-04-26T20:20:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x564591877200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459187c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459187edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e3be28740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5645917db380 con 0x56459187c940 2024-04-26T20:20:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e3be28740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564591777c30 con 0x5645918777d0 2024-04-26T20:20:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e3be28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5645917d7340 con 0x564591876de0 2024-04-26T20:20:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e31400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x564591877200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.025+0000 7f2e31400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x564591877200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52334/0 (socket says 172.21.15.131:52334) 2024-04-26T20:20:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 -- 172.21.15.131:0/327031396 learned_addr learned my addr 172.21.15.131:0/327031396 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31e00640 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459187c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459187edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 -- 172.21.15.131:0/327031396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 msgr2=0x56459187c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459187c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 -- 172.21.15.131:0/327031396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 msgr2=0x56459187edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459187edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 -- 172.21.15.131:0/327031396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564591799940 con 0x564591876de0 2024-04-26T20:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.026+0000 7f2e31400640 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x564591877200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e20011900 tx=0x7f2e20009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f0fade800ac10f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.027+0000 7f2e2aa00640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e180a3c30 0x7f2e180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.027+0000 7f2e1f400640 1 -- 172.21.15.131:0/327031396 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000f8c0 con 0x564591876de0 2024-04-26T20:20:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.027+0000 7f2e1f400640 1 -- 172.21.15.131:0/327031396 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2001e420 con 0x564591876de0 2024-04-26T20:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.027+0000 7f2e2aa00640 1 -- 172.21.15.131:0/1793110296 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2e2c0d1150 con 0x7f2e3410a6a0 2024-04-26T20:20:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.027+0000 7f2e31e00640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e180a3c30 0x7f2e180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.028+0000 7f2e3be28740 1 -- 172.21.15.131:0/327031396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 msgr2=0x564591877200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.028+0000 7f2e3be28740 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x564591877200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e20011900 tx=0x7f2e20009d10 comp rx=0 tx=0).stop 2024-04-26T20:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.028+0000 7f2e3be28740 1 -- 172.21.15.131:0/327031396 shutdown_connections 2024-04-26T20:20:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.028+0000 7f2e3be28740 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459187edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.028+0000 7f2e3be28740 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459187c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.028+0000 7f2e3be28740 1 --2- 172.21.15.131:0/327031396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x564591877200 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.028+0000 7f2e3be28740 1 -- 172.21.15.131:0/327031396 >> 172.21.15.131:0/327031396 conn(0x564591872450 msgr2=0x564591872850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.029+0000 7f2e31e00640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e180a3c30 0x7f2e180a60f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.029+0000 7f2e3be28740 1 -- 172.21.15.131:0/327031396 shutdown_connections 2024-04-26T20:20:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.029+0000 7f2e3be28740 1 -- 172.21.15.131:0/327031396 wait complete. 2024-04-26T20:20:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 -- start start 2024-04-26T20:20:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 Processor -- start 2024-04-26T20:20:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564591876de0 0x564591889920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591889e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x564591890e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56459187f990 con 0x56459187c940 2024-04-26T20:20:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564591762c60 con 0x564591876de0 2024-04-26T20:20:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.030+0000 7f2e3be28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56459187fb10 con 0x5645918777d0 2024-04-26T20:20:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e3be28740 10 client.0 _ll_register_callbacks cb 0x56459178ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:20:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591889e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564591876de0 0x564591889920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591889e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52348/0 (socket says 172.21.15.131:52348) 2024-04-26T20:20:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564591876de0 0x564591889920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:51752/0 (socket says 172.21.15.131:51752) 2024-04-26T20:20:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 -- 172.21.15.131:0/2309952679 learned_addr learned my addr 172.21.15.131:0/2309952679 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e32800640 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x564591890e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 -- 172.21.15.131:0/2309952679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564591876de0 msgr2=0x564591889920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564591876de0 0x564591889920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 -- 172.21.15.131:0/2309952679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 msgr2=0x564591890e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x564591890e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e31400640 1 -- 172.21.15.131:0/2309952679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564591771510 con 0x5645918777d0 2024-04-26T20:20:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.031+0000 7f2e32800640 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x564591890e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:20:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.032+0000 7f2e31400640 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591889e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009c30 tx=0x7f2e20005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=127eb51c2f4fce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.032+0000 7f2e13e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:20:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.032+0000 7f2e13e00640 1 -- 172.21.15.131:0/2309952679 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20005520 con 0x5645918777d0 2024-04-26T20:20:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.032+0000 7f2e13e00640 1 -- 172.21.15.131:0/2309952679 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2e2000f8c0 con 0x5645918777d0 2024-04-26T20:20:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.033+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e2001e420 con 0x5645918777d0 2024-04-26T20:20:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.033+0000 7f2e13e00640 1 -- 172.21.15.131:0/2309952679 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e2000fac0 con 0x5645918777d0 2024-04-26T20:20:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.033+0000 7f2e3be28740 10 client.25864 fetch_fsmap learned FSMap version 27 2024-04-26T20:20:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.033+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645914ed210 con 0x5645918777d0 2024-04-26T20:20:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.033+0000 7f2e13e00640 1 -- 172.21.15.131:0/2309952679 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2001e420 con 0x5645918777d0 2024-04-26T20:20:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.033+0000 7f2e13e00640 1 -- 172.21.15.131:0/2309952679 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2e2001a050 con 0x5645918777d0 2024-04-26T20:20:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.034+0000 7f2e3be28740 10 client.25864 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:20:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.034+0000 7f2e3be28740 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.034+0000 7f2e3be28740 20 client.25864 populate_metadata read hostname 'smithi131' 2024-04-26T20:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.034+0000 7f2e3be28740 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564591895180 0x5645918975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.034+0000 7f2e3be28740 4 client.25864 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:20:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.034+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645917e8f90 con 0x564591895180 2024-04-26T20:20:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.034+0000 7f2e31e00640 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564591895180 0x5645918975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.035+0000 7f2e31e00640 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564591895180 0x5645918975a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe1628f0c964e56 server_cookie=81620f3594231639 in_seq=0 out_seq=0 2024-04-26T20:20:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.035+0000 7f2e13e00640 10 client.25864 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e13e00640 1 -- 172.21.15.131:0/2309952679 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645917e8f90 con 0x564591895180 2024-04-26T20:20:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e13e00640 10 client.25864 handle_command_reply: tid=0 2024-04-26T20:20:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 client.25864 shutdown 2024-04-26T20:20:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 msgr2=0x564591889e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591889e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009c30 tx=0x7f2e20005c50 comp rx=0 tx=0).stop 2024-04-26T20:20:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 shutdown_connections 2024-04-26T20:20:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564591895180 0x5645918975a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x564591890e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591889e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 --2- 172.21.15.131:0/2309952679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564591876de0 0x564591889920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 >> 172.21.15.131:0/2309952679 conn(0x564591872450 msgr2=0x564591871650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e13e00640 0 client.25864 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.036+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 shutdown_connections 2024-04-26T20:20:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.037+0000 7f2e3be28740 1 -- 172.21.15.131:0/2309952679 wait complete. 2024-04-26T20:20:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.037+0000 7f2e3be28740 20 client.25864 trim_cache size 0 max 16384 2024-04-26T20:20:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 Processor -- start 2024-04-26T20:20:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 -- start start 2024-04-26T20:20:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564591876de0 0x564591870dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591890df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56459187c940 0x564591871300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341abab0 con 0x564591876de0 2024-04-26T20:20:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341a9380 con 0x56459187c940 2024-04-26T20:20:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e3be28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341a71d0 con 0x5645918777d0 2024-04-26T20:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e31e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564591876de0 0x564591870dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.067+0000 7f2e31400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591890df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591890df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52362/0 (socket says 172.21.15.131:52362) 2024-04-26T20:20:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 -- 172.21.15.131:0/3682981589 learned_addr learned my addr 172.21.15.131:0/3682981589 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e32800640 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56459187c940 0x564591871300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 -- 172.21.15.131:0/3682981589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56459187c940 msgr2=0x564591871300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56459187c940 0x564591871300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 -- 172.21.15.131:0/3682981589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564591876de0 msgr2=0x564591870dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564591876de0 0x564591870dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 -- 172.21.15.131:0/3682981589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645915b9620 con 0x5645918777d0 2024-04-26T20:20:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.068+0000 7f2e31400640 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591890df0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000ff20 tx=0x7f2e20005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0440f4327a4db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.069+0000 7f2e1ea00640 1 -- 172.21.15.131:0/3682981589 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2003a070 con 0x5645918777d0 2024-04-26T20:20:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.069+0000 7f2e1ea00640 1 -- 172.21.15.131:0/3682981589 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2000fc60 con 0x5645918777d0 2024-04-26T20:20:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.069+0000 7f2e1ea00640 1 -- 172.21.15.131:0/3682981589 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20003b60 con 0x5645918777d0 2024-04-26T20:20:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 -- 172.21.15.131:0/3682981589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 msgr2=0x564591890df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591890df0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000ff20 tx=0x7f2e20005fd0 comp rx=0 tx=0).stop 2024-04-26T20:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 -- 172.21.15.131:0/3682981589 shutdown_connections 2024-04-26T20:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56459187c940 0x564591871300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645918777d0 0x564591890df0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3682981589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564591876de0 0x564591870dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 -- 172.21.15.131:0/3682981589 >> 172.21.15.131:0/3682981589 conn(0x564591872450 msgr2=0x5645918789f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 -- 172.21.15.131:0/3682981589 shutdown_connections 2024-04-26T20:20:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 -- 172.21.15.131:0/3682981589 wait complete. 2024-04-26T20:20:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.070+0000 7f2e3be28740 1 -- start start 2024-04-26T20:20:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 1 Processor -- start 2024-04-26T20:20:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x56459188e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459188e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459188f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341a7840 con 0x56459187c940 2024-04-26T20:20:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341a8cb0 con 0x5645918777d0 2024-04-26T20:20:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341a7f10 con 0x564591876de0 2024-04-26T20:20:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.071+0000 7f2e3be28740 10 client.0 _ll_register_callbacks cb 0x7f2e20011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:20:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x56459188e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e32800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459188f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x56459188e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52370/0 (socket says 172.21.15.131:52370) 2024-04-26T20:20:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 -- 172.21.15.131:0/3594574176 learned_addr learned my addr 172.21.15.131:0/3594574176 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31400640 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459188e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 -- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 msgr2=0x56459188e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459188e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 -- 172.21.15.131:0/3594574176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 msgr2=0x56459188f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459188f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 -- 172.21.15.131:0/3594574176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564591795cb0 con 0x564591876de0 2024-04-26T20:20:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e32800640 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459188f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e31e00640 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x56459188e000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005ec0 tx=0x7f2e24009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18db7cdb8056c519 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.072+0000 7f2e13e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:20:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.073+0000 7f2e13e00640 1 -- 172.21.15.131:0/3594574176 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24035050 con 0x564591876de0 2024-04-26T20:20:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.073+0000 7f2e13e00640 1 -- 172.21.15.131:0/3594574176 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2e24052500 con 0x564591876de0 2024-04-26T20:20:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.073+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e2000fc60 con 0x564591876de0 2024-04-26T20:20:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.073+0000 7f2e13e00640 1 -- 172.21.15.131:0/3594574176 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e24005550 con 0x564591876de0 2024-04-26T20:20:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.073+0000 7f2e3be28740 10 client.25870 fetch_fsmap learned FSMap version 27 2024-04-26T20:20:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.073+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564591890ca0 con 0x564591876de0 2024-04-26T20:20:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e13e00640 1 -- 172.21.15.131:0/3594574176 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2000fc60 con 0x564591876de0 2024-04-26T20:20:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e13e00640 1 -- 172.21.15.131:0/3594574176 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2e24003680 con 0x564591876de0 2024-04-26T20:20:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e3be28740 10 client.25870 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:20:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e3be28740 10 client.25870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:20:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e3be28740 20 client.25870 populate_metadata read hostname 'smithi131' 2024-04-26T20:20:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5645918fba20 0x5645918fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e3be28740 4 client.25870 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:20:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.074+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564591889710 con 0x5645918fba20 2024-04-26T20:20:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.075+0000 7f2e31400640 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5645918fba20 0x5645918fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.076+0000 7f2e31400640 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5645918fba20 0x5645918fde10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2f1f63eeb20b13 server_cookie=b1db67a02f7597e0 in_seq=0 out_seq=0 2024-04-26T20:20:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.076+0000 7f2e13e00640 10 client.25870 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.079+0000 7f2e13e00640 1 -- 172.21.15.131:0/3594574176 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564591889710 con 0x5645918fba20 2024-04-26T20:20:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.079+0000 7f2e13e00640 10 client.25870 handle_command_reply: tid=0 2024-04-26T20:20:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 client.25870 shutdown 2024-04-26T20:20:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 msgr2=0x56459188e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x56459188e000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005ec0 tx=0x7f2e24009260 comp rx=0 tx=0).stop 2024-04-26T20:20:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 shutdown_connections 2024-04-26T20:20:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5645918fba20 0x5645918fde10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56459187c940 0x56459188f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645918777d0 0x56459188e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 --2- 172.21.15.131:0/3594574176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564591876de0 0x56459188e000 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 >> 172.21.15.131:0/3594574176 conn(0x564591872450 msgr2=0x5645918751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e13e00640 0 client.25870 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 shutdown_connections 2024-04-26T20:20:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 1 -- 172.21.15.131:0/3594574176 wait complete. 2024-04-26T20:20:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.080+0000 7f2e3be28740 20 client.25870 trim_cache size 0 max 16384 2024-04-26T20:20:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:20:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:20:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:20:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e180a3c30 msgr2=0x7f2e180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e180a3c30 0x7f2e180a60f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24005a20 comp rx=0 tx=0).stop 2024-04-26T20:20:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e3410a6a0 msgr2=0x7f2e341a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e3410a6a0 0x7f2e341a2190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009fd0 tx=0x7f2e2c00ef10 comp rx=0 tx=0).stop 2024-04-26T20:20:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 shutdown_connections 2024-04-26T20:20:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f2e180a3c30 0x7f2e180a60f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e3410a6a0 0x7f2e341a2190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e340768f0 0x7f2e3419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 --2- 172.21.15.131:0/1793110296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34075310 0x7f2e3419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 >> 172.21.15.131:0/1793110296 conn(0x7f2e340fdd60 msgr2=0x7f2e34106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 shutdown_connections 2024-04-26T20:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:04.081+0000 7f2e33200640 1 -- 172.21.15.131:0/1793110296 wait complete. 2024-04-26T20:20:20.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:05 smithi131 ceph-mon[27919]: pgmap v652: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:20.678 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:07 smithi131 ceph-mon[27919]: pgmap v653: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:20.873 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:09 smithi131 ceph-mon[27919]: pgmap v654: 129 pgs: 129 active+clean; 671 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:20:20.883 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:11 smithi131 ceph-mon[27919]: pgmap v655: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:20.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:13 smithi131 ceph-mon[27919]: pgmap v656: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:21.071 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:15 smithi131 ceph-mon[27919]: pgmap v657: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:21.201 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:03 smithi148 ceph-mon[32281]: pgmap v711: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:21.269 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:05 smithi148 ceph-mon[32281]: pgmap v712: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:20:21.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:07 smithi148 ceph-mon[32281]: pgmap v713: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:21.282 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:09 smithi148 ceph-mon[32281]: pgmap v714: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:21.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:11 smithi148 ceph-mon[32281]: pgmap v715: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:21.493 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:13 smithi148 ceph-mon[32281]: pgmap v716: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:21.510 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:15 smithi148 ceph-mon[32281]: pgmap v717: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:21.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:16 smithi148 ceph-mon[32281]: pgmap v718: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:21.523 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:19 smithi148 ceph-mon[32281]: pgmap v719: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-26T20:20:21.531 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:03 smithi157 ceph-mon[32703]: pgmap v711: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:21.610 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:05 smithi157 ceph-mon[32703]: pgmap v712: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:20:21.657 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:07 smithi157 ceph-mon[32703]: pgmap v713: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:21.767 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:09 smithi157 ceph-mon[32703]: pgmap v714: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:22.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:11 smithi157 ceph-mon[32703]: pgmap v715: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:22.112 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:13 smithi157 ceph-mon[32703]: pgmap v716: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:22.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:15 smithi157 ceph-mon[32703]: pgmap v717: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:22.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:16 smithi157 ceph-mon[32703]: pgmap v718: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:22.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:19 smithi157 ceph-mon[32703]: pgmap v719: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-26T20:20:22.260 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:21 smithi148 ceph-mon[32281]: pgmap v720: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:22.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:21 smithi157 ceph-mon[32703]: pgmap v720: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 -- 172.21.15.131:0/3248132984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49010ed90 msgr2=0x7fa490111180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 --2- 172.21.15.131:0/3248132984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49010ed90 0x7fa490111180 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa488009cd0 tx=0x7fa48802a8e0 comp rx=0 tx=0).stop 2024-04-26T20:20:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 -- 172.21.15.131:0/3248132984 shutdown_connections 2024-04-26T20:20:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 --2- 172.21.15.131:0/3248132984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49010ed90 0x7fa490111180 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 --2- 172.21.15.131:0/3248132984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa490075390 0x7fa490075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 --2- 172.21.15.131:0/3248132984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa490076970 0x7fa490074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 -- 172.21.15.131:0/3248132984 >> 172.21.15.131:0/3248132984 conn(0x7fa4900fdd30 msgr2=0x7fa490100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 -- 172.21.15.131:0/3248132984 shutdown_connections 2024-04-26T20:20:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.215+0000 7fa48fe00640 1 -- 172.21.15.131:0/3248132984 wait complete. 2024-04-26T20:20:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 Processor -- start 2024-04-26T20:20:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 -- start start 2024-04-26T20:20:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490075390 0x7fa49019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa490076970 0x7fa49019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49010ed90 0x7fa4901a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa490114910 con 0x7fa490075390 2024-04-26T20:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa490114790 con 0x7fa490076970 2024-04-26T20:20:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.216+0000 7fa48fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa490114a90 con 0x7fa49010ed90 2024-04-26T20:20:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490075390 0x7fa49019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490075390 0x7fa49019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59338/0 (socket says 172.21.15.131:59338) 2024-04-26T20:20:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48ea00640 1 -- 172.21.15.131:0/2950648489 learned_addr learned my addr 172.21.15.131:0/2950648489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48e000640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa490076970 0x7fa49019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48f400640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49010ed90 0x7fa4901a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48e000640 1 -- 172.21.15.131:0/2950648489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49010ed90 msgr2=0x7fa4901a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48e000640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49010ed90 0x7fa4901a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48e000640 1 -- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490075390 msgr2=0x7fa49019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48e000640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490075390 0x7fa49019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48e000640 1 -- 172.21.15.131:0/2950648489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4901a6b80 con 0x7fa490076970 2024-04-26T20:20:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48ea00640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490075390 0x7fa49019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.217+0000 7fa48e000640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa490076970 0x7fa49019f4a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00e9f0 tx=0x7fa47c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.218+0000 7fa487400640 1 -- 172.21.15.131:0/2950648489 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00ccc0 con 0x7fa490076970 2024-04-26T20:20:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.218+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4901a6e70 con 0x7fa490076970 2024-04-26T20:20:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.218+0000 7fa487400640 1 -- 172.21.15.131:0/2950648489 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c0108e0 con 0x7fa490076970 2024-04-26T20:20:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.218+0000 7fa487400640 1 -- 172.21.15.131:0/2950648489 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c018880 con 0x7fa490076970 2024-04-26T20:20:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.218+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4901a7380 con 0x7fa490076970 2024-04-26T20:20:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 Processor -- start 2024-04-26T20:20:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 -- start start 2024-04-26T20:20:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b44de0 0x55c547b45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b457d0 0x55c547b4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b4cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c547aa9380 con 0x55c547b457d0 2024-04-26T20:20:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c547a45c30 con 0x55c547b44de0 2024-04-26T20:20:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa498a7a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c547aa5340 con 0x55c547b4a940 2024-04-26T20:20:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.221+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b457d0 0x55c547b4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b457d0 0x55c547b4a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59346/0 (socket says 172.21.15.131:59346) 2024-04-26T20:20:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 -- 172.21.15.131:0/3875510047 learned_addr learned my addr 172.21.15.131:0/3875510047 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48f400640 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b44de0 0x55c547b45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa487400640 1 -- 172.21.15.131:0/2950648489 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa47c010430 con 0x7fa490076970 2024-04-26T20:20:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 -- 172.21.15.131:0/3875510047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 msgr2=0x55c547b4cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b4cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 -- 172.21.15.131:0/3875510047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b44de0 msgr2=0x55c547b45200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b44de0 0x55c547b45200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 -- 172.21.15.131:0/3875510047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c547a67940 con 0x55c547b457d0 2024-04-26T20:20:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48f400640 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b44de0 0x55c547b45200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.222+0000 7fa48ea00640 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b457d0 0x55c547b4a2d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa480011650 tx=0x7fa480009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f885977e3e8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.223+0000 7fa47be00640 1 -- 172.21.15.131:0/3875510047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48001e070 con 0x55c547b457d0 2024-04-26T20:20:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.223+0000 7fa47be00640 1 -- 172.21.15.131:0/3875510047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48000fb70 con 0x55c547b457d0 2024-04-26T20:20:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.223+0000 7fa47be00640 1 -- 172.21.15.131:0/3875510047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480019c00 con 0x55c547b457d0 2024-04-26T20:20:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa487400640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa4740a3c30 0x7fa4740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa487400640 1 -- 172.21.15.131:0/2950648489 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa47c0d0c50 con 0x7fa490076970 2024-04-26T20:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa48f400640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa4740a3c30 0x7fa4740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 -- 172.21.15.131:0/3875510047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b457d0 msgr2=0x55c547b4a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b457d0 0x55c547b4a2d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa480011650 tx=0x7fa480009f90 comp rx=0 tx=0).stop 2024-04-26T20:20:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 -- 172.21.15.131:0/3875510047 shutdown_connections 2024-04-26T20:20:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b4cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b457d0 0x55c547b4a2d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3875510047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b44de0 0x55c547b45200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 -- 172.21.15.131:0/3875510047 >> 172.21.15.131:0/3875510047 conn(0x55c547b40450 msgr2=0x55c547b40850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 -- 172.21.15.131:0/3875510047 shutdown_connections 2024-04-26T20:20:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.224+0000 7fa498a7a740 1 -- 172.21.15.131:0/3875510047 wait complete. 2024-04-26T20:20:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa48f400640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa4740a3c30 0x7fa4740a60f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa488006010 tx=0x7fa488031850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 -- start start 2024-04-26T20:20:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 Processor -- start 2024-04-26T20:20:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b4f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b4f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c547b4d8f0 con 0x55c547b4a940 2024-04-26T20:20:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c547a30c60 con 0x55c547b457d0 2024-04-26T20:20:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c547b4da70 con 0x55c547b44de0 2024-04-26T20:20:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.225+0000 7fa498a7a740 10 client.0 _ll_register_callbacks cb 0x55c547a5ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:20:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b4f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59348/0 (socket says 172.21.15.131:59348) 2024-04-26T20:20:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48f400640 1 -- 172.21.15.131:0/3053431467 learned_addr learned my addr 172.21.15.131:0/3053431467 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48ea00640 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b4f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48e000640 1 -- 172.21.15.131:0/3053431467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 msgr2=0x55c547b4f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48e000640 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b4f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48e000640 1 -- 172.21.15.131:0/3053431467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 msgr2=0x55c547b565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48e000640 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48e000640 1 -- 172.21.15.131:0/3053431467 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c547a3f510 con 0x55c547b457d0 2024-04-26T20:20:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48f400640 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa48e000640 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b4f570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0ce210 tx=0x7fa47c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc7523790b2346d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa478c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:20:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.226+0000 7fa478c00640 1 -- 172.21.15.131:0/3053431467 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0a5650 con 0x55c547b457d0 2024-04-26T20:20:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa478c00640 1 -- 172.21.15.131:0/3053431467 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa47c010a90 con 0x55c547b457d0 2024-04-26T20:20:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48000fb70 con 0x55c547b457d0 2024-04-26T20:20:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa478c00640 1 -- 172.21.15.131:0/3053431467 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa47c0bbba0 con 0x55c547b457d0 2024-04-26T20:20:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 10 client.25892 fetch_fsmap learned FSMap version 27 2024-04-26T20:20:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5477bb210 con 0x55c547b457d0 2024-04-26T20:20:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa478c00640 1 -- 172.21.15.131:0/3053431467 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48000fb70 con 0x55c547b457d0 2024-04-26T20:20:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa478c00640 1 -- 172.21.15.131:0/3053431467 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa47c09b950 con 0x55c547b457d0 2024-04-26T20:20:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 10 client.25892 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:20:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 10 client.25892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:20:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 20 client.25892 populate_metadata read hostname 'smithi131' 2024-04-26T20:20:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547b5ab10 0x55c547b5cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 4 client.25892 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:20:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.227+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c547ab6f90 con 0x55c547b5ab10 2024-04-26T20:20:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.228+0000 7fa48ea00640 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547b5ab10 0x55c547b5cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.228+0000 7fa478c00640 10 client.25892 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.228+0000 7fa48ea00640 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547b5ab10 0x55c547b5cf00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de58afe4a7a87bc2 server_cookie=893ccf9649d01bac in_seq=0 out_seq=0 2024-04-26T20:20:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.229+0000 7fa478c00640 1 -- 172.21.15.131:0/3053431467 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c547ab6f90 con 0x55c547b5ab10 2024-04-26T20:20:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.229+0000 7fa478c00640 10 client.25892 handle_command_reply: tid=0 2024-04-26T20:20:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.229+0000 7fa498a7a740 1 client.25892 shutdown 2024-04-26T20:20:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 msgr2=0x55c547b4f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b4f570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0ce210 tx=0x7fa47c010860 comp rx=0 tx=0).stop 2024-04-26T20:20:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 shutdown_connections 2024-04-26T20:20:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547b5ab10 0x55c547b5cf00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa478c00640 0 client.25892 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b4f570 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 --2- 172.21.15.131:0/3053431467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b4f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 >> 172.21.15.131:0/3053431467 conn(0x55c547b40450 msgr2=0x55c547b3f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 shutdown_connections 2024-04-26T20:20:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 1 -- 172.21.15.131:0/3053431467 wait complete. 2024-04-26T20:20:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.230+0000 7fa498a7a740 20 client.25892 trim_cache size 0 max 16384 2024-04-26T20:20:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 Processor -- start 2024-04-26T20:20:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 -- start start 2024-04-26T20:20:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b43b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b56570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b62780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901b4410 con 0x55c547b4a940 2024-04-26T20:20:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901b1ce0 con 0x55c547b457d0 2024-04-26T20:20:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.246+0000 7fa498a7a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901afb30 con 0x55c547b44de0 2024-04-26T20:20:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b62780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b62780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59358/0 (socket says 172.21.15.131:59358) 2024-04-26T20:20:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 -- 172.21.15.131:0/190928778 learned_addr learned my addr 172.21.15.131:0/190928778 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48ea00640 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b43b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48e000640 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b56570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 -- 172.21.15.131:0/190928778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 msgr2=0x55c547b43b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b43b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 -- 172.21.15.131:0/190928778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 msgr2=0x55c547b56570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b56570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 -- 172.21.15.131:0/190928778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c547b5d270 con 0x55c547b4a940 2024-04-26T20:20:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48e000640 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b56570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.247+0000 7fa48f400640 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b62780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa4880315c0 tx=0x7fa488044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87a8a53e838b8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa47b400640 1 -- 172.21.15.131:0/190928778 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488004040 con 0x55c547b4a940 2024-04-26T20:20:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa47b400640 1 -- 172.21.15.131:0/190928778 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa488036040 con 0x55c547b4a940 2024-04-26T20:20:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa47b400640 1 -- 172.21.15.131:0/190928778 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488030b90 con 0x55c547b4a940 2024-04-26T20:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa498a7a740 1 -- 172.21.15.131:0/190928778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 msgr2=0x55c547b62780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa498a7a740 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b62780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa4880315c0 tx=0x7fa488044000 comp rx=0 tx=0).stop 2024-04-26T20:20:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa498a7a740 1 -- 172.21.15.131:0/190928778 shutdown_connections 2024-04-26T20:20:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa498a7a740 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b4a940 0x55c547b62780 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa498a7a740 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b56570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa498a7a740 1 --2- 172.21.15.131:0/190928778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b44de0 0x55c547b43b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.248+0000 7fa498a7a740 1 -- 172.21.15.131:0/190928778 >> 172.21.15.131:0/190928778 conn(0x55c547b40450 msgr2=0x55c547b43400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 -- 172.21.15.131:0/190928778 shutdown_connections 2024-04-26T20:20:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 -- 172.21.15.131:0/190928778 wait complete. 2024-04-26T20:20:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 -- start start 2024-04-26T20:20:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 Processor -- start 2024-04-26T20:20:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b44de0 0x55c547b53c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b55170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901b01a0 con 0x55c547b44de0 2024-04-26T20:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901b1610 con 0x55c547b457d0 2024-04-26T20:20:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.249+0000 7fa498a7a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901b0870 con 0x55c547b4a940 2024-04-26T20:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa498a7a740 10 client.0 _ll_register_callbacks cb 0x7fa47c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b44de0 0x55c547b53c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b44de0 0x55c547b53c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59366/0 (socket says 172.21.15.131:59366) 2024-04-26T20:20:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48ea00640 1 -- 172.21.15.131:0/2659700296 learned_addr learned my addr 172.21.15.131:0/2659700296 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48e000640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48f400640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b55170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48f400640 1 -- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 msgr2=0x55c547b541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48f400640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48f400640 1 -- 172.21.15.131:0/2659700296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b44de0 msgr2=0x55c547b53c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48f400640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b44de0 0x55c547b53c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48f400640 1 -- 172.21.15.131:0/2659700296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c547887620 con 0x55c547b4a940 2024-04-26T20:20:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48ea00640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b44de0 0x55c547b53c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.250+0000 7fa48e000640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa48f400640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b55170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa488031780 tx=0x7fa488030f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c26d45a21e52732 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa478c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:20:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa478c00640 1 -- 172.21.15.131:0/2659700296 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48805f020 con 0x55c547b4a940 2024-04-26T20:20:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa488036040 con 0x55c547b4a940 2024-04-26T20:20:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa478c00640 1 -- 172.21.15.131:0/2659700296 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa48802e040 con 0x55c547b4a940 2024-04-26T20:20:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa478c00640 1 -- 172.21.15.131:0/2659700296 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa488031d00 con 0x55c547b4a940 2024-04-26T20:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa498a7a740 10 client.25885 fetch_fsmap learned FSMap version 27 2024-04-26T20:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c547a63cb0 con 0x55c547b4a940 2024-04-26T20:20:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.251+0000 7fa478c00640 1 -- 172.21.15.131:0/2659700296 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa488036040 con 0x55c547b4a940 2024-04-26T20:20:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa478c00640 1 -- 172.21.15.131:0/2659700296 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa488030070 con 0x55c547b4a940 2024-04-26T20:20:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa498a7a740 10 client.25885 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:20:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa498a7a740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa498a7a740 20 client.25885 populate_metadata read hostname 'smithi131' 2024-04-26T20:20:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa498a7a740 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547bc1640 0x55c547bc3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa498a7a740 4 client.25885 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:20:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c547ab0e00 con 0x55c547bc1640 2024-04-26T20:20:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.252+0000 7fa48ea00640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547bc1640 0x55c547bc3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.253+0000 7fa478c00640 10 client.25885 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.253+0000 7fa48ea00640 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547bc1640 0x55c547bc3a60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7c9696891339e2 server_cookie=ce74250e1c8c444e in_seq=0 out_seq=0 2024-04-26T20:20:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.256+0000 7fa478c00640 1 -- 172.21.15.131:0/2659700296 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c547ab0e00 con 0x55c547bc1640 2024-04-26T20:20:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.256+0000 7fa478c00640 10 client.25885 handle_command_reply: tid=0 2024-04-26T20:20:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.256+0000 7fa498a7a740 1 client.25885 shutdown 2024-04-26T20:20:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 msgr2=0x55c547b55170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b55170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa488031780 tx=0x7fa488030f00 comp rx=0 tx=0).stop 2024-04-26T20:20:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 shutdown_connections 2024-04-26T20:20:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c547bc1640 0x55c547bc3a60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c547b4a940 0x55c547b55170 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa478c00640 0 client.25885 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c547b457d0 0x55c547b541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 --2- 172.21.15.131:0/2659700296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c547b44de0 0x55c547b53c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 >> 172.21.15.131:0/2659700296 conn(0x55c547b40450 msgr2=0x55c547b3edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 shutdown_connections 2024-04-26T20:20:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 1 -- 172.21.15.131:0/2659700296 wait complete. 2024-04-26T20:20:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.257+0000 7fa498a7a740 20 client.25885 trim_cache size 0 max 16384 2024-04-26T20:20:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:20:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:20:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:20:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:20:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa4740a3c30 msgr2=0x7fa4740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa4740a3c30 0x7fa4740a60f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa488006010 tx=0x7fa488031850 comp rx=0 tx=0).stop 2024-04-26T20:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa490076970 msgr2=0x7fa49019f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa490076970 0x7fa49019f4a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00e9f0 tx=0x7fa47c00eeb0 comp rx=0 tx=0).stop 2024-04-26T20:20:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 shutdown_connections 2024-04-26T20:20:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa4740a3c30 0x7fa4740a60f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49010ed90 0x7fa4901a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa490076970 0x7fa49019f4a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 --2- 172.21.15.131:0/2950648489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490075390 0x7fa49019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 >> 172.21.15.131:0/2950648489 conn(0x7fa4900fdd30 msgr2=0x7fa490102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 shutdown_connections 2024-04-26T20:20:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:17.258+0000 7fa48fe00640 1 -- 172.21.15.131:0/2950648489 wait complete. 2024-04-26T20:20:37.757 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:17 smithi131 ceph-mon[27919]: pgmap v658: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:37.849 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:23 smithi157 ceph-mon[32703]: pgmap v721: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:37.856 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:25 smithi157 ceph-mon[32703]: pgmap v722: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:37.866 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:27 smithi157 ceph-mon[32703]: pgmap v723: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:37.872 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:29 smithi157 ceph-mon[32703]: pgmap v724: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:37.972 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:31 smithi157 ceph-mon[32703]: pgmap v725: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:38.094 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:33 smithi157 ceph-mon[32703]: pgmap v726: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:38.105 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:35 smithi157 ceph-mon[32703]: pgmap v727: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:38.174 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:37 smithi157 ceph-mon[32703]: pgmap v728: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:38.223 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:23 smithi148 ceph-mon[32281]: pgmap v721: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:38.229 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:25 smithi148 ceph-mon[32281]: pgmap v722: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:38.235 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:27 smithi148 ceph-mon[32281]: pgmap v723: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:38.242 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:29 smithi148 ceph-mon[32281]: pgmap v724: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:38.356 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:31 smithi148 ceph-mon[32281]: pgmap v725: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:38.450 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:33 smithi148 ceph-mon[32281]: pgmap v726: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:38.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:35 smithi148 ceph-mon[32281]: pgmap v727: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:38.585 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:37 smithi148 ceph-mon[32281]: pgmap v728: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:Updating files: 61% (7547/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T20:20:38.747 INFO:tasks.workunit.client.0.smithi131.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T20:20:38.810 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-04-26T20:20:38.819 INFO:tasks.workunit.client.0.smithi131.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T20:20:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T20:20:39.082 INFO:tasks.workunit.client.0.smithi131.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T20:20:39.226 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-04-26T20:20:39.345 INFO:tasks.workunit.client.0.smithi131.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T20:20:39.437 INFO:tasks.workunit.client.0.smithi131.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T20:20:39.530 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-04-26T20:20:39.537 INFO:tasks.workunit.client.0.smithi131.stderr: git switch -c 2024-04-26T20:20:39.544 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-04-26T20:20:39.625 INFO:tasks.workunit.client.0.smithi131.stderr:Or undo this operation with: 2024-04-26T20:20:39.665 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-04-26T20:20:39.810 INFO:tasks.workunit.client.0.smithi131.stderr: git switch - 2024-04-26T20:20:39.922 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-04-26T20:20:39.929 INFO:tasks.workunit.client.0.smithi131.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T20:20:39.935 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-04-26T20:20:40.015 INFO:tasks.workunit.client.0.smithi131.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T20:20:40.038 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T20:20:40.139 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:19 smithi131 ceph-mon[27919]: pgmap v659: 129 pgs: 129 active+clean; 672 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:40.292 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:21 smithi131 ceph-mon[27919]: pgmap v660: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:20:40.400 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:23 smithi131 ceph-mon[27919]: pgmap v661: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:40.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:25 smithi131 ceph-mon[27919]: pgmap v662: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:40.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:27 smithi131 ceph-mon[27919]: pgmap v663: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:40.513 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:29 smithi131 ceph-mon[27919]: pgmap v664: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:40.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:39 smithi157 ceph-mon[32703]: pgmap v729: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:40.532 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:39 smithi148 ceph-mon[32281]: pgmap v729: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.398+0000 7f9b33000640 1 -- 172.21.15.131:0/444920484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c108de0 msgr2=0x7f9b2c109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.398+0000 7f9b33000640 1 --2- 172.21.15.131:0/444920484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c108de0 0x7f9b2c109200 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009900 tx=0x7f9b2002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:20:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.399+0000 7f9b33000640 1 -- 172.21.15.131:0/444920484 shutdown_connections 2024-04-26T20:20:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.399+0000 7f9b33000640 1 --2- 172.21.15.131:0/444920484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2c10a990 0x7f9b2c111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.399+0000 7f9b33000640 1 --2- 172.21.15.131:0/444920484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b2c109fd0 0x7f9b2c10a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.399+0000 7f9b33000640 1 --2- 172.21.15.131:0/444920484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c108de0 0x7f9b2c109200 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.399+0000 7f9b33000640 1 -- 172.21.15.131:0/444920484 >> 172.21.15.131:0/444920484 conn(0x7f9b2c077dc0 msgr2=0x7f9b2c07a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.399+0000 7f9b33000640 1 -- 172.21.15.131:0/444920484 shutdown_connections 2024-04-26T20:20:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.399+0000 7f9b33000640 1 -- 172.21.15.131:0/444920484 wait complete. 2024-04-26T20:20:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 Processor -- start 2024-04-26T20:20:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 -- start start 2024-04-26T20:20:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b2c108de0 0x7f9b2c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c109fd0 0x7f9b2c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2c10a990 0x7f9b2c1a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114960 con 0x7f9b2c108de0 2024-04-26T20:20:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1147e0 con 0x7f9b2c109fd0 2024-04-26T20:20:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.400+0000 7f9b33000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114ae0 con 0x7f9b2c10a990 2024-04-26T20:20:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b2c108de0 0x7f9b2c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c109fd0 0x7f9b2c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b32600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2c10a990 0x7f9b2c1a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c109fd0 0x7f9b2c19f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50556/0 (socket says 172.21.15.131:50556) 2024-04-26T20:20:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 -- 172.21.15.131:0/4098225174 learned_addr learned my addr 172.21.15.131:0/4098225174 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 -- 172.21.15.131:0/4098225174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2c10a990 msgr2=0x7f9b2c1a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2c10a990 0x7f9b2c1a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 -- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b2c108de0 msgr2=0x7f9b2c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b2c108de0 0x7f9b2c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 -- 172.21.15.131:0/4098225174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c1a6c00 con 0x7f9b2c109fd0 2024-04-26T20:20:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.401+0000 7f9b31200640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c109fd0 0x7f9b2c19f520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800ea60 tx=0x7f9b1800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.402+0000 7f9b2a000640 1 -- 172.21.15.131:0/4098225174 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800cde0 con 0x7f9b2c109fd0 2024-04-26T20:20:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.402+0000 7f9b2a000640 1 -- 172.21.15.131:0/4098225174 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b180035f0 con 0x7f9b2c109fd0 2024-04-26T20:20:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.402+0000 7f9b2a000640 1 -- 172.21.15.131:0/4098225174 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18010690 con 0x7f9b2c109fd0 2024-04-26T20:20:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.402+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c1a6ef0 con 0x7f9b2c109fd0 2024-04-26T20:20:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.402+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c1a73b0 con 0x7f9b2c109fd0 2024-04-26T20:20:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.405+0000 7f9b3736c740 1 Processor -- start 2024-04-26T20:20:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.405+0000 7f9b3736c740 1 -- start start 2024-04-26T20:20:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.405+0000 7f9b2a000640 1 -- 172.21.15.131:0/4098225174 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9b180037a0 con 0x7f9b2c109fd0 2024-04-26T20:20:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.405+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.405+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f17d0 0x55e0df3f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.405+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f6940 0x55e0df3f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.405+0000 7f9b3736c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e0df355380 con 0x55e0df3f17d0 2024-04-26T20:20:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b3736c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e0df2f1c30 con 0x55e0df3f0de0 2024-04-26T20:20:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b3736c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e0df351340 con 0x55e0df3f6940 2024-04-26T20:20:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f17d0 0x55e0df3f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f17d0 0x55e0df3f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57510/0 (socket says 172.21.15.131:57510) 2024-04-26T20:20:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b32600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 -- 172.21.15.131:0/2201625402 learned_addr learned my addr 172.21.15.131:0/2201625402 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31200640 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f6940 0x55e0df3f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 -- 172.21.15.131:0/2201625402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f6940 msgr2=0x55e0df3f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f6940 0x55e0df3f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 -- 172.21.15.131:0/2201625402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 msgr2=0x55e0df3f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b31c00640 1 -- 172.21.15.131:0/2201625402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0df313940 con 0x55e0df3f17d0 2024-04-26T20:20:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.406+0000 7f9b32600640 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.407+0000 7f9b31c00640 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f17d0 0x55e0df3f62d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006010 tx=0x7f9b20004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88973dc360553caa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.407+0000 7f9b2a000640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b140a3c30 0x7f9b140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.407+0000 7f9b1ea00640 1 -- 172.21.15.131:0/2201625402 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20035030 con 0x55e0df3f17d0 2024-04-26T20:20:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.407+0000 7f9b2a000640 1 -- 172.21.15.131:0/4098225174 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9b180d0c80 con 0x7f9b2c109fd0 2024-04-26T20:20:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.407+0000 7f9b1ea00640 1 -- 172.21.15.131:0/2201625402 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b20040070 con 0x55e0df3f17d0 2024-04-26T20:20:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.407+0000 7f9b32600640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b140a3c30 0x7f9b140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.407+0000 7f9b1ea00640 1 -- 172.21.15.131:0/2201625402 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20031890 con 0x55e0df3f17d0 2024-04-26T20:20:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 -- 172.21.15.131:0/2201625402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f17d0 msgr2=0x55e0df3f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f17d0 0x55e0df3f62d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006010 tx=0x7f9b20004040 comp rx=0 tx=0).stop 2024-04-26T20:20:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b32600640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b140a3c30 0x7f9b140a60f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55e0df3f2030 tx=0x7f9b24009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 -- 172.21.15.131:0/2201625402 shutdown_connections 2024-04-26T20:20:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f6940 0x55e0df3f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f17d0 0x55e0df3f62d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 --2- 172.21.15.131:0/2201625402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 -- 172.21.15.131:0/2201625402 >> 172.21.15.131:0/2201625402 conn(0x55e0df3ec450 msgr2=0x55e0df3ec850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 -- 172.21.15.131:0/2201625402 shutdown_connections 2024-04-26T20:20:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.408+0000 7f9b3736c740 1 -- 172.21.15.131:0/2201625402 wait complete. 2024-04-26T20:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 -- start start 2024-04-26T20:20:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 Processor -- start 2024-04-26T20:20:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df402500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e0df3f9890 con 0x55e0df3f0de0 2024-04-26T20:20:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e0df2dcc60 con 0x55e0df3f6940 2024-04-26T20:20:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e0df3f9a10 con 0x55e0df3f17d0 2024-04-26T20:20:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.409+0000 7f9b3736c740 10 client.0 _ll_register_callbacks cb 0x55e0df308e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:20:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3faff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57522/0 (socket says 172.21.15.131:57522) 2024-04-26T20:20:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 -- 172.21.15.131:0/4275778468 learned_addr learned my addr 172.21.15.131:0/4275778468 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b32600640 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df402500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 -- 172.21.15.131:0/4275778468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 msgr2=0x55e0df3fb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 -- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 msgr2=0x55e0df402500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df402500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 -- 172.21.15.131:0/4275778468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0df2eb510 con 0x55e0df3f0de0 2024-04-26T20:20:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b31c00640 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3faff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9b2002a910 tx=0x7f9b20004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5561f82bd089030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b13400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:20:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.410+0000 7f9b13400640 1 -- 172.21.15.131:0/4275778468 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20035070 con 0x55e0df3f0de0 2024-04-26T20:20:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b13400640 1 -- 172.21.15.131:0/4275778468 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9b20003a00 con 0x55e0df3f0de0 2024-04-26T20:20:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b20040070 con 0x55e0df3f0de0 2024-04-26T20:20:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b13400640 1 -- 172.21.15.131:0/4275778468 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b20003c00 con 0x55e0df3f0de0 2024-04-26T20:20:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b3736c740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-04-26T20:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0df067210 con 0x55e0df3f0de0 2024-04-26T20:20:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b13400640 1 -- 172.21.15.131:0/4275778468 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b20040070 con 0x55e0df3f0de0 2024-04-26T20:20:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b13400640 1 -- 172.21.15.131:0/4275778468 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9b2004d030 con 0x55e0df3f0de0 2024-04-26T20:20:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b3736c740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:20:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b3736c740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:20:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b3736c740 20 client.15789 populate_metadata read hostname 'smithi131' 2024-04-26T20:20:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.411+0000 7f9b3736c740 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df406660 0x55e0df408a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.412+0000 7f9b3736c740 4 client.15789 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:20:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.412+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0df362f90 con 0x55e0df406660 2024-04-26T20:20:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.412+0000 7f9b31200640 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df406660 0x55e0df408a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.412+0000 7f9b31200640 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df406660 0x55e0df408a50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669f221c9876233f server_cookie=53a53ab0bacce690 in_seq=0 out_seq=0 2024-04-26T20:20:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.412+0000 7f9b13400640 10 client.15789 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b13400640 1 -- 172.21.15.131:0/4275778468 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0df362f90 con 0x55e0df406660 2024-04-26T20:20:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b13400640 10 client.15789 handle_command_reply: tid=0 2024-04-26T20:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 client.15789 shutdown 2024-04-26T20:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 msgr2=0x55e0df3faff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3faff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9b2002a910 tx=0x7f9b20004440 comp rx=0 tx=0).stop 2024-04-26T20:20:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 shutdown_connections 2024-04-26T20:20:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df406660 0x55e0df408a50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df402500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 --2- 172.21.15.131:0/4275778468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3faff0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b13400640 0 client.15789 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.413+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 >> 172.21.15.131:0/4275778468 conn(0x55e0df3ec450 msgr2=0x55e0df3eb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.414+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 shutdown_connections 2024-04-26T20:20:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.414+0000 7f9b3736c740 1 -- 172.21.15.131:0/4275778468 wait complete. 2024-04-26T20:20:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.414+0000 7f9b3736c740 20 client.15789 trim_cache size 0 max 16384 2024-04-26T20:20:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 Processor -- start 2024-04-26T20:20:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 -- start start 2024-04-26T20:20:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df4024c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df3eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b4340 con 0x55e0df3f0de0 2024-04-26T20:20:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b1c10 con 0x55e0df3f6940 2024-04-26T20:20:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.430+0000 7f9b3736c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1afa60 con 0x55e0df3f17d0 2024-04-26T20:20:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3efa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57526/0 (socket says 172.21.15.131:57526) 2024-04-26T20:20:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 -- 172.21.15.131:0/1938926470 learned_addr learned my addr 172.21.15.131:0/1938926470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b32600640 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df3eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31200640 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df4024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 -- 172.21.15.131:0/1938926470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 msgr2=0x55e0df4024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df4024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 -- 172.21.15.131:0/1938926470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 msgr2=0x55e0df3eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df3eadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 -- 172.21.15.131:0/1938926470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0df133620 con 0x55e0df3f0de0 2024-04-26T20:20:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b32600640 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df3eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.431+0000 7f9b31c00640 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3efa20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b20000c00 tx=0x7f9b20009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08b5d3038fd80a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b1e000640 1 -- 172.21.15.131:0/1938926470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20035040 con 0x55e0df3f0de0 2024-04-26T20:20:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b1e000640 1 -- 172.21.15.131:0/1938926470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2005c020 con 0x55e0df3f0de0 2024-04-26T20:20:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b1e000640 1 -- 172.21.15.131:0/1938926470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20003a40 con 0x55e0df3f0de0 2024-04-26T20:20:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b3736c740 1 -- 172.21.15.131:0/1938926470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 msgr2=0x55e0df3efa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3efa20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b20000c00 tx=0x7f9b20009660 comp rx=0 tx=0).stop 2024-04-26T20:20:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b3736c740 1 -- 172.21.15.131:0/1938926470 shutdown_connections 2024-04-26T20:20:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f6940 0x55e0df3eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df4024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1938926470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f0de0 0x55e0df3efa20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.432+0000 7f9b3736c740 1 -- 172.21.15.131:0/1938926470 >> 172.21.15.131:0/1938926470 conn(0x55e0df3ec450 msgr2=0x55e0df3ef310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.433+0000 7f9b3736c740 1 -- 172.21.15.131:0/1938926470 shutdown_connections 2024-04-26T20:20:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.433+0000 7f9b3736c740 1 -- 172.21.15.131:0/1938926470 wait complete. 2024-04-26T20:20:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.433+0000 7f9b3736c740 1 -- start start 2024-04-26T20:20:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.433+0000 7f9b3736c740 1 Processor -- start 2024-04-26T20:20:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b3736c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f6940 0x55e0df400e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b3736c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b00d0 con 0x55e0df3f6940 2024-04-26T20:20:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b3736c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b1540 con 0x55e0df3f0de0 2024-04-26T20:20:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b3736c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b07a0 con 0x55e0df3f17d0 2024-04-26T20:20:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b3736c740 10 client.0 _ll_register_callbacks cb 0x7f9b24009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:20:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b32600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f6940 0x55e0df400e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b31200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b31c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b31200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46674/0 (socket says 172.21.15.131:46674) 2024-04-26T20:20:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b31200640 1 -- 172.21.15.131:0/1118356238 learned_addr learned my addr 172.21.15.131:0/1118356238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b32600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f6940 0x55e0df400e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57542/0 (socket says 172.21.15.131:57542) 2024-04-26T20:20:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b32600640 1 -- 172.21.15.131:0/1118356238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 msgr2=0x55e0df3fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b32600640 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b32600640 1 -- 172.21.15.131:0/1118356238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 msgr2=0x55e0df3ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:20:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b32600640 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b32600640 1 -- 172.21.15.131:0/1118356238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0df30fcb0 con 0x55e0df3f6940 2024-04-26T20:20:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.434+0000 7f9b31200640 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b32600640 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f6940 0x55e0df400e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55e0df402ac0 tx=0x7f9b24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21af133653a2955 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:51.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b13400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:20:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b13400640 1 -- 172.21.15.131:0/1118356238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2402e060 con 0x55e0df3f6940 2024-04-26T20:20:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b13400640 1 -- 172.21.15.131:0/1118356238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9b24004250 con 0x55e0df3f6940 2024-04-26T20:20:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b2005c020 con 0x55e0df3f6940 2024-04-26T20:20:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b13400640 1 -- 172.21.15.131:0/1118356238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b24035050 con 0x55e0df3f6940 2024-04-26T20:20:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b3736c740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-04-26T20:20:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.435+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0df402610 con 0x55e0df3f6940 2024-04-26T20:20:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b13400640 1 -- 172.21.15.131:0/1118356238 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2005c020 con 0x55e0df3f6940 2024-04-26T20:20:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b13400640 1 -- 172.21.15.131:0/1118356238 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9b2401eb00 con 0x55e0df3f6940 2024-04-26T20:20:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b3736c740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:20:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b3736c740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:20:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b3736c740 20 client.15795 populate_metadata read hostname 'smithi131' 2024-04-26T20:20:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df46d090 0x55e0df46f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b3736c740 4 client.15795 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:20:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0df35ce00 con 0x55e0df46d090 2024-04-26T20:20:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.436+0000 7f9b31c00640 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df46d090 0x55e0df46f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.437+0000 7f9b31c00640 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df46d090 0x55e0df46f4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89abd063374f3e60 server_cookie=cbb1083a7865b1d6 in_seq=0 out_seq=0 2024-04-26T20:20:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.437+0000 7f9b13400640 10 client.15795 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.440+0000 7f9b13400640 1 -- 172.21.15.131:0/1118356238 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0df35ce00 con 0x55e0df46d090 2024-04-26T20:20:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.440+0000 7f9b13400640 10 client.15795 handle_command_reply: tid=0 2024-04-26T20:20:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.440+0000 7f9b3736c740 1 client.15795 shutdown 2024-04-26T20:20:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f6940 msgr2=0x55e0df400e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f6940 0x55e0df400e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55e0df402ac0 tx=0x7f9b24004a40 comp rx=0 tx=0).stop 2024-04-26T20:20:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 shutdown_connections 2024-04-26T20:20:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e0df46d090 0x55e0df46f4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0df3f6940 0x55e0df400e10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e0df3f17d0 0x55e0df3fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 --2- 172.21.15.131:0/1118356238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0df3f0de0 0x55e0df3ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b13400640 0 client.15795 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:20:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 >> 172.21.15.131:0/1118356238 conn(0x55e0df3ec450 msgr2=0x55e0df3eadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 shutdown_connections 2024-04-26T20:20:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 1 -- 172.21.15.131:0/1118356238 wait complete. 2024-04-26T20:20:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.441+0000 7f9b3736c740 20 client.15795 trim_cache size 0 max 16384 2024-04-26T20:20:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:20:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:20:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:20:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:20:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b140a3c30 msgr2=0x7f9b140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b140a3c30 0x7f9b140a60f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55e0df3f2030 tx=0x7f9b24009500 comp rx=0 tx=0).stop 2024-04-26T20:20:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c109fd0 msgr2=0x7f9b2c19f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c109fd0 0x7f9b2c19f520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800ea60 tx=0x7f9b1800ef20 comp rx=0 tx=0).stop 2024-04-26T20:20:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 shutdown_connections 2024-04-26T20:20:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b140a3c30 0x7f9b140a60f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2c10a990 0x7f9b2c1a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c109fd0 0x7f9b2c19f520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 --2- 172.21.15.131:0/4098225174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b2c108de0 0x7f9b2c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 >> 172.21.15.131:0/4098225174 conn(0x7f9b2c077dc0 msgr2=0x7f9b2c10d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 shutdown_connections 2024-04-26T20:20:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:30.442+0000 7f9b33000640 1 -- 172.21.15.131:0/4098225174 wait complete. 2024-04-26T20:20:54.325 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:31 smithi131 ceph-mon[27919]: pgmap v665: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:54.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:33 smithi131 ceph-mon[27919]: pgmap v666: 129 pgs: 129 active+clean; 673 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:20:54.424 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:35 smithi131 ceph-mon[27919]: pgmap v667: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:54.533 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:37 smithi131 ceph-mon[27919]: pgmap v668: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:54.859 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:39 smithi131 ceph-mon[27919]: pgmap v669: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:54.919 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:41 smithi131 ceph-mon[27919]: pgmap v670: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:55.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:41 smithi148 ceph-mon[32281]: pgmap v730: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:55.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:43 smithi148 ceph-mon[32281]: pgmap v731: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:55.031 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:45 smithi148 ceph-mon[32281]: pgmap v732: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:20:55.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:47 smithi148 ceph-mon[32281]: pgmap v733: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:55.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:49 smithi148 ceph-mon[32281]: pgmap v734: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:55.125 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:51 smithi148 ceph-mon[32281]: pgmap v735: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:55.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:51 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:20:55.138 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:51 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:55.146 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:51 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:55.227 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:51 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:55.234 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:51 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:55.240 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:20:55.247 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:20:55.255 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:55.290 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:53 smithi148 ceph-mon[32281]: pgmap v736: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:55.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:41 smithi157 ceph-mon[32703]: pgmap v730: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:55.474 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:43 smithi157 ceph-mon[32703]: pgmap v731: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:55.561 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:45 smithi157 ceph-mon[32703]: pgmap v732: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:20:55.615 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:47 smithi157 ceph-mon[32703]: pgmap v733: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:55.623 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:49 smithi157 ceph-mon[32703]: pgmap v734: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:55.669 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:51 smithi157 ceph-mon[32703]: pgmap v735: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:20:55.756 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:51 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:20:55.872 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:51 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:55.953 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:51 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:55.960 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:51 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:56.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:51 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:56.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:20:56.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:20:56.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:20:56.393 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:53 smithi157 ceph-mon[32703]: pgmap v736: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:20:56.401 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:55 smithi148 ceph-mon[32281]: pgmap v737: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:56.495 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:55 smithi157 ceph-mon[32703]: pgmap v737: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:20:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 -- 172.21.15.131:0/2723366371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4301029a0 msgr2=0x7ff430102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 --2- 172.21.15.131:0/2723366371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4301029a0 0x7ff430102dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff424009900 tx=0x7ff42402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:20:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 -- 172.21.15.131:0/2723366371 shutdown_connections 2024-04-26T20:20:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 --2- 172.21.15.131:0/2723366371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 0x7ff43010ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 --2- 172.21.15.131:0/2723366371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff430103b90 0x7ff430104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 --2- 172.21.15.131:0/2723366371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4301029a0 0x7ff430102dc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 -- 172.21.15.131:0/2723366371 >> 172.21.15.131:0/2723366371 conn(0x7ff4300fdf10 msgr2=0x7ff430100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:20:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 -- 172.21.15.131:0/2723366371 shutdown_connections 2024-04-26T20:20:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.576+0000 7ff437e00640 1 -- 172.21.15.131:0/2723366371 wait complete. 2024-04-26T20:20:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.577+0000 7ff437e00640 1 Processor -- start 2024-04-26T20:20:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff437e00640 1 -- start start 2024-04-26T20:20:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff437e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4301029a0 0x7ff43019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff437e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff430103b90 0x7ff43019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff437e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 0x7ff4301a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:57.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff437e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff43010e510 con 0x7ff430104550 2024-04-26T20:20:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff437e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff43010e390 con 0x7ff4301029a0 2024-04-26T20:20:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff437e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff43010e690 con 0x7ff430103b90 2024-04-26T20:20:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff434a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff430103b90 0x7ff43019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff435e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 0x7ff4301a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4301029a0 0x7ff43019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff434a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff430103b90 0x7ff43019b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46840/0 (socket says 172.21.15.131:46840) 2024-04-26T20:20:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff435e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 0x7ff4301a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48876/0 (socket says 172.21.15.131:48876) 2024-04-26T20:20:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff434a00640 1 -- 172.21.15.131:0/4086478538 learned_addr learned my addr 172.21.15.131:0/4086478538 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.578+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4301029a0 0x7ff43019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:59136/0 (socket says 172.21.15.131:59136) 2024-04-26T20:20:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff434a00640 1 -- 172.21.15.131:0/4086478538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4301029a0 msgr2=0x7ff43019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff434a00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4301029a0 0x7ff43019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff434a00640 1 -- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 msgr2=0x7ff4301a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:20:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff434a00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 0x7ff4301a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff434a00640 1 -- 172.21.15.131:0/4086478538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4301a2980 con 0x7ff430103b90 2024-04-26T20:20:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff435e00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 0x7ff4301a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:20:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff434a00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff430103b90 0x7ff43019b2a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9f0 tx=0x7ff41c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff42e000640 1 -- 172.21.15.131:0/4086478538 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00cd80 con 0x7ff430103b90 2024-04-26T20:20:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff42e000640 1 -- 172.21.15.131:0/4086478538 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c0054d0 con 0x7ff430103b90 2024-04-26T20:20:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff42e000640 1 -- 172.21.15.131:0/4086478538 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c010690 con 0x7ff430103b90 2024-04-26T20:20:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.579+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4301a2c70 con 0x7ff430103b90 2024-04-26T20:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.580+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4301a3170 con 0x7ff430103b90 2024-04-26T20:20:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.583+0000 7ff42e000640 1 -- 172.21.15.131:0/4086478538 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff41c004e90 con 0x7ff430103b90 2024-04-26T20:20:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.583+0000 7ff43abad740 1 Processor -- start 2024-04-26T20:20:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.583+0000 7ff43abad740 1 -- start start 2024-04-26T20:20:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff43abad740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb60ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff43abad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff43abad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb68bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff43abad740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563aecac5380 con 0x563aecb60bd0 2024-04-26T20:20:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff43abad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563aeca61c30 con 0x563aecb615c0 2024-04-26T20:20:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff43abad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563aecac1340 con 0x563aecb66730 2024-04-26T20:20:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff434a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb68bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff435e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb60ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:59148/0 (socket says 172.21.15.131:59148) 2024-04-26T20:20:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff435400640 1 -- 172.21.15.131:0/2266444768 learned_addr learned my addr 172.21.15.131:0/2266444768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:20:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff434a00640 1 -- 172.21.15.131:0/2266444768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 msgr2=0x563aecb660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff434a00640 1 --2- 172.21.15.131:0/2266444768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff434a00640 1 -- 172.21.15.131:0/2266444768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 msgr2=0x563aecb60ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.584+0000 7ff434a00640 1 --2- 172.21.15.131:0/2266444768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb60ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff434a00640 1 -- 172.21.15.131:0/2266444768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aeca83940 con 0x563aecb66730 2024-04-26T20:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff42e000640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff4180a3d70 0x7ff4180a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff42e000640 1 -- 172.21.15.131:0/4086478538 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff41c0d0d40 con 0x7ff430103b90 2024-04-26T20:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff435400640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff4180a3d70 0x7ff4180a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff434a00640 1 --2- 172.21.15.131:0/2266444768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb68bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0ce270 tx=0x7ff41c0d4de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f4823b355534ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff422600640 1 -- 172.21.15.131:0/2266444768 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c09c840 con 0x563aecb66730 2024-04-26T20:21:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff422600640 1 -- 172.21.15.131:0/2266444768 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c005680 con 0x563aecb66730 2024-04-26T20:21:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.585+0000 7ff422600640 1 -- 172.21.15.131:0/2266444768 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c005180 con 0x563aecb66730 2024-04-26T20:21:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 -- 172.21.15.131:0/2266444768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 msgr2=0x563aecb68bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 --2- 172.21.15.131:0/2266444768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb68bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0ce270 tx=0x7ff41c0d4de0 comp rx=0 tx=0).stop 2024-04-26T20:21:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff435400640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff4180a3d70 0x7ff4180a6230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x563aecb62030 tx=0x7ff424005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 -- 172.21.15.131:0/2266444768 shutdown_connections 2024-04-26T20:21:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 --2- 172.21.15.131:0/2266444768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb68bc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 --2- 172.21.15.131:0/2266444768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 --2- 172.21.15.131:0/2266444768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb60ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 -- 172.21.15.131:0/2266444768 >> 172.21.15.131:0/2266444768 conn(0x563aecb5c240 msgr2=0x563aecb5c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 -- 172.21.15.131:0/2266444768 shutdown_connections 2024-04-26T20:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.586+0000 7ff43abad740 1 -- 172.21.15.131:0/2266444768 wait complete. 2024-04-26T20:21:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 -- start start 2024-04-26T20:21:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 Processor -- start 2024-04-26T20:21:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb6ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb6b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb72390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563aecb69680 con 0x563aecb60bd0 2024-04-26T20:21:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563aeca4cc60 con 0x563aecb615c0 2024-04-26T20:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563aecb69800 con 0x563aecb66730 2024-04-26T20:21:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.587+0000 7ff43abad740 10 client.0 _ll_register_callbacks cb 0x563aeca78e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb6b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb6b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:59162/0 (socket says 172.21.15.131:59162) 2024-04-26T20:21:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 -- 172.21.15.131:0/3925186721 learned_addr learned my addr 172.21.15.131:0/3925186721 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff435e00640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb72390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff435400640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb6ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 -- 172.21.15.131:0/3925186721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 msgr2=0x563aecb72390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb72390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 -- 172.21.15.131:0/3925186721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 msgr2=0x563aecb6ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb6ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 -- 172.21.15.131:0/3925186721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aeca5b510 con 0x563aecb615c0 2024-04-26T20:21:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff435400640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb6ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff434a00640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb6b3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0ce240 tx=0x7ff41c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef70816748c12629 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff416a00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:21:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.588+0000 7ff416a00640 1 -- 172.21.15.131:0/3925186721 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c09c3d0 con 0x563aecb615c0 2024-04-26T20:21:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff41c005680 con 0x563aecb615c0 2024-04-26T20:21:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff416a00640 1 -- 172.21.15.131:0/3925186721 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff41c09ccd0 con 0x563aecb615c0 2024-04-26T20:21:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff416a00640 1 -- 172.21.15.131:0/3925186721 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff41c09c5b0 con 0x563aecb615c0 2024-04-26T20:21:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 10 client.25928 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aec7d7210 con 0x563aecb615c0 2024-04-26T20:21:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff416a00640 1 -- 172.21.15.131:0/3925186721 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c005680 con 0x563aecb615c0 2024-04-26T20:21:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff416a00640 1 -- 172.21.15.131:0/3925186721 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff41c0a5250 con 0x563aecb615c0 2024-04-26T20:21:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 10 client.25928 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 10 client.25928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 20 client.25928 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecb764f0 0x563aecb788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 4 client.25928 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:21:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.589+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aecad2f90 con 0x563aecb764f0 2024-04-26T20:21:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.590+0000 7ff435400640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecb764f0 0x563aecb788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.590+0000 7ff416a00640 10 client.25928 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.590+0000 7ff435400640 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecb764f0 0x563aecb788e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c4656e28ad3880 server_cookie=69e83829f1df6f05 in_seq=0 out_seq=0 2024-04-26T20:21:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.591+0000 7ff416a00640 1 -- 172.21.15.131:0/3925186721 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563aecad2f90 con 0x563aecb764f0 2024-04-26T20:21:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.591+0000 7ff416a00640 10 client.25928 handle_command_reply: tid=0 2024-04-26T20:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.591+0000 7ff43abad740 1 client.25928 shutdown 2024-04-26T20:21:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.591+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 msgr2=0x563aecb6b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.591+0000 7ff43abad740 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb6b3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0ce240 tx=0x7ff41c0048f0 comp rx=0 tx=0).stop 2024-04-26T20:21:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 shutdown_connections 2024-04-26T20:21:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecb764f0 0x563aecb788e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb72390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb615c0 0x563aecb6b3c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 --2- 172.21.15.131:0/3925186721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb60bd0 0x563aecb6ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 >> 172.21.15.131:0/3925186721 conn(0x563aecb5c240 msgr2=0x563aecb5b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff416a00640 0 client.25928 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 shutdown_connections 2024-04-26T20:21:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 1 -- 172.21.15.131:0/3925186721 wait complete. 2024-04-26T20:21:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.592+0000 7ff43abad740 20 client.25928 trim_cache size 0 max 16384 2024-04-26T20:21:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 Processor -- start 2024-04-26T20:21:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 -- start start 2024-04-26T20:21:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb615c0 0x563aecb72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb66730 0x563aecb5f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301b0120 con 0x563aecb66730 2024-04-26T20:21:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301ad9f0 con 0x563aecb60bd0 2024-04-26T20:21:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.608+0000 7ff43abad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301ab840 con 0x563aecb615c0 2024-04-26T20:21:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb615c0 0x563aecb72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff435e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb66730 0x563aecb5f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb615c0 0x563aecb72350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46874/0 (socket says 172.21.15.131:46874) 2024-04-26T20:21:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb5f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:59164/0 (socket says 172.21.15.131:59164) 2024-04-26T20:21:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 -- 172.21.15.131:0/1166152519 learned_addr learned my addr 172.21.15.131:0/1166152519 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 -- 172.21.15.131:0/1166152519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 msgr2=0x563aecb5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 -- 172.21.15.131:0/1166152519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb66730 msgr2=0x563aecb5f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb66730 0x563aecb5f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 -- 172.21.15.131:0/1166152519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aec8a3620 con 0x563aecb615c0 2024-04-26T20:21:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff435400640 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.609+0000 7ff434a00640 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb615c0 0x563aecb72350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff41c09ad20 tx=0x7ff41c0d4de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3a4812a5bba2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff421c00640 1 -- 172.21.15.131:0/1166152519 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c09c3d0 con 0x563aecb615c0 2024-04-26T20:21:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff421c00640 1 -- 172.21.15.131:0/1166152519 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c005680 con 0x563aecb615c0 2024-04-26T20:21:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff421c00640 1 -- 172.21.15.131:0/1166152519 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0d4280 con 0x563aecb615c0 2024-04-26T20:21:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff43abad740 1 -- 172.21.15.131:0/1166152519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb615c0 msgr2=0x563aecb72350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff43abad740 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb615c0 0x563aecb72350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff41c09ad20 tx=0x7ff41c0d4de0 comp rx=0 tx=0).stop 2024-04-26T20:21:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff43abad740 1 -- 172.21.15.131:0/1166152519 shutdown_connections 2024-04-26T20:21:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff43abad740 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb66730 0x563aecb5f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff43abad740 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb615c0 0x563aecb72350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.610+0000 7ff43abad740 1 --2- 172.21.15.131:0/1166152519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.611+0000 7ff43abad740 1 -- 172.21.15.131:0/1166152519 >> 172.21.15.131:0/1166152519 conn(0x563aecb5c240 msgr2=0x563aecb628c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.611+0000 7ff43abad740 1 -- 172.21.15.131:0/1166152519 shutdown_connections 2024-04-26T20:21:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.611+0000 7ff43abad740 1 -- 172.21.15.131:0/1166152519 wait complete. 2024-04-26T20:21:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.611+0000 7ff43abad740 1 -- start start 2024-04-26T20:21:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.611+0000 7ff43abad740 1 Processor -- start 2024-04-26T20:21:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.611+0000 7ff43abad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb6f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff43abad740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb615c0 0x563aecb6fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff43abad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb70d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff43abad740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301abeb0 con 0x563aecb615c0 2024-04-26T20:21:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff43abad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301ad320 con 0x563aecb60bd0 2024-04-26T20:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff43abad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301ac580 con 0x563aecb66730 2024-04-26T20:21:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff43abad740 10 client.0 _ll_register_callbacks cb 0x7ff428009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb6f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff434a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb615c0 0x563aecb6fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb70d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb6f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:59180/0 (socket says 172.21.15.131:59180) 2024-04-26T20:21:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 -- 172.21.15.131:0/2773456641 learned_addr learned my addr 172.21.15.131:0/2773456641 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 -- 172.21.15.131:0/2773456641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 msgr2=0x563aecb70d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb70d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 -- 172.21.15.131:0/2773456641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb615c0 msgr2=0x563aecb6fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb615c0 0x563aecb6fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 -- 172.21.15.131:0/2773456641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aeca7fcb0 con 0x563aecb60bd0 2024-04-26T20:21:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff435400640 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb6f800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff4240096c0 tx=0x7ff42403f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d35ef2e2afd6590 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.612+0000 7ff416a00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:21:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.613+0000 7ff416a00640 1 -- 172.21.15.131:0/2773456641 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424055070 con 0x563aecb60bd0 2024-04-26T20:21:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.613+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff41c005680 con 0x563aecb60bd0 2024-04-26T20:21:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.613+0000 7ff416a00640 1 -- 172.21.15.131:0/2773456641 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff424051440 con 0x563aecb60bd0 2024-04-26T20:21:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.613+0000 7ff416a00640 1 -- 172.21.15.131:0/2773456641 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff42405e050 con 0x563aecb60bd0 2024-04-26T20:21:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.613+0000 7ff43abad740 10 client.25934 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.613+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aecb724f0 con 0x563aecb60bd0 2024-04-26T20:21:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.613+0000 7ff416a00640 1 -- 172.21.15.131:0/2773456641 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c005680 con 0x563aecb60bd0 2024-04-26T20:21:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff416a00640 1 -- 172.21.15.131:0/2773456641 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff424058020 con 0x563aecb60bd0 2024-04-26T20:21:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff43abad740 10 client.25934 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff43abad740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff43abad740 20 client.25934 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff43abad740 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecbdce60 0x563aecbdf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff43abad740 4 client.25934 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:21:10.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563aecacce00 con 0x563aecbdce60 2024-04-26T20:21:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff434a00640 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecbdce60 0x563aecbdf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff434a00640 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecbdce60 0x563aecbdf280 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be20158be04faeae server_cookie=b36b26d931dd0af4 in_seq=0 out_seq=0 2024-04-26T20:21:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.614+0000 7ff416a00640 10 client.25934 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff416a00640 1 -- 172.21.15.131:0/2773456641 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563aecacce00 con 0x563aecbdce60 2024-04-26T20:21:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff416a00640 10 client.25934 handle_command_reply: tid=0 2024-04-26T20:21:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 client.25934 shutdown 2024-04-26T20:21:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 msgr2=0x563aecb6f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb6f800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff4240096c0 tx=0x7ff42403f040 comp rx=0 tx=0).stop 2024-04-26T20:21:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 shutdown_connections 2024-04-26T20:21:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563aecbdce60 0x563aecbdf280 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563aecb66730 0x563aecb70d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563aecb615c0 0x563aecb6fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff416a00640 0 client.25934 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 --2- 172.21.15.131:0/2773456641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563aecb60bd0 0x563aecb6f800 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 >> 172.21.15.131:0/2773456641 conn(0x563aecb5c240 msgr2=0x563aecb5f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 shutdown_connections 2024-04-26T20:21:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 1 -- 172.21.15.131:0/2773456641 wait complete. 2024-04-26T20:21:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.618+0000 7ff43abad740 20 client.25934 trim_cache size 0 max 16384 2024-04-26T20:21:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:21:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:21:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:21:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:21:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff4180a3d70 msgr2=0x7ff4180a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff4180a3d70 0x7ff4180a6230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x563aecb62030 tx=0x7ff424005a20 comp rx=0 tx=0).stop 2024-04-26T20:21:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff430103b90 msgr2=0x7ff43019b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff430103b90 0x7ff43019b2a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9f0 tx=0x7ff41c00edf0 comp rx=0 tx=0).stop 2024-04-26T20:21:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 shutdown_connections 2024-04-26T20:21:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff430104550 0x7ff4301a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff4180a3d70 0x7ff4180a6230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff430103b90 0x7ff43019b2a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 --2- 172.21.15.131:0/4086478538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4301029a0 0x7ff43019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.619+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 >> 172.21.15.131:0/4086478538 conn(0x7ff4300fdf10 msgr2=0x7ff430074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.620+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 shutdown_connections 2024-04-26T20:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:43.620+0000 7ff437e00640 1 -- 172.21.15.131:0/4086478538 wait complete. 2024-04-26T20:21:12.988 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:43 smithi131 ceph-mon[27919]: pgmap v671: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:12.996 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:43 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:21:13.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:21:13.103 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:21:13.110 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:44 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:21:13.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:45 smithi131 ceph-mon[27919]: pgmap v672: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:13.317 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:47 smithi131 ceph-mon[27919]: pgmap v673: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:13.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:49 smithi131 ceph-mon[27919]: pgmap v674: 129 pgs: 129 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:21:13.405 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:51 smithi131 ceph-mon[27919]: pgmap v675: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:13.584 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:53 smithi131 ceph-mon[27919]: pgmap v676: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:13.679 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:55 smithi131 ceph-mon[27919]: pgmap v677: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:13.789 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:57 smithi148 ceph-mon[32281]: pgmap v738: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:13.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:20:59 smithi148 ceph-mon[32281]: pgmap v739: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:21:13.899 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:01 smithi148 ceph-mon[32281]: pgmap v740: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:13.906 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:03 smithi148 ceph-mon[32281]: pgmap v741: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:13.991 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:05 smithi148 ceph-mon[32281]: pgmap v742: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:14.103 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:07 smithi148 ceph-mon[32281]: pgmap v743: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:14.211 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:09 smithi148 ceph-mon[32281]: pgmap v744: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:14.218 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:11 smithi148 ceph-mon[32281]: pgmap v745: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:14.225 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:57 smithi157 ceph-mon[32703]: pgmap v738: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:14.233 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:20:59 smithi157 ceph-mon[32703]: pgmap v739: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:21:14.333 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:01 smithi157 ceph-mon[32703]: pgmap v740: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:14.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:03 smithi157 ceph-mon[32703]: pgmap v741: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:14.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:05 smithi157 ceph-mon[32703]: pgmap v742: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:14.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:07 smithi157 ceph-mon[32703]: pgmap v743: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:14.684 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:09 smithi157 ceph-mon[32703]: pgmap v744: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:14.825 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:11 smithi157 ceph-mon[32703]: pgmap v745: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:15.000 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:13 smithi148 ceph-mon[32281]: pgmap v746: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:15.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:13 smithi157 ceph-mon[32703]: pgmap v746: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.758+0000 7f8705800640 1 -- 172.21.15.131:0/4243936881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870010a1b0 msgr2=0x7f870010a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.758+0000 7f8705800640 1 --2- 172.21.15.131:0/4243936881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870010a1b0 0x7f870010a630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009860 tx=0x7f86e802a340 comp rx=0 tx=0).stop 2024-04-26T20:21:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.759+0000 7f8705800640 1 -- 172.21.15.131:0/4243936881 shutdown_connections 2024-04-26T20:21:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.759+0000 7f8705800640 1 --2- 172.21.15.131:0/4243936881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010ab70 0x7f8700111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.759+0000 7f8705800640 1 --2- 172.21.15.131:0/4243936881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870010a1b0 0x7f870010a630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.759+0000 7f8705800640 1 --2- 172.21.15.131:0/4243936881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8700108fc0 0x7f87001093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.759+0000 7f8705800640 1 -- 172.21.15.131:0/4243936881 >> 172.21.15.131:0/4243936881 conn(0x7f8700077f80 msgr2=0x7f870007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.759+0000 7f8705800640 1 -- 172.21.15.131:0/4243936881 shutdown_connections 2024-04-26T20:21:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.759+0000 7f8705800640 1 -- 172.21.15.131:0/4243936881 wait complete. 2024-04-26T20:21:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 Processor -- start 2024-04-26T20:21:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 -- start start 2024-04-26T20:21:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8700108fc0 0x7f870019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010a1b0 0x7f870019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870010ab70 0x7f87001a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8700114b00 con 0x7f870010a1b0 2024-04-26T20:21:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8700114980 con 0x7f8700108fc0 2024-04-26T20:21:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f8705800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8700114c80 con 0x7f870010ab70 2024-04-26T20:21:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.760+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010a1b0 0x7f870019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010a1b0 0x7f870019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52726/0 (socket says 172.21.15.131:52726) 2024-04-26T20:21:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 -- 172.21.15.131:0/2167475779 learned_addr learned my addr 172.21.15.131:0/2167475779 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86ff400640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870010ab70 0x7f87001a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86fea00640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8700108fc0 0x7f870019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 -- 172.21.15.131:0/2167475779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870010ab70 msgr2=0x7f87001a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870010ab70 0x7f87001a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 -- 172.21.15.131:0/2167475779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8700108fc0 msgr2=0x7f870019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8700108fc0 0x7f870019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 -- 172.21.15.131:0/2167475779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87001a6d20 con 0x7f870010a1b0 2024-04-26T20:21:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86ff400640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870010ab70 0x7f87001a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.761+0000 7f86f7e00640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010a1b0 0x7f870019f640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86e80057c0 tx=0x7f86e8032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.762+0000 7f86f7400640 1 -- 172.21.15.131:0/2167475779 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8037040 con 0x7f870010a1b0 2024-04-26T20:21:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.762+0000 7f86f7400640 1 -- 172.21.15.131:0/2167475779 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e8030bd0 con 0x7f870010a1b0 2024-04-26T20:21:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.762+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87001a6fb0 con 0x7f870010a1b0 2024-04-26T20:21:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.762+0000 7f86f7400640 1 -- 172.21.15.131:0/2167475779 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8005280 con 0x7f870010a1b0 2024-04-26T20:21:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.762+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87001a73c0 con 0x7f870010a1b0 2024-04-26T20:21:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 Processor -- start 2024-04-26T20:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f86f7400640 1 -- 172.21.15.131:0/2167475779 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f86e8043020 con 0x7f870010a1b0 2024-04-26T20:21:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 -- start start 2024-04-26T20:21:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d94559fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a5730 0x55d9455a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d945504380 con 0x55d9455a5730 2024-04-26T20:21:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d9454a0c30 con 0x55d9455a05c0 2024-04-26T20:21:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.765+0000 7f870863d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d945500340 con 0x55d94559fbd0 2024-04-26T20:21:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a5730 0x55d9455a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d94559fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d94559fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44384/0 (socket says 172.21.15.131:44384) 2024-04-26T20:21:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 -- 172.21.15.131:0/6635710 learned_addr learned my addr 172.21.15.131:0/6635710 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86fea00640 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 -- 172.21.15.131:0/6635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 msgr2=0x55d9455a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 -- 172.21.15.131:0/6635710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a5730 msgr2=0x55d9455a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a5730 0x55d9455a7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 -- 172.21.15.131:0/6635710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9454c2940 con 0x55d94559fbd0 2024-04-26T20:21:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86fea00640 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.766+0000 7f86ff400640 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d94559fff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f86f800e8f0 tx=0x7f86f80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27c6d77dbfc02ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.767+0000 7f86f7400640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86e00a3d70 0x7f86e00a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.767+0000 7f86efe00640 1 -- 172.21.15.131:0/6635710 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8017420 con 0x55d94559fbd0 2024-04-26T20:21:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.767+0000 7f86f7400640 1 -- 172.21.15.131:0/2167475779 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f86e80efe60 con 0x7f870010a1b0 2024-04-26T20:21:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.767+0000 7f86efe00640 1 -- 172.21.15.131:0/6635710 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f801c070 con 0x55d94559fbd0 2024-04-26T20:21:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.767+0000 7f86efe00640 1 -- 172.21.15.131:0/6635710 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8016600 con 0x55d94559fbd0 2024-04-26T20:21:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.767+0000 7f86fea00640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86e00a3d70 0x7f86e00a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 -- 172.21.15.131:0/6635710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 msgr2=0x55d94559fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d94559fff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f86f800e8f0 tx=0x7f86f80095c0 comp rx=0 tx=0).stop 2024-04-26T20:21:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f86fea00640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86e00a3d70 0x7f86e00a6230 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d9455a09c0 tx=0x7f86f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 -- 172.21.15.131:0/6635710 shutdown_connections 2024-04-26T20:21:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a5730 0x55d9455a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 --2- 172.21.15.131:0/6635710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d94559fff0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 -- 172.21.15.131:0/6635710 >> 172.21.15.131:0/6635710 conn(0x55d94559b240 msgr2=0x55d94559b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 -- 172.21.15.131:0/6635710 shutdown_connections 2024-04-26T20:21:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 -- 172.21.15.131:0/6635710 wait complete. 2024-04-26T20:21:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.768+0000 7f870863d740 1 -- start start 2024-04-26T20:21:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 1 Processor -- start 2024-04-26T20:21:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9455a86e0 con 0x55d94559fbd0 2024-04-26T20:21:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d94548bc60 con 0x55d9455a05c0 2024-04-26T20:21:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d9455a8860 con 0x55d9455a5730 2024-04-26T20:21:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f870863d740 10 client.0 _ll_register_callbacks cb 0x55d9454b7e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f86fea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f86fea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52734/0 (socket says 172.21.15.131:52734) 2024-04-26T20:21:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f86fea00640 1 -- 172.21.15.131:0/4229841891 learned_addr learned my addr 172.21.15.131:0/4229841891 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.769+0000 7f86ff400640 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86f7e00640 1 -- 172.21.15.131:0/4229841891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 msgr2=0x55d9455b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86f7e00640 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86f7e00640 1 -- 172.21.15.131:0/4229841891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 msgr2=0x55d9455a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86f7e00640 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455a9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86f7e00640 1 -- 172.21.15.131:0/4229841891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94549a510 con 0x55d9455a05c0 2024-04-26T20:21:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86f7e00640 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455aa3f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86e80ed4d0 tx=0x7f86e802a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139fc9e0f07ed3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:21:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86ecc00640 1 -- 172.21.15.131:0/4229841891 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80dae60 con 0x55d9455a05c0 2024-04-26T20:21:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86ecc00640 1 -- 172.21.15.131:0/4229841891 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f86e80ed610 con 0x55d9455a05c0 2024-04-26T20:21:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f801c070 con 0x55d9455a05c0 2024-04-26T20:21:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.770+0000 7f86ecc00640 1 -- 172.21.15.131:0/4229841891 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86e80bb7e0 con 0x55d9455a05c0 2024-04-26T20:21:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 10 client.25946 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d945216210 con 0x55d9455a05c0 2024-04-26T20:21:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f86ecc00640 1 -- 172.21.15.131:0/4229841891 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f801c070 con 0x55d9455a05c0 2024-04-26T20:21:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f86ecc00640 1 -- 172.21.15.131:0/4229841891 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f86e80dc0c0 con 0x55d9455a05c0 2024-04-26T20:21:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 10 client.25946 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 20 client.25946 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d9455b58b0 0x55d9455b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 4 client.25946 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:21:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d945511f90 con 0x55d9455b58b0 2024-04-26T20:21:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.771+0000 7f86fea00640 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d9455b58b0 0x55d9455b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.772+0000 7f86fea00640 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d9455b58b0 0x55d9455b7cc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41436ea25f1c2a5e server_cookie=437d3ff319ef2111 in_seq=0 out_seq=0 2024-04-26T20:21:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.772+0000 7f86ecc00640 10 client.25946 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f86ecc00640 1 -- 172.21.15.131:0/4229841891 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d945511f90 con 0x55d9455b58b0 2024-04-26T20:21:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f86ecc00640 10 client.25946 handle_command_reply: tid=0 2024-04-26T20:21:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 client.25946 shutdown 2024-04-26T20:21:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 msgr2=0x55d9455aa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455aa3f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86e80ed4d0 tx=0x7f86e802a340 comp rx=0 tx=0).stop 2024-04-26T20:21:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 shutdown_connections 2024-04-26T20:21:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d9455b58b0 0x55d9455b7cc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455aa3f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 --2- 172.21.15.131:0/4229841891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f86ecc00640 0 client.25946 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.773+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 >> 172.21.15.131:0/4229841891 conn(0x55d94559b240 msgr2=0x55d94559a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.774+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 shutdown_connections 2024-04-26T20:21:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.774+0000 7f870863d740 1 -- 172.21.15.131:0/4229841891 wait complete. 2024-04-26T20:21:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.774+0000 7f870863d740 20 client.25946 trim_cache size 0 max 16384 2024-04-26T20:21:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 Processor -- start 2024-04-26T20:21:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 -- start start 2024-04-26T20:21:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d9455bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a05c0 0x55d9455b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a5730 0x55d9455bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b4520 con 0x55d9455a05c0 2024-04-26T20:21:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b1df0 con 0x55d9455a5730 2024-04-26T20:21:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.790+0000 7f870863d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87001afc40 con 0x55d94559fbd0 2024-04-26T20:21:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a05c0 0x55d9455b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d9455bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d9455bc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44398/0 (socket says 172.21.15.131:44398) 2024-04-26T20:21:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86ff400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a5730 0x55d9455bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 -- 172.21.15.131:0/3899486177 learned_addr learned my addr 172.21.15.131:0/3899486177 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 -- 172.21.15.131:0/3899486177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a5730 msgr2=0x55d9455bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a5730 0x55d9455bcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 -- 172.21.15.131:0/3899486177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a05c0 msgr2=0x55d9455b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a05c0 0x55d9455b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 -- 172.21.15.131:0/3899486177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9452e2620 con 0x55d94559fbd0 2024-04-26T20:21:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86f7e00640 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a05c0 0x55d9455b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.791+0000 7f86fea00640 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d9455bc5c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55d9455b8540 tx=0x7f86f001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa84995924d852e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f86ef400640 1 -- 172.21.15.131:0/3899486177 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f001dd90 con 0x55d94559fbd0 2024-04-26T20:21:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f86ef400640 1 -- 172.21.15.131:0/3899486177 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0005020 con 0x55d94559fbd0 2024-04-26T20:21:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f86ef400640 1 -- 172.21.15.131:0/3899486177 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0004a20 con 0x55d94559fbd0 2024-04-26T20:21:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f870863d740 1 -- 172.21.15.131:0/3899486177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 msgr2=0x55d9455bc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f870863d740 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d9455bc5c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55d9455b8540 tx=0x7f86f001b000 comp rx=0 tx=0).stop 2024-04-26T20:21:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f870863d740 1 -- 172.21.15.131:0/3899486177 shutdown_connections 2024-04-26T20:21:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f870863d740 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a5730 0x55d9455bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f870863d740 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9455a05c0 0x55d9455b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f870863d740 1 --2- 172.21.15.131:0/3899486177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d94559fbd0 0x55d9455bc5c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.792+0000 7f870863d740 1 -- 172.21.15.131:0/3899486177 >> 172.21.15.131:0/3899486177 conn(0x55d94559b240 msgr2=0x55d94559e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 -- 172.21.15.131:0/3899486177 shutdown_connections 2024-04-26T20:21:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 -- 172.21.15.131:0/3899486177 wait complete. 2024-04-26T20:21:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 -- start start 2024-04-26T20:21:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 Processor -- start 2024-04-26T20:21:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b02b0 con 0x55d94559fbd0 2024-04-26T20:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b1720 con 0x55d9455a05c0 2024-04-26T20:21:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.793+0000 7f870863d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b0980 con 0x55d9455a5730 2024-04-26T20:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f870863d740 10 client.0 _ll_register_callbacks cb 0x7f86e80f3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86fea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86fea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52742/0 (socket says 172.21.15.131:52742) 2024-04-26T20:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86ff400640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86f7e00640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86fea00640 1 -- 172.21.15.131:0/262681898 learned_addr learned my addr 172.21.15.131:0/262681898 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86ff400640 1 -- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 msgr2=0x55d9455af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86ff400640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86ff400640 1 -- 172.21.15.131:0/262681898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 msgr2=0x55d9455b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86ff400640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86ff400640 1 -- 172.21.15.131:0/262681898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9454becb0 con 0x55d9455a5730 2024-04-26T20:21:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.794+0000 7f86f7e00640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f86ff400640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455afbd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f86f800e540 tx=0x7f86f800dec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca180026b30a73e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f86ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:21:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f86ecc00640 1 -- 172.21.15.131:0/262681898 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8003e40 con 0x55d9455a5730 2024-04-26T20:21:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f0005020 con 0x55d9455a5730 2024-04-26T20:21:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f86ecc00640 1 -- 172.21.15.131:0/262681898 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f86f8005040 con 0x55d9455a5730 2024-04-26T20:21:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f86ecc00640 1 -- 172.21.15.131:0/262681898 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86f8035050 con 0x55d9455a5730 2024-04-26T20:21:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f870863d740 10 client.25939 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d94561c980 con 0x55d9455a5730 2024-04-26T20:21:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.795+0000 7f86ecc00640 1 -- 172.21.15.131:0/262681898 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0005020 con 0x55d9455a5730 2024-04-26T20:21:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f86ecc00640 1 -- 172.21.15.131:0/262681898 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f86f800b040 con 0x55d9455a5730 2024-04-26T20:21:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f870863d740 10 client.25939 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f870863d740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f870863d740 20 client.25939 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f870863d740 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d94561d120 0x55d94561f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f870863d740 4 client.25939 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:21:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d94550be00 con 0x55d94561d120 2024-04-26T20:21:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.796+0000 7f86fea00640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d94561d120 0x55d94561f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.797+0000 7f86fea00640 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d94561d120 0x55d94561f540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e829ad1a6fa97a7 server_cookie=42c5e1868f9c0de in_seq=0 out_seq=0 2024-04-26T20:21:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.797+0000 7f86ecc00640 10 client.25939 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.800+0000 7f86ecc00640 1 -- 172.21.15.131:0/262681898 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d94550be00 con 0x55d94561d120 2024-04-26T20:21:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.800+0000 7f86ecc00640 10 client.25939 handle_command_reply: tid=0 2024-04-26T20:21:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.800+0000 7f870863d740 1 client.25939 shutdown 2024-04-26T20:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.800+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 msgr2=0x55d9455afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.800+0000 7f870863d740 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455afbd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f86f800e540 tx=0x7f86f800dec0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 shutdown_connections 2024-04-26T20:21:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d94561d120 0x55d94561f540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9455a5730 0x55d9455afbd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9455a05c0 0x55d9455af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 --2- 172.21.15.131:0/262681898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d94559fbd0 0x55d9455b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 >> 172.21.15.131:0/262681898 conn(0x55d94559b240 msgr2=0x55d945599bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f86ecc00640 0 client.25939 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 shutdown_connections 2024-04-26T20:21:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 1 -- 172.21.15.131:0/262681898 wait complete. 2024-04-26T20:21:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.801+0000 7f870863d740 20 client.25939 trim_cache size 0 max 16384 2024-04-26T20:21:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:21:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:21:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:21:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:21:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86e00a3d70 msgr2=0x7f86e00a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86e00a3d70 0x7f86e00a6230 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d9455a09c0 tx=0x7f86f0005f90 comp rx=0 tx=0).stop 2024-04-26T20:21:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010a1b0 msgr2=0x7f870019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010a1b0 0x7f870019f640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86e80057c0 tx=0x7f86e8032da0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 shutdown_connections 2024-04-26T20:21:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f86e00a3d70 0x7f86e00a6230 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870010ab70 0x7f87001a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f870010a1b0 0x7f870019f640 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 --2- 172.21.15.131:0/2167475779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8700108fc0 0x7f870019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 >> 172.21.15.131:0/2167475779 conn(0x7f8700077f80 msgr2=0x7f870010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 shutdown_connections 2024-04-26T20:21:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:18:56.802+0000 7f8705800640 1 -- 172.21.15.131:0/2167475779 wait complete. 2024-04-26T20:21:28.821 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:57 smithi131 ceph-mon[27919]: pgmap v678: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:28.923 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:18:59 smithi131 ceph-mon[27919]: pgmap v679: 129 pgs: 129 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:29.026 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:01 smithi131 ceph-mon[27919]: pgmap v680: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:21:29.033 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:03 smithi131 ceph-mon[27919]: pgmap v681: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.041 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:05 smithi131 ceph-mon[27919]: pgmap v682: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:29.048 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:07 smithi131 ceph-mon[27919]: pgmap v683: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.058 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:15 smithi148 ceph-mon[32281]: pgmap v747: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:29.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:17 smithi148 ceph-mon[32281]: pgmap v748: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.123 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:19 smithi148 ceph-mon[32281]: pgmap v749: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:29.219 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:21 smithi148 ceph-mon[32281]: pgmap v750: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:29.289 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:23 smithi148 ceph-mon[32281]: pgmap v751: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.345 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:25 smithi148 ceph-mon[32281]: pgmap v752: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:21:29.352 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:27 smithi148 ceph-mon[32281]: pgmap v753: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.359 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:15 smithi157 ceph-mon[32703]: pgmap v747: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:29.367 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:17 smithi157 ceph-mon[32703]: pgmap v748: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:19 smithi157 ceph-mon[32703]: pgmap v749: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:29.532 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:21 smithi157 ceph-mon[32703]: pgmap v750: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:29.565 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:23 smithi157 ceph-mon[32703]: pgmap v751: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.573 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:25 smithi157 ceph-mon[32703]: pgmap v752: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:21:29.578 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:27 smithi157 ceph-mon[32703]: pgmap v753: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:29.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 -- 172.21.15.131:0/2955678959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 msgr2=0x7f9b78106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:29.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/2955678959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 0x7f9b78106220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009860 tx=0x7f9b6402a340 comp rx=0 tx=0).stop 2024-04-26T20:21:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 -- 172.21.15.131:0/2955678959 shutdown_connections 2024-04-26T20:21:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/2955678959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b78106760 0x7f9b7810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/2955678959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 0x7f9b78106220 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:29.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/2955678959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 0x7f9b78104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:29.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 -- 172.21.15.131:0/2955678959 >> 172.21.15.131:0/2955678959 conn(0x7f9b78100120 msgr2=0x7f9b78102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:29.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 -- 172.21.15.131:0/2955678959 shutdown_connections 2024-04-26T20:21:29.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.241+0000 7f9b7fc00640 1 -- 172.21.15.131:0/2955678959 wait complete. 2024-04-26T20:21:30.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.242+0000 7f9b7fc00640 1 Processor -- start 2024-04-26T20:21:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7fc00640 1 -- start start 2024-04-26T20:21:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7fc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 0x7f9b7819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 0x7f9b7819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:30.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7fc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b78106760 0x7f9b781a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:30.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7fc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781106e0 con 0x7f9b78104bb0 2024-04-26T20:21:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7fc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78110560 con 0x7f9b78106760 2024-04-26T20:21:30.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78110860 con 0x7f9b78105da0 2024-04-26T20:21:30.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 0x7f9b7819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:30.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 0x7f9b7819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38668/0 (socket says 172.21.15.131:38668) 2024-04-26T20:21:30.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7d200640 1 -- 172.21.15.131:0/3310992220 learned_addr learned my addr 172.21.15.131:0/3310992220 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:30.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b78106760 0x7f9b781a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:30.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.243+0000 7f9b77e00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 0x7f9b7819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b7d200640 1 -- 172.21.15.131:0/3310992220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 msgr2=0x7f9b7819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:30.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b7d200640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 0x7f9b7819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b7d200640 1 -- 172.21.15.131:0/3310992220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b78106760 msgr2=0x7f9b781a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:30.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b7d200640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b78106760 0x7f9b781a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:30.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b7d200640 1 -- 172.21.15.131:0/3310992220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b781a2a30 con 0x7f9b78104bb0 2024-04-26T20:21:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b7d200640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 0x7f9b7819ae10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800e9f0 tx=0x7f9b6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b75600640 1 -- 172.21.15.131:0/3310992220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800ccc0 con 0x7f9b78104bb0 2024-04-26T20:21:31.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b781a2d20 con 0x7f9b78104bb0 2024-04-26T20:21:31.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.244+0000 7f9b75600640 1 -- 172.21.15.131:0/3310992220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b680035f0 con 0x7f9b78104bb0 2024-04-26T20:21:31.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.245+0000 7f9b75600640 1 -- 172.21.15.131:0/3310992220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800ce60 con 0x7f9b78104bb0 2024-04-26T20:21:31.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.245+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b781a3180 con 0x7f9b78104bb0 2024-04-26T20:21:31.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 Processor -- start 2024-04-26T20:21:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 -- start start 2024-04-26T20:21:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b75600640 1 -- 172.21.15.131:0/3310992220 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9b68004000 con 0x7f9b78104bb0 2024-04-26T20:21:31.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378cea0 0x55fa4378d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:31.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378d890 0x55fa43792390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:31.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa43794e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa436f0c70 con 0x55fa4378d890 2024-04-26T20:21:31.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa436a9980 con 0x55fa4378cea0 2024-04-26T20:21:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b829f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fa436ecc30 con 0x55fa43792a00 2024-04-26T20:21:31.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.248+0000 7f9b7d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378d890 0x55fa43792390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:31.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378cea0 0x55fa4378d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378d890 0x55fa43792390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38680/0 (socket says 172.21.15.131:38680) 2024-04-26T20:21:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7d200640 1 -- 172.21.15.131:0/4015517495 learned_addr learned my addr 172.21.15.131:0/4015517495 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b77e00640 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa43794e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:31.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7dc00640 1 -- 172.21.15.131:0/4015517495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 msgr2=0x55fa43794e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:32.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa43794e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:32.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7dc00640 1 -- 172.21.15.131:0/4015517495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378d890 msgr2=0x55fa43792390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:32.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378d890 0x55fa43792390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:32.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7dc00640 1 -- 172.21.15.131:0/4015517495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa436817b0 con 0x55fa4378cea0 2024-04-26T20:21:32.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.249+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378cea0 0x55fa4378d2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009f00 tx=0x7f9b70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4177e4bf494512 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:32.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b6e000640 1 -- 172.21.15.131:0/4015517495 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70004d20 con 0x55fa4378cea0 2024-04-26T20:21:32.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b6e000640 1 -- 172.21.15.131:0/4015517495 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70004030 con 0x55fa4378cea0 2024-04-26T20:21:32.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b6e000640 1 -- 172.21.15.131:0/4015517495 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70005280 con 0x55fa4378cea0 2024-04-26T20:21:32.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b75600640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b5c0a3c40 0x7f9b5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:32.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b75600640 1 -- 172.21.15.131:0/3310992220 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9b680d0ba0 con 0x7f9b78104bb0 2024-04-26T20:21:32.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b7d200640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b5c0a3c40 0x7f9b5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:32.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b829f1740 1 -- 172.21.15.131:0/4015517495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378cea0 msgr2=0x55fa4378d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:32.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.250+0000 7f9b829f1740 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378cea0 0x55fa4378d2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009f00 tx=0x7f9b70011810 comp rx=0 tx=0).stop 2024-04-26T20:21:32.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 -- 172.21.15.131:0/4015517495 shutdown_connections 2024-04-26T20:21:32.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa43794e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378d890 0x55fa43792390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:32.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 --2- 172.21.15.131:0/4015517495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378cea0 0x55fa4378d2c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:32.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 -- 172.21.15.131:0/4015517495 >> 172.21.15.131:0/4015517495 conn(0x55fa43788510 msgr2=0x55fa43788910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 -- 172.21.15.131:0/4015517495 shutdown_connections 2024-04-26T20:21:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 -- 172.21.15.131:0/4015517495 wait complete. 2024-04-26T20:21:33.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b7d200640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b5c0a3c40 0x7f9b5c0a6100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9b68003fd0 tx=0x7f9b680d4650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:33.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 -- start start 2024-04-26T20:21:33.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.251+0000 7f9b829f1740 1 Processor -- start 2024-04-26T20:21:33.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa43796d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa437972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa4379e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:33.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b829f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa436eff80 con 0x55fa4378cea0 2024-04-26T20:21:33.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b829f1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa4369d260 con 0x55fa4378d890 2024-04-26T20:21:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b829f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fa436f52a0 con 0x55fa43792a00 2024-04-26T20:21:33.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b829f1740 10 client.0 _ll_register_callbacks cb 0x55fa436e7960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:33.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b7dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa43796d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b7dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa43796d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38692/0 (socket says 172.21.15.131:38692) 2024-04-26T20:21:33.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b7dc00640 1 -- 172.21.15.131:0/168519025 learned_addr learned my addr 172.21.15.131:0/168519025 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b77e00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa437972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.252+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa4379e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b7dc00640 1 -- 172.21.15.131:0/168519025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 msgr2=0x55fa4379e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:33.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa4379e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:34.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b7dc00640 1 -- 172.21.15.131:0/168519025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 msgr2=0x55fa437972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:34.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa437972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:34.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b7dc00640 1 -- 172.21.15.131:0/168519025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4367f1c0 con 0x55fa4378cea0 2024-04-26T20:21:34.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b77e00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa437972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:34.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa43796d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009270 tx=0x7f9b70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db95cf3422593468 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b62a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:21:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b62a00640 1 -- 172.21.15.131:0/168519025 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7002d060 con 0x55fa4378cea0 2024-04-26T20:21:34.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b62a00640 1 -- 172.21.15.131:0/168519025 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9b70003e00 con 0x55fa4378cea0 2024-04-26T20:21:34.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.253+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b70004030 con 0x55fa4378cea0 2024-04-26T20:21:34.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b62a00640 1 -- 172.21.15.131:0/168519025 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b70005040 con 0x55fa4378cea0 2024-04-26T20:21:34.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:34.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa434023f0 con 0x55fa4378cea0 2024-04-26T20:21:34.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b62a00640 1 -- 172.21.15.131:0/168519025 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70004030 con 0x55fa4378cea0 2024-04-26T20:21:34.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b62a00640 1 -- 172.21.15.131:0/168519025 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9b70024070 con 0x55fa4378cea0 2024-04-26T20:21:34.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 10 client.15843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-26T20:21:34.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 20 client.15843 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa437a2560 0x55fa437a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 4 client.15843 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:21:34.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa436fe880 con 0x55fa437a2560 2024-04-26T20:21:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.254+0000 7f9b77e00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa437a2560 0x55fa437a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.255+0000 7f9b77e00640 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa437a2560 0x55fa437a4970 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3b7b1f5732ddd98 server_cookie=d857f73c8cf4db64 in_seq=0 out_seq=0 2024-04-26T20:21:34.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.255+0000 7f9b62a00640 10 client.15843 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:21:35.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b62a00640 1 -- 172.21.15.131:0/168519025 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa436fe880 con 0x55fa437a2560 2024-04-26T20:21:35.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b62a00640 10 client.15843 handle_command_reply: tid=0 2024-04-26T20:21:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 client.15843 shutdown 2024-04-26T20:21:35.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 msgr2=0x55fa43796d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa43796d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009270 tx=0x7f9b70004a40 comp rx=0 tx=0).stop 2024-04-26T20:21:35.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 shutdown_connections 2024-04-26T20:21:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa437a2560 0x55fa437a4970 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa43792a00 0x55fa4379e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa437972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 --2- 172.21.15.131:0/168519025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa43796d80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:35.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b62a00640 0 client.15843 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:21:35.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 >> 172.21.15.131:0/168519025 conn(0x55fa43788510 msgr2=0x55fa43787690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 shutdown_connections 2024-04-26T20:21:35.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 1 -- 172.21.15.131:0/168519025 wait complete. 2024-04-26T20:21:35.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.256+0000 7f9b829f1740 20 client.15843 trim_cache size 0 max 16384 2024-04-26T20:21:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 Processor -- start 2024-04-26T20:21:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 -- start start 2024-04-26T20:21:35.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378cea0 0x55fa4378bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa4379e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa43792a00 0x55fa43786e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:35.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781b01c0 con 0x55fa43792a00 2024-04-26T20:21:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781ada90 con 0x55fa4378d890 2024-04-26T20:21:36.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.271+0000 7f9b829f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781ab8e0 con 0x55fa4378cea0 2024-04-26T20:21:36.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378cea0 0x55fa4378bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378cea0 0x55fa4378bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37542/0 (socket says 172.21.15.131:37542) 2024-04-26T20:21:36.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 -- 172.21.15.131:0/1342316090 learned_addr learned my addr 172.21.15.131:0/1342316090 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b77e00640 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa4379e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:36.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa43792a00 0x55fa43786e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 -- 172.21.15.131:0/1342316090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 msgr2=0x55fa4379e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa4379e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:36.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 -- 172.21.15.131:0/1342316090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa43792a00 msgr2=0x55fa43786e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa43792a00 0x55fa43786e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:36.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 -- 172.21.15.131:0/1342316090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa434cc7d0 con 0x55fa4378cea0 2024-04-26T20:21:36.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b77e00640 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa4379e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.272+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378cea0 0x55fa4378bd90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b70002230 tx=0x7f9b70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8a44af771de2339 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:36.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b6d600640 1 -- 172.21.15.131:0/1342316090 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7002d070 con 0x55fa4378cea0 2024-04-26T20:21:36.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b6d600640 1 -- 172.21.15.131:0/1342316090 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70011d10 con 0x55fa4378cea0 2024-04-26T20:21:36.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b6d600640 1 -- 172.21.15.131:0/1342316090 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70028a90 con 0x55fa4378cea0 2024-04-26T20:21:36.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 -- 172.21.15.131:0/1342316090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378cea0 msgr2=0x55fa4378bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:37.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378cea0 0x55fa4378bd90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b70002230 tx=0x7f9b70011810 comp rx=0 tx=0).stop 2024-04-26T20:21:37.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 -- 172.21.15.131:0/1342316090 shutdown_connections 2024-04-26T20:21:37.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa43792a00 0x55fa43786e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa4378d890 0x55fa4379e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:37.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 --2- 172.21.15.131:0/1342316090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378cea0 0x55fa4378bd90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:37.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 -- 172.21.15.131:0/1342316090 >> 172.21.15.131:0/1342316090 conn(0x55fa43788510 msgr2=0x55fa43797000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:37.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.273+0000 7f9b829f1740 1 -- 172.21.15.131:0/1342316090 shutdown_connections 2024-04-26T20:21:37.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.274+0000 7f9b829f1740 1 -- 172.21.15.131:0/1342316090 wait complete. 2024-04-26T20:21:37.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.274+0000 7f9b829f1740 1 -- start start 2024-04-26T20:21:37.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.274+0000 7f9b829f1740 1 Processor -- start 2024-04-26T20:21:37.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.274+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa4379af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:37.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.274+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378d890 0x55fa4379b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:37.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b829f1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa43792a00 0x55fa43806ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:37.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b829f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781abf50 con 0x55fa4378cea0 2024-04-26T20:21:37.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b829f1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781ad3c0 con 0x55fa43792a00 2024-04-26T20:21:37.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b829f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781ac620 con 0x55fa4378d890 2024-04-26T20:21:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b829f1740 10 client.0 _ll_register_callbacks cb 0x7f9b680d4520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:37.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa4379af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa4379af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38700/0 (socket says 172.21.15.131:38700) 2024-04-26T20:21:37.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 -- 172.21.15.131:0/3017668187 learned_addr learned my addr 172.21.15.131:0/3017668187 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:37.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b77e00640 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378d890 0x55fa4379b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:37.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa43792a00 0x55fa43806ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 -- 172.21.15.131:0/3017668187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378d890 msgr2=0x55fa4379b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378d890 0x55fa4379b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 -- 172.21.15.131:0/3017668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa43792a00 msgr2=0x55fa43806ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa43792a00 0x55fa43806ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 -- 172.21.15.131:0/3017668187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa436972a0 con 0x55fa4378cea0 2024-04-26T20:21:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b7dc00640 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa4379af10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9b70004590 tx=0x7f9b700287c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9548bf076112c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:38.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.275+0000 7f9b62a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:21:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b62a00640 1 -- 172.21.15.131:0/3017668187 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7002d070 con 0x55fa4378cea0 2024-04-26T20:21:38.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b62a00640 1 -- 172.21.15.131:0/3017668187 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9b70028a90 con 0x55fa4378cea0 2024-04-26T20:21:38.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b70011d10 con 0x55fa4378cea0 2024-04-26T20:21:38.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b62a00640 1 -- 172.21.15.131:0/3017668187 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b7003d040 con 0x55fa4378cea0 2024-04-26T20:21:38.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b829f1740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa43807e20 con 0x55fa4378cea0 2024-04-26T20:21:38.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b62a00640 1 -- 172.21.15.131:0/3017668187 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70011d10 con 0x55fa4378cea0 2024-04-26T20:21:38.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.276+0000 7f9b62a00640 1 -- 172.21.15.131:0/3017668187 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9b70024050 con 0x55fa4378cea0 2024-04-26T20:21:38.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b829f1740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b829f1740 10 client.15849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-26T20:21:38.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b829f1740 20 client.15849 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:38.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b829f1740 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa438085e0 0x55fa4380aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b829f1740 4 client.15849 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:21:38.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa436f86f0 con 0x55fa438085e0 2024-04-26T20:21:38.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b77e00640 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa438085e0 0x55fa4380aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b77e00640 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa438085e0 0x55fa4380aa00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb901df857029719 server_cookie=eb37a4ed777ef890 in_seq=0 out_seq=0 2024-04-26T20:21:38.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.277+0000 7f9b62a00640 10 client.15849 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:21:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b62a00640 1 -- 172.21.15.131:0/3017668187 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa436f86f0 con 0x55fa438085e0 2024-04-26T20:21:38.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b62a00640 10 client.15849 handle_command_reply: tid=0 2024-04-26T20:21:38.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 client.15849 shutdown 2024-04-26T20:21:38.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 msgr2=0x55fa4379af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa4379af10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9b70004590 tx=0x7f9b700287c0 comp rx=0 tx=0).stop 2024-04-26T20:21:39.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 shutdown_connections 2024-04-26T20:21:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55fa438085e0 0x55fa4380aa00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:39.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa43792a00 0x55fa43806ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:39.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa4378d890 0x55fa4379b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:39.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 --2- 172.21.15.131:0/3017668187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa4378cea0 0x55fa4379af10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:39.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b62a00640 0 client.15849 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:21:39.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 >> 172.21.15.131:0/3017668187 conn(0x55fa43788510 msgr2=0x55fa4378bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:39.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 shutdown_connections 2024-04-26T20:21:39.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 1 -- 172.21.15.131:0/3017668187 wait complete. 2024-04-26T20:21:39.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.278+0000 7f9b829f1740 20 client.15849 trim_cache size 0 max 16384 2024-04-26T20:21:39.502 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:21:39.543 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:29 smithi148 ceph-mon[32281]: pgmap v754: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:39.550 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:31 smithi148 ceph-mon[32281]: pgmap v755: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:39.596 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:33 smithi148 ceph-mon[32281]: pgmap v756: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:39.617 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:35 smithi148 ceph-mon[32281]: pgmap v757: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:39.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:37 smithi148 ceph-mon[32281]: pgmap v758: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:39.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:29 smithi157 ceph-mon[32703]: pgmap v754: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:39.759 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:31 smithi157 ceph-mon[32703]: pgmap v755: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:39.925 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:33 smithi157 ceph-mon[32703]: pgmap v756: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:39.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:35 smithi157 ceph-mon[32703]: pgmap v757: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:37 smithi157 ceph-mon[32703]: pgmap v758: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:40.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b5c0a3c40 msgr2=0x7f9b5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:40.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b5c0a3c40 0x7f9b5c0a6100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9b68003fd0 tx=0x7f9b680d4650 comp rx=0 tx=0).stop 2024-04-26T20:21:40.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 msgr2=0x7f9b7819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:40.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 0x7f9b7819ae10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800e9f0 tx=0x7f9b6800eeb0 comp rx=0 tx=0).stop 2024-04-26T20:21:40.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 shutdown_connections 2024-04-26T20:21:40.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b78106760 0x7f9b781a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:40.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9b5c0a3c40 0x7f9b5c0a6100 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b78105da0 0x7f9b7819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:40.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 --2- 172.21.15.131:0/3310992220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b78104bb0 0x7f9b7819ae10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:40.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 >> 172.21.15.131:0/3310992220 conn(0x7f9b78100120 msgr2=0x7f9b781091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:40.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 shutdown_connections 2024-04-26T20:21:40.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:19:08.279+0000 7f9b7fc00640 1 -- 172.21.15.131:0/3310992220 wait complete. 2024-04-26T20:21:40.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T20:21:40.577 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:39 smithi148 ceph-mon[32281]: pgmap v759: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:21:40.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:39 smithi157 ceph-mon[32703]: pgmap v759: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:21:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.946+0000 7f0df8c00640 1 -- 172.21.15.131:0/745026627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4105db0 msgr2=0x7f0df4106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.946+0000 7f0df8c00640 1 --2- 172.21.15.131:0/745026627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4105db0 0x7f0df4106230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009900 tx=0x7f0de002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:21:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.947+0000 7f0df8c00640 1 -- 172.21.15.131:0/745026627 shutdown_connections 2024-04-26T20:21:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.947+0000 7f0df8c00640 1 --2- 172.21.15.131:0/745026627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0df4106770 0x7f0df410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.947+0000 7f0df8c00640 1 --2- 172.21.15.131:0/745026627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4105db0 0x7f0df4106230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.947+0000 7f0df8c00640 1 --2- 172.21.15.131:0/745026627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0df4104bc0 0x7f0df4104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.947+0000 7f0df8c00640 1 -- 172.21.15.131:0/745026627 >> 172.21.15.131:0/745026627 conn(0x7f0df4100150 msgr2=0x7f0df4102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.947+0000 7f0df8c00640 1 -- 172.21.15.131:0/745026627 shutdown_connections 2024-04-26T20:21:41.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.947+0000 7f0df8c00640 1 -- 172.21.15.131:0/745026627 wait complete. 2024-04-26T20:21:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.948+0000 7f0df8c00640 1 Processor -- start 2024-04-26T20:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.948+0000 7f0df8c00640 1 -- start start 2024-04-26T20:21:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.948+0000 7f0df8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4104bc0 0x7f0df419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0df4105db0 0x7f0df419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0df4106770 0x7f0df41a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df8c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4110720 con 0x7f0df4106770 2024-04-26T20:21:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41105a0 con 0x7f0df4105db0 2024-04-26T20:21:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41108a0 con 0x7f0df4104bc0 2024-04-26T20:21:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4104bc0 0x7f0df419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4104bc0 0x7f0df419ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37554/0 (socket says 172.21.15.131:37554) 2024-04-26T20:21:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 -- 172.21.15.131:0/432633044 learned_addr learned my addr 172.21.15.131:0/432633044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1400640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0df4105db0 0x7f0df419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df2800640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0df4106770 0x7f0df41a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 -- 172.21.15.131:0/432633044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0df4105db0 msgr2=0x7f0df419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0df4105db0 0x7f0df419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 -- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0df4106770 msgr2=0x7f0df41a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0df4106770 0x7f0df41a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.949+0000 7f0df1e00640 1 -- 172.21.15.131:0/432633044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df41a2960 con 0x7f0df4104bc0 2024-04-26T20:21:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.950+0000 7f0df1e00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4104bc0 0x7f0df419ad50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0de400e9c0 tx=0x7f0de400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.950+0000 7f0deaa00640 1 -- 172.21.15.131:0/432633044 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de400cd80 con 0x7f0df4104bc0 2024-04-26T20:21:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.950+0000 7f0deaa00640 1 -- 172.21.15.131:0/432633044 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de40035f0 con 0x7f0df4104bc0 2024-04-26T20:21:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.950+0000 7f0deaa00640 1 -- 172.21.15.131:0/432633044 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4010690 con 0x7f0df4104bc0 2024-04-26T20:21:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.950+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df41a2c50 con 0x7f0df4104bc0 2024-04-26T20:21:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.950+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df41a31a0 con 0x7f0df4104bc0 2024-04-26T20:21:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.953+0000 7f0dfbbef740 1 Processor -- start 2024-04-26T20:21:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.953+0000 7f0deaa00640 1 -- 172.21.15.131:0/432633044 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0de40037a0 con 0x7f0df4104bc0 2024-04-26T20:21:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0dfbbef740 1 -- start start 2024-04-26T20:21:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba345bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba2a2380 con 0x55c2ba33dbd0 2024-04-26T20:21:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba23ec30 con 0x55c2ba33e5c0 2024-04-26T20:21:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba29e340 con 0x55c2ba343730 2024-04-26T20:21:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0df2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0df1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba345bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0df1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba345bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37568/0 (socket says 172.21.15.131:37568) 2024-04-26T20:21:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.954+0000 7f0df1400640 1 -- 172.21.15.131:0/3320582233 learned_addr learned my addr 172.21.15.131:0/3320582233 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df1e00640 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df1400640 1 -- 172.21.15.131:0/3320582233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 msgr2=0x55c2ba3430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df1400640 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df1400640 1 -- 172.21.15.131:0/3320582233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 msgr2=0x55c2ba33dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df1400640 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df1400640 1 -- 172.21.15.131:0/3320582233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ba260940 con 0x55c2ba343730 2024-04-26T20:21:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df2800640 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0df1400640 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba345bc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006010 tx=0x7f0de0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ce2069742ee5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0deaa00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0dd80a3d70 0x7f0dd80a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0deaa00640 1 -- 172.21.15.131:0/432633044 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0de40d0d10 con 0x7f0df4104bc0 2024-04-26T20:21:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.955+0000 7f0ddf000640 1 -- 172.21.15.131:0/3320582233 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0035030 con 0x55c2ba343730 2024-04-26T20:21:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0df2800640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0dd80a3d70 0x7f0dd80a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0ddf000640 1 -- 172.21.15.131:0/3320582233 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de0040070 con 0x55c2ba343730 2024-04-26T20:21:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0ddf000640 1 -- 172.21.15.131:0/3320582233 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0031890 con 0x55c2ba343730 2024-04-26T20:21:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3320582233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 msgr2=0x55c2ba345bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba345bc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006010 tx=0x7f0de0004040 comp rx=0 tx=0).stop 2024-04-26T20:21:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0df2800640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0dd80a3d70 0x7f0dd80a6230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55c2ba33ee20 tx=0x7f0dec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3320582233 shutdown_connections 2024-04-26T20:21:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba345bc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3320582233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3320582233 >> 172.21.15.131:0/3320582233 conn(0x55c2ba339240 msgr2=0x55c2ba339640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.956+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3320582233 shutdown_connections 2024-04-26T20:21:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.957+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3320582233 wait complete. 2024-04-26T20:21:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.957+0000 7f0dfbbef740 1 -- start start 2024-04-26T20:21:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.957+0000 7f0dfbbef740 1 Processor -- start 2024-04-26T20:21:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33dbd0 0x55c2ba347d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba343730 0x55c2ba34f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba346680 con 0x55c2ba33e5c0 2024-04-26T20:21:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba229c60 con 0x55c2ba343730 2024-04-26T20:21:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba346800 con 0x55c2ba33dbd0 2024-04-26T20:21:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0dfbbef740 10 client.0 _ll_register_callbacks cb 0x55c2ba255e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33dbd0 0x55c2ba347d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38752/0 (socket says 172.21.15.131:38752) 2024-04-26T20:21:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 -- 172.21.15.131:0/1176111596 learned_addr learned my addr 172.21.15.131:0/1176111596 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:44.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df2800640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba343730 0x55c2ba34f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 -- 172.21.15.131:0/1176111596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33dbd0 msgr2=0x55c2ba347d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33dbd0 0x55c2ba347d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 -- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba343730 msgr2=0x55c2ba34f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba343730 0x55c2ba34f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1400640 1 -- 172.21.15.131:0/1176111596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ba238510 con 0x55c2ba33e5c0 2024-04-26T20:21:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df1e00640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33dbd0 0x55c2ba347d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.958+0000 7f0df2800640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba343730 0x55c2ba34f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0df1400640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3482c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0de002a9e0 tx=0x7f0de00044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d034f8c34c2b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0dd3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:21:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0dd3400640 1 -- 172.21.15.131:0/1176111596 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0053060 con 0x55c2ba33e5c0 2024-04-26T20:21:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0dd3400640 1 -- 172.21.15.131:0/1176111596 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0de0003a00 con 0x55c2ba33e5c0 2024-04-26T20:21:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0de0040070 con 0x55c2ba33e5c0 2024-04-26T20:21:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0dd3400640 1 -- 172.21.15.131:0/1176111596 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0de005a050 con 0x55c2ba33e5c0 2024-04-26T20:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0dfbbef740 10 client.15861 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.959+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2b9fb4210 con 0x55c2ba33e5c0 2024-04-26T20:21:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dd3400640 1 -- 172.21.15.131:0/1176111596 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de0040070 con 0x55c2ba33e5c0 2024-04-26T20:21:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dd3400640 1 -- 172.21.15.131:0/1176111596 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0de0045570 con 0x55c2ba33e5c0 2024-04-26T20:21:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dfbbef740 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dfbbef740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dfbbef740 20 client.15861 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3534b0 0x55c2ba3558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dfbbef740 4 client.15861 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:21:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2ba2aff90 con 0x55c2ba3534b0 2024-04-26T20:21:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.960+0000 7f0df1e00640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3534b0 0x55c2ba3558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.961+0000 7f0df1e00640 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3534b0 0x55c2ba3558a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4458fb97ca788cff server_cookie=cf7efc6175de3e03 in_seq=0 out_seq=0 2024-04-26T20:21:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.961+0000 7f0dd3400640 10 client.15861 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dd3400640 1 -- 172.21.15.131:0/1176111596 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2ba2aff90 con 0x55c2ba3534b0 2024-04-26T20:21:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dd3400640 10 client.15861 handle_command_reply: tid=0 2024-04-26T20:21:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 client.15861 shutdown 2024-04-26T20:21:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33e5c0 msgr2=0x55c2ba3482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3482c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0de002a9e0 tx=0x7f0de00044e0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 shutdown_connections 2024-04-26T20:21:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3534b0 0x55c2ba3558a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba343730 0x55c2ba34f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33e5c0 0x55c2ba3482c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dd3400640 0 client.15861 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/1176111596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33dbd0 0x55c2ba347d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 >> 172.21.15.131:0/1176111596 conn(0x55c2ba339240 msgr2=0x55c2ba338440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 shutdown_connections 2024-04-26T20:21:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 1 -- 172.21.15.131:0/1176111596 wait complete. 2024-04-26T20:21:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.962+0000 7f0dfbbef740 20 client.15861 trim_cache size 0 max 16384 2024-04-26T20:21:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.978+0000 7f0dfbbef740 1 Processor -- start 2024-04-26T20:21:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.978+0000 7f0dfbbef740 1 -- start start 2024-04-26T20:21:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33e5c0 0x55c2ba33fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba343730 0x55c2ba34f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41afff0 con 0x55c2ba343730 2024-04-26T20:21:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41ad8c0 con 0x55c2ba33dbd0 2024-04-26T20:21:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41ab710 con 0x55c2ba33e5c0 2024-04-26T20:21:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0df2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba343730 0x55c2ba34f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0df1e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0df2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba343730 0x55c2ba34f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38756/0 (socket says 172.21.15.131:38756) 2024-04-26T20:21:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0df2800640 1 -- 172.21.15.131:0/3553802279 learned_addr learned my addr 172.21.15.131:0/3553802279 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.979+0000 7f0df1400640 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33e5c0 0x55c2ba33fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df1e00640 1 -- 172.21.15.131:0/3553802279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33e5c0 msgr2=0x55c2ba33fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df1e00640 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33e5c0 0x55c2ba33fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df1e00640 1 -- 172.21.15.131:0/3553802279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba343730 msgr2=0x55c2ba34f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df1e00640 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba343730 0x55c2ba34f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df1e00640 1 -- 172.21.15.131:0/3553802279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ba080620 con 0x55c2ba33dbd0 2024-04-26T20:21:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df1400640 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33e5c0 0x55c2ba33fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:21:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df2800640 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba343730 0x55c2ba34f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0df1e00640 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33f7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0de40ce2d0 tx=0x7f0de40d3200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d79d5e13dbe048 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0dde600640 1 -- 172.21.15.131:0/3553802279 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4098cb0 con 0x55c2ba33dbd0 2024-04-26T20:21:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0dde600640 1 -- 172.21.15.131:0/3553802279 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de40d2070 con 0x55c2ba33dbd0 2024-04-26T20:21:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.980+0000 7f0dde600640 1 -- 172.21.15.131:0/3553802279 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de409abe0 con 0x55c2ba33dbd0 2024-04-26T20:21:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3553802279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33dbd0 msgr2=0x55c2ba33f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33f7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0de40ce2d0 tx=0x7f0de40d3200 comp rx=0 tx=0).stop 2024-04-26T20:21:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3553802279 shutdown_connections 2024-04-26T20:21:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba343730 0x55c2ba34f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba33e5c0 0x55c2ba33fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/3553802279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33dbd0 0x55c2ba33f7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3553802279 >> 172.21.15.131:0/3553802279 conn(0x55c2ba339240 msgr2=0x55c2ba347d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3553802279 shutdown_connections 2024-04-26T20:21:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.981+0000 7f0dfbbef740 1 -- 172.21.15.131:0/3553802279 wait complete. 2024-04-26T20:21:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 -- start start 2024-04-26T20:21:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 Processor -- start 2024-04-26T20:21:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba34c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba34ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba34d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41abd80 con 0x55c2ba33dbd0 2024-04-26T20:21:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41ad1f0 con 0x55c2ba33e5c0 2024-04-26T20:21:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41ac450 con 0x55c2ba343730 2024-04-26T20:21:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.982+0000 7f0dfbbef740 10 client.0 _ll_register_callbacks cb 0x7f0dec009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba34c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba34c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38764/0 (socket says 172.21.15.131:38764) 2024-04-26T20:21:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 -- 172.21.15.131:0/2139743667 learned_addr learned my addr 172.21.15.131:0/2139743667 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1400640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba34ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df2800640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba34d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 -- 172.21.15.131:0/2139743667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 msgr2=0x55c2ba34d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba34d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 -- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 msgr2=0x55c2ba34ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba34ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 -- 172.21.15.131:0/2139743667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ba25ccb0 con 0x55c2ba33dbd0 2024-04-26T20:21:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df2800640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba34d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:21:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1400640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba34ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0df1e00640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba34c4c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c6a0 tx=0x7f0de409af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5b69c74390171f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0dd3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:21:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.983+0000 7f0dd3400640 1 -- 172.21.15.131:0/2139743667 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de409c400 con 0x55c2ba33dbd0 2024-04-26T20:21:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dd3400640 1 -- 172.21.15.131:0/2139743667 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0de40a5280 con 0x55c2ba33dbd0 2024-04-26T20:21:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0de40d2070 con 0x55c2ba33dbd0 2024-04-26T20:21:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dd3400640 1 -- 172.21.15.131:0/2139743667 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0de409c5e0 con 0x55c2ba33dbd0 2024-04-26T20:21:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2ba34f0d0 con 0x55c2ba33dbd0 2024-04-26T20:21:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dd3400640 1 -- 172.21.15.131:0/2139743667 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de40d2070 con 0x55c2ba33dbd0 2024-04-26T20:21:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dd3400640 1 -- 172.21.15.131:0/2139743667 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0de40870e0 con 0x55c2ba33dbd0 2024-04-26T20:21:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 20 client.15867 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3b9ac0 0x55c2ba3bbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 4 client.15867 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:21:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.984+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2ba2a9e00 con 0x55c2ba3b9ac0 2024-04-26T20:21:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.985+0000 7f0df1400640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3b9ac0 0x55c2ba3bbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.985+0000 7f0df1400640 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3b9ac0 0x55c2ba3bbee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c44021c58f5676 server_cookie=ae474dea5d7e453e in_seq=0 out_seq=0 2024-04-26T20:21:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.985+0000 7f0dd3400640 10 client.15867 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dd3400640 1 -- 172.21.15.131:0/2139743667 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2ba2a9e00 con 0x55c2ba3b9ac0 2024-04-26T20:21:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dd3400640 10 client.15867 handle_command_reply: tid=0 2024-04-26T20:21:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 client.15867 shutdown 2024-04-26T20:21:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 msgr2=0x55c2ba34c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba34c4c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c6a0 tx=0x7f0de409af90 comp rx=0 tx=0).stop 2024-04-26T20:21:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 shutdown_connections 2024-04-26T20:21:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55c2ba3b9ac0 0x55c2ba3bbee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ba343730 0x55c2ba34d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba33e5c0 0x55c2ba34ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 --2- 172.21.15.131:0/2139743667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ba33dbd0 0x55c2ba34c4c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dd3400640 0 client.15867 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.989+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 >> 172.21.15.131:0/2139743667 conn(0x55c2ba339240 msgr2=0x55c2ba337bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 shutdown_connections 2024-04-26T20:21:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0dfbbef740 1 -- 172.21.15.131:0/2139743667 wait complete. 2024-04-26T20:21:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0dfbbef740 20 client.15867 trim_cache size 0 max 16384 2024-04-26T20:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:21:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:21:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:21:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:21:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0dd80a3d70 msgr2=0x7f0dd80a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0dd80a3d70 0x7f0dd80a6230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55c2ba33ee20 tx=0x7f0dec009500 comp rx=0 tx=0).stop 2024-04-26T20:21:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4104bc0 msgr2=0x7f0df419ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4104bc0 0x7f0df419ad50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0de400e9c0 tx=0x7f0de400ee80 comp rx=0 tx=0).stop 2024-04-26T20:21:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 shutdown_connections 2024-04-26T20:21:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0df4106770 0x7f0df41a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0df4105db0 0x7f0df419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0dd80a3d70 0x7f0dd80a6230 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 --2- 172.21.15.131:0/432633044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df4104bc0 0x7f0df419ad50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.990+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 >> 172.21.15.131:0/432633044 conn(0x7f0df4100150 msgr2=0x7f0df4109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.991+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 shutdown_connections 2024-04-26T20:21:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:09.991+0000 7f0df8c00640 1 -- 172.21.15.131:0/432633044 wait complete. 2024-04-26T20:21:50.292 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:09 smithi131 ceph-mon[27919]: pgmap v684: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:50.300 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:11 smithi131 ceph-mon[27919]: pgmap v685: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:50.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:12 smithi131 ceph-mon[27919]: pgmap v686: 129 pgs: 129 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:21:50.472 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:14 smithi131 ceph-mon[27919]: pgmap v687: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:50.479 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:16 smithi131 ceph-mon[27919]: pgmap v688: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:50.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:19 smithi131 ceph-mon[27919]: pgmap v689: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:50.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:21 smithi131 ceph-mon[27919]: pgmap v690: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:50.783 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:41 smithi157 ceph-mon[32703]: pgmap v760: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:50.848 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:43 smithi157 ceph-mon[32703]: pgmap v761: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:50.924 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:45 smithi157 ceph-mon[32703]: pgmap v762: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:50.931 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:47 smithi157 ceph-mon[32703]: pgmap v763: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:49 smithi157 ceph-mon[32703]: pgmap v764: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:50.999 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:41 smithi148 ceph-mon[32281]: pgmap v760: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:51.005 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:43 smithi148 ceph-mon[32281]: pgmap v761: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:51.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:45 smithi148 ceph-mon[32281]: pgmap v762: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:21:51.018 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:47 smithi148 ceph-mon[32281]: pgmap v763: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:21:51.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:49 smithi148 ceph-mon[32281]: pgmap v764: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:21:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 -- 172.21.15.131:0/3591414957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c10ab70 msgr2=0x7f3b3c111400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 --2- 172.21.15.131:0/3591414957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c10ab70 0x7f3b3c111400 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009ca0 tx=0x7f3b3402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 -- 172.21.15.131:0/3591414957 shutdown_connections 2024-04-26T20:21:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 --2- 172.21.15.131:0/3591414957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c10ab70 0x7f3b3c111400 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 --2- 172.21.15.131:0/3591414957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b3c10a1b0 0x7f3b3c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 --2- 172.21.15.131:0/3591414957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b3c108fc0 0x7f3b3c1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 -- 172.21.15.131:0/3591414957 >> 172.21.15.131:0/3591414957 conn(0x7f3b3c077f80 msgr2=0x7f3b3c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 -- 172.21.15.131:0/3591414957 shutdown_connections 2024-04-26T20:21:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.131+0000 7f3b42200640 1 -- 172.21.15.131:0/3591414957 wait complete. 2024-04-26T20:21:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 Processor -- start 2024-04-26T20:21:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 -- start start 2024-04-26T20:21:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c108fc0 0x7f3b3c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b3c10a1b0 0x7f3b3c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b3c10ab70 0x7f3b3c1a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c114ac0 con 0x7f3b3c108fc0 2024-04-26T20:21:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c114940 con 0x7f3b3c10a1b0 2024-04-26T20:21:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.132+0000 7f3b42200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c114c40 con 0x7f3b3c10ab70 2024-04-26T20:21:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c108fc0 0x7f3b3c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c108fc0 0x7f3b3c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52042/0 (socket says 172.21.15.131:52042) 2024-04-26T20:21:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 -- 172.21.15.131:0/1111019383 learned_addr learned my addr 172.21.15.131:0/1111019383 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b3c10a1b0 0x7f3b3c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3be00640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b3c10ab70 0x7f3b3c1a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 -- 172.21.15.131:0/1111019383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b3c10ab70 msgr2=0x7f3b3c1a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b3c10ab70 0x7f3b3c1a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 -- 172.21.15.131:0/1111019383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b3c10a1b0 msgr2=0x7f3b3c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b3c10a1b0 0x7f3b3c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 -- 172.21.15.131:0/1111019383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c1a6cc0 con 0x7f3b3c108fc0 2024-04-26T20:21:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b3c10a1b0 0x7f3b3c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.133+0000 7f3b3b400640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c108fc0 0x7f3b3c19f0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9c0 tx=0x7f3b2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.134+0000 7f3b33e00640 1 -- 172.21.15.131:0/1111019383 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00cd80 con 0x7f3b3c108fc0 2024-04-26T20:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.134+0000 7f3b33e00640 1 -- 172.21.15.131:0/1111019383 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2c0035f0 con 0x7f3b3c108fc0 2024-04-26T20:21:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.134+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3c1a6fb0 con 0x7f3b3c108fc0 2024-04-26T20:21:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.134+0000 7f3b33e00640 1 -- 172.21.15.131:0/1111019383 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c010690 con 0x7f3b3c108fc0 2024-04-26T20:21:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.134+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3c1a7410 con 0x7f3b3c108fc0 2024-04-26T20:21:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.137+0000 7f3b4518d740 1 Processor -- start 2024-04-26T20:21:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.137+0000 7f3b33e00640 1 -- 172.21.15.131:0/1111019383 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3b2c0037a0 con 0x7f3b3c108fc0 2024-04-26T20:21:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.137+0000 7f3b4518d740 1 -- start start 2024-04-26T20:21:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.137+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a941304ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.137+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.137+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a94130cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b4518d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a941269380 con 0x55a94130a730 2024-04-26T20:21:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b4518d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a941205c30 con 0x55a9413055c0 2024-04-26T20:21:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b4518d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a941265340 con 0x55a941304bd0 2024-04-26T20:21:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a94130cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a941304ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a94130cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52058/0 (socket says 172.21.15.131:52058) 2024-04-26T20:21:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3aa00640 1 -- 172.21.15.131:0/2357979012 learned_addr learned my addr 172.21.15.131:0/2357979012 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3be00640 1 -- 172.21.15.131:0/2357979012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 msgr2=0x55a94130a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3be00640 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3be00640 1 -- 172.21.15.131:0/2357979012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 msgr2=0x55a94130cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3be00640 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a94130cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3be00640 1 -- 172.21.15.131:0/2357979012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a941227940 con 0x55a941304bd0 2024-04-26T20:21:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.138+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a94130cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.139+0000 7f3b3be00640 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a941304ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b34005e30 tx=0x7f3b34005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59d01de3759db3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.139+0000 7f3b33e00640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3b200a3cf0 0x7f3b200a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.139+0000 7f3b27e00640 1 -- 172.21.15.131:0/2357979012 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34031650 con 0x55a941304bd0 2024-04-26T20:21:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.139+0000 7f3b33e00640 1 -- 172.21.15.131:0/1111019383 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3b2c0d0d40 con 0x7f3b3c108fc0 2024-04-26T20:21:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.139+0000 7f3b27e00640 1 -- 172.21.15.131:0/2357979012 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b34036030 con 0x55a941304bd0 2024-04-26T20:21:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.139+0000 7f3b3b400640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3b200a3cf0 0x7f3b200a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.139+0000 7f3b27e00640 1 -- 172.21.15.131:0/2357979012 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340047c0 con 0x55a941304bd0 2024-04-26T20:21:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 -- 172.21.15.131:0/2357979012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 msgr2=0x55a941304ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a941304ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b34005e30 tx=0x7f3b34005ef0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b3b400640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3b200a3cf0 0x7f3b200a61b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a9413059c0 tx=0x7f3b2c0d47f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 -- 172.21.15.131:0/2357979012 shutdown_connections 2024-04-26T20:21:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a94130cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 --2- 172.21.15.131:0/2357979012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a941304ff0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 -- 172.21.15.131:0/2357979012 >> 172.21.15.131:0/2357979012 conn(0x55a941300240 msgr2=0x55a941300640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 -- 172.21.15.131:0/2357979012 shutdown_connections 2024-04-26T20:21:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.140+0000 7f3b4518d740 1 -- 172.21.15.131:0/2357979012 wait complete. 2024-04-26T20:21:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 -- start start 2024-04-26T20:21:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 Processor -- start 2024-04-26T20:21:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a94130ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a94130d500 con 0x55a94130a730 2024-04-26T20:21:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a9411f0c60 con 0x55a9413055c0 2024-04-26T20:21:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a94130d680 con 0x55a941304bd0 2024-04-26T20:21:55.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.141+0000 7f3b4518d740 10 client.0 _ll_register_callbacks cb 0x55a94121ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a94130ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a94130ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43814/0 (socket says 172.21.15.131:43814) 2024-04-26T20:21:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3be00640 1 -- 172.21.15.131:0/3895508232 learned_addr learned my addr 172.21.15.131:0/3895508232 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3be00640 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3aa00640 1 -- 172.21.15.131:0/3895508232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 msgr2=0x55a94130ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a94130ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3aa00640 1 -- 172.21.15.131:0/3895508232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 msgr2=0x55a9413161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3aa00640 1 -- 172.21.15.131:0/3895508232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9411ff510 con 0x55a9413055c0 2024-04-26T20:21:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130f180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b280129c0 tx=0x7f3b28012e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83e43b3dea81c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.142+0000 7f3b24c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:21:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.143+0000 7f3b24c00640 1 -- 172.21.15.131:0/3895508232 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800fa40 con 0x55a9413055c0 2024-04-26T20:21:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.143+0000 7f3b24c00640 1 -- 172.21.15.131:0/3895508232 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3b2800fbe0 con 0x55a9413055c0 2024-04-26T20:21:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.143+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b34036030 con 0x55a9413055c0 2024-04-26T20:21:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.143+0000 7f3b24c00640 1 -- 172.21.15.131:0/3895508232 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b28025050 con 0x55a9413055c0 2024-04-26T20:21:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.143+0000 7f3b4518d740 10 client.26000 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.143+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a940f7b210 con 0x55a9413055c0 2024-04-26T20:21:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.143+0000 7f3b24c00640 1 -- 172.21.15.131:0/3895508232 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b34036030 con 0x55a9413055c0 2024-04-26T20:21:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b24c00640 1 -- 172.21.15.131:0/3895508232 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3b28003ae0 con 0x55a9413055c0 2024-04-26T20:21:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b4518d740 10 client.26000 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b4518d740 10 client.26000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b4518d740 20 client.26000 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b4518d740 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a94131a700 0x55a94131caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b4518d740 4 client.26000 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:21:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a941276f90 con 0x55a94131a700 2024-04-26T20:21:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.144+0000 7f3b3be00640 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a94131a700 0x55a94131caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.145+0000 7f3b3be00640 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a94131a700 0x55a94131caf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91761e9dfdd2146c server_cookie=8bfa5ce293e91c19 in_seq=0 out_seq=0 2024-04-26T20:21:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.145+0000 7f3b24c00640 10 client.26000 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.145+0000 7f3b24c00640 1 -- 172.21.15.131:0/3895508232 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a941276f90 con 0x55a94131a700 2024-04-26T20:21:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.145+0000 7f3b24c00640 10 client.26000 handle_command_reply: tid=0 2024-04-26T20:21:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 client.26000 shutdown 2024-04-26T20:21:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 msgr2=0x55a94130f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130f180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b280129c0 tx=0x7f3b28012e80 comp rx=0 tx=0).stop 2024-04-26T20:21:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 shutdown_connections 2024-04-26T20:21:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a94131a700 0x55a94131caf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a9413055c0 0x55a94130f180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b24c00640 0 client.26000 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 --2- 172.21.15.131:0/3895508232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a94130ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 >> 172.21.15.131:0/3895508232 conn(0x55a941300240 msgr2=0x55a9412ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 shutdown_connections 2024-04-26T20:21:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 1 -- 172.21.15.131:0/3895508232 wait complete. 2024-04-26T20:21:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.146+0000 7f3b4518d740 20 client.26000 trim_cache size 0 max 16384 2024-04-26T20:21:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.162+0000 7f3b4518d740 1 Processor -- start 2024-04-26T20:21:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.162+0000 7f3b4518d740 1 -- start start 2024-04-26T20:21:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.162+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a9413067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9413055c0 0x55a941306d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a94130a730 0x55a941316180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b4518d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b43f0 con 0x55a9413055c0 2024-04-26T20:21:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b4518d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b1cc0 con 0x55a94130a730 2024-04-26T20:21:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b4518d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1afb10 con 0x55a941304bd0 2024-04-26T20:21:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a9413067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a9413067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43824/0 (socket says 172.21.15.131:43824) 2024-04-26T20:21:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3be00640 1 -- 172.21.15.131:0/1522397697 learned_addr learned my addr 172.21.15.131:0/1522397697 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9413055c0 0x55a941306d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3be00640 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a94130a730 0x55a941316180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3aa00640 1 -- 172.21.15.131:0/1522397697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 msgr2=0x55a9413067e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a9413067e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3aa00640 1 -- 172.21.15.131:0/1522397697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a94130a730 msgr2=0x55a941316180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a94130a730 0x55a941316180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3aa00640 1 -- 172.21.15.131:0/1522397697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94131ce60 con 0x55a9413055c0 2024-04-26T20:21:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.163+0000 7f3b3be00640 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a9413067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:21:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.164+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9413055c0 0x55a941306d20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3b2801c4b0 tx=0x7f3b28012600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb72784f62aad6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.164+0000 7f3b27400640 1 -- 172.21.15.131:0/1522397697 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2802e070 con 0x55a9413055c0 2024-04-26T20:21:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.164+0000 7f3b27400640 1 -- 172.21.15.131:0/1522397697 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2801c530 con 0x55a9413055c0 2024-04-26T20:21:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.164+0000 7f3b27400640 1 -- 172.21.15.131:0/1522397697 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2801cc60 con 0x55a9413055c0 2024-04-26T20:21:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 -- 172.21.15.131:0/1522397697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9413055c0 msgr2=0x55a941306d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:21:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9413055c0 0x55a941306d20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3b2801c4b0 tx=0x7f3b28012600 comp rx=0 tx=0).stop 2024-04-26T20:21:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 -- 172.21.15.131:0/1522397697 shutdown_connections 2024-04-26T20:21:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a94130a730 0x55a941316180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9413055c0 0x55a941306d20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 --2- 172.21.15.131:0/1522397697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a941304bd0 0x55a9413067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 -- 172.21.15.131:0/1522397697 >> 172.21.15.131:0/1522397697 conn(0x55a941300240 msgr2=0x55a94130ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:21:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 -- 172.21.15.131:0/1522397697 shutdown_connections 2024-04-26T20:21:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 -- 172.21.15.131:0/1522397697 wait complete. 2024-04-26T20:21:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 -- start start 2024-04-26T20:21:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.165+0000 7f3b4518d740 1 Processor -- start 2024-04-26T20:21:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a941304bd0 0x55a941313190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9413055c0 0x55a9413136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b4518d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b4518d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b0180 con 0x55a94130a730 2024-04-26T20:21:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b4518d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b15f0 con 0x55a941304bd0 2024-04-26T20:21:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b4518d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b0850 con 0x55a9413055c0 2024-04-26T20:21:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b4518d740 10 client.0 _ll_register_callbacks cb 0x7f3b28009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:21:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52082/0 (socket says 172.21.15.131:52082) 2024-04-26T20:21:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b3be00640 1 -- 172.21.15.131:0/131907712 learned_addr learned my addr 172.21.15.131:0/131907712 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:21:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b3be00640 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a941304bd0 0x55a941313190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.166+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9413055c0 0x55a9413136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b3be00640 1 -- 172.21.15.131:0/131907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9413055c0 msgr2=0x55a9413136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b3be00640 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9413055c0 0x55a9413136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b3be00640 1 -- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a941304bd0 msgr2=0x55a941313190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:21:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b3be00640 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a941304bd0 0x55a941313190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:21:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b3be00640 1 -- 172.21.15.131:0/131907712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a941047620 con 0x55a94130a730 2024-04-26T20:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b3be00640 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413146e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b34031a20 tx=0x7f3b28012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eb0d4f95c3342a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:21:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b24c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:21:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b24c00640 1 -- 172.21.15.131:0/131907712 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34031450 con 0x55a94130a730 2024-04-26T20:21:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b24c00640 1 -- 172.21.15.131:0/131907712 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3b3404a040 con 0x55a94130a730 2024-04-26T20:21:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2801c530 con 0x55a94130a730 2024-04-26T20:21:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b24c00640 1 -- 172.21.15.131:0/131907712 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b340315f0 con 0x55a94130a730 2024-04-26T20:21:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b4518d740 10 client.15885 fetch_fsmap learned FSMap version 27 2024-04-26T20:21:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.167+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a941223cb0 con 0x55a94130a730 2024-04-26T20:21:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b24c00640 1 -- 172.21.15.131:0/131907712 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2801c530 con 0x55a94130a730 2024-04-26T20:21:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b24c00640 1 -- 172.21.15.131:0/131907712 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3b34043070 con 0x55a94130a730 2024-04-26T20:21:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b4518d740 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:21:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b4518d740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:21:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b4518d740 20 client.15885 populate_metadata read hostname 'smithi131' 2024-04-26T20:21:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b4518d740 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a941380c70 0x55a941383060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:21:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b4518d740 4 client.15885 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:21:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a941270e00 con 0x55a941380c70 2024-04-26T20:21:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.168+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a941380c70 0x55a941383060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:21:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.169+0000 7f3b24c00640 10 client.15885 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:21:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.169+0000 7f3b3aa00640 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a941380c70 0x55a941383060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286ce66fe9669924 server_cookie=afd1e994bd3743f6 in_seq=0 out_seq=0 2024-04-26T20:22:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b24c00640 1 -- 172.21.15.131:0/131907712 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a941270e00 con 0x55a941380c70 2024-04-26T20:22:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b24c00640 10 client.15885 handle_command_reply: tid=0 2024-04-26T20:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 client.15885 shutdown 2024-04-26T20:22:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 msgr2=0x55a9413146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413146e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b34031a20 tx=0x7f3b28012590 comp rx=0 tx=0).stop 2024-04-26T20:22:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 shutdown_connections 2024-04-26T20:22:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a941380c70 0x55a941383060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94130a730 0x55a9413146e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9413055c0 0x55a9413136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 --2- 172.21.15.131:0/131907712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a941304bd0 0x55a941313190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b24c00640 0 client.15885 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 >> 172.21.15.131:0/131907712 conn(0x55a941300240 msgr2=0x55a94130ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 shutdown_connections 2024-04-26T20:22:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 1 -- 172.21.15.131:0/131907712 wait complete. 2024-04-26T20:22:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.173+0000 7f3b4518d740 20 client.15885 trim_cache size 0 max 16384 2024-04-26T20:22:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:22:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:22:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:22:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:22:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3b200a3cf0 msgr2=0x7f3b200a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3b200a3cf0 0x7f3b200a61b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a9413059c0 tx=0x7f3b2c0d47f0 comp rx=0 tx=0).stop 2024-04-26T20:22:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c108fc0 msgr2=0x7f3b3c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c108fc0 0x7f3b3c19f0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9c0 tx=0x7f3b2c00ee80 comp rx=0 tx=0).stop 2024-04-26T20:22:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 shutdown_connections 2024-04-26T20:22:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3b200a3cf0 0x7f3b200a61b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b3c10ab70 0x7f3b3c1a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b3c10a1b0 0x7f3b3c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 --2- 172.21.15.131:0/1111019383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c108fc0 0x7f3b3c19f0a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 >> 172.21.15.131:0/1111019383 conn(0x7f3b3c077f80 msgr2=0x7f3b3c10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 shutdown_connections 2024-04-26T20:22:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:23.174+0000 7f3b42200640 1 -- 172.21.15.131:0/1111019383 wait complete. 2024-04-26T20:22:01.530 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:23 smithi131 ceph-mon[27919]: pgmap v691: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:01.633 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:25 smithi131 ceph-mon[27919]: pgmap v692: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:01.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:27 smithi131 ceph-mon[27919]: pgmap v693: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:22:01.735 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:29 smithi131 ceph-mon[27919]: pgmap v694: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:22:01.742 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:31 smithi131 ceph-mon[27919]: pgmap v695: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:01.749 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:33 smithi131 ceph-mon[27919]: pgmap v696: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:01.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:35 smithi131 ceph-mon[27919]: pgmap v697: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:01.766 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:51 smithi157 ceph-mon[32703]: pgmap v765: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:01.797 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:52 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:01.898 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:53 smithi157 ceph-mon[32703]: pgmap v766: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:22:01.930 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:01.996 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:02.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:02.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:55 smithi157 ceph-mon[32703]: pgmap v767: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:02.065 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:56 smithi157 ceph-mon[32703]: pgmap v768: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:02.182 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:21:59 smithi157 ceph-mon[32703]: pgmap v769: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:02.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:01 smithi157 ceph-mon[32703]: pgmap v770: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:02.304 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:51 smithi148 ceph-mon[32281]: pgmap v765: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:02.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:52 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:02.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:53 smithi148 ceph-mon[32281]: pgmap v766: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:22:02.423 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:02.429 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:02.436 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:02.510 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:55 smithi148 ceph-mon[32281]: pgmap v767: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:02.520 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:56 smithi148 ceph-mon[32281]: pgmap v768: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:02.533 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:21:59 smithi148 ceph-mon[32281]: pgmap v769: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:02.540 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:01 smithi148 ceph-mon[32281]: pgmap v770: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 -- 172.21.15.131:0/848409775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10f040 msgr2=0x7ffa5c111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 --2- 172.21.15.131:0/848409775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10f040 0x7ffa5c111500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005fd0 tx=0x7ffa5402b3e0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 -- 172.21.15.131:0/848409775 shutdown_connections 2024-04-26T20:22:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 --2- 172.21.15.131:0/848409775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10f040 0x7ffa5c111500 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 --2- 172.21.15.131:0/848409775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa5c10c560 0x7ffa5c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 --2- 172.21.15.131:0/848409775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 0x7ffa5c10bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 -- 172.21.15.131:0/848409775 >> 172.21.15.131:0/848409775 conn(0x7ffa5c06fb60 msgr2=0x7ffa5c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 -- 172.21.15.131:0/848409775 shutdown_connections 2024-04-26T20:22:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.315+0000 7ffa60c00640 1 -- 172.21.15.131:0/848409775 wait complete. 2024-04-26T20:22:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.316+0000 7ffa60c00640 1 Processor -- start 2024-04-26T20:22:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa60c00640 1 -- start start 2024-04-26T20:22:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa60c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 0x7ffa5c10bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa60c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10c560 0x7ffa5c10a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa60c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa5c10f040 0x7ffa5c10a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa60c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c114ad0 con 0x7ffa5c109ba0 2024-04-26T20:22:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa60c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c114950 con 0x7ffa5c10f040 2024-04-26T20:22:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa60c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c114c50 con 0x7ffa5c10c560 2024-04-26T20:22:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa59e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 0x7ffa5c10bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa59e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 0x7ffa5c10bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42530/0 (socket says 172.21.15.131:42530) 2024-04-26T20:22:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa59e00640 1 -- 172.21.15.131:0/1319121982 learned_addr learned my addr 172.21.15.131:0/1319121982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa5a800640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa5c10f040 0x7ffa5c10a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.317+0000 7ffa59400640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10c560 0x7ffa5c10a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa59e00640 1 -- 172.21.15.131:0/1319121982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10c560 msgr2=0x7ffa5c10a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa59e00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10c560 0x7ffa5c10a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa59e00640 1 -- 172.21.15.131:0/1319121982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa5c10f040 msgr2=0x7ffa5c10a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa59e00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa5c10f040 0x7ffa5c10a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa59e00640 1 -- 172.21.15.131:0/1319121982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c10af10 con 0x7ffa5c109ba0 2024-04-26T20:22:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa5a800640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa5c10f040 0x7ffa5c10a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa59e00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 0x7ffa5c10bb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9c0 tx=0x7ffa4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa52a00640 1 -- 172.21.15.131:0/1319121982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00cd80 con 0x7ffa5c109ba0 2024-04-26T20:22:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa52a00640 1 -- 172.21.15.131:0/1319121982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4c0054d0 con 0x7ffa5c109ba0 2024-04-26T20:22:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa52a00640 1 -- 172.21.15.131:0/1319121982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c010430 con 0x7ffa5c109ba0 2024-04-26T20:22:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.318+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1ab420 con 0x7ffa5c109ba0 2024-04-26T20:22:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.319+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1ab970 con 0x7ffa5c109ba0 2024-04-26T20:22:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 Processor -- start 2024-04-26T20:22:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 -- start start 2024-04-26T20:22:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa52a00640 1 -- 172.21.15.131:0/1319121982 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ffa4c004fd0 con 0x7ffa5c109ba0 2024-04-26T20:22:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af84150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af841560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af84158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55af840b5380 con 0x55af84150bd0 2024-04-26T20:22:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55af84051c30 con 0x55af841515c0 2024-04-26T20:22:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa6398d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55af840b1340 con 0x55af84156730 2024-04-26T20:22:04.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.322+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af84150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af84150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42538/0 (socket says 172.21.15.131:42538) 2024-04-26T20:22:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 -- 172.21.15.131:0/274511314 learned_addr learned my addr 172.21.15.131:0/274511314 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa59e00640 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af841560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa59400640 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af84158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 -- 172.21.15.131:0/274511314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 msgr2=0x55af84158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af84158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 -- 172.21.15.131:0/274511314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 msgr2=0x55af841560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af841560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 -- 172.21.15.131:0/274511314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af84073940 con 0x55af84150bd0 2024-04-26T20:22:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.323+0000 7ffa5a800640 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af84150ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa54032780 tx=0x7ffa54032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe96e49fce9fcd88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.324+0000 7ffa52a00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ffa400a3cf0 0x7ffa400a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.324+0000 7ffa47000640 1 -- 172.21.15.131:0/274511314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54037040 con 0x55af84150bd0 2024-04-26T20:22:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.324+0000 7ffa47000640 1 -- 172.21.15.131:0/274511314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa54007410 con 0x55af84150bd0 2024-04-26T20:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.324+0000 7ffa47000640 1 -- 172.21.15.131:0/274511314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400a580 con 0x55af84150bd0 2024-04-26T20:22:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.324+0000 7ffa52a00640 1 -- 172.21.15.131:0/1319121982 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ffa4c0d0da0 con 0x7ffa5c109ba0 2024-04-26T20:22:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.324+0000 7ffa59e00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ffa400a3cf0 0x7ffa400a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 -- 172.21.15.131:0/274511314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 msgr2=0x55af84150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa59e00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ffa400a3cf0 0x7ffa400a61b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55af84152030 tx=0x7ffa4c0d48f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af84150ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa54032780 tx=0x7ffa54032590 comp rx=0 tx=0).stop 2024-04-26T20:22:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 -- 172.21.15.131:0/274511314 shutdown_connections 2024-04-26T20:22:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af84158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af841560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 --2- 172.21.15.131:0/274511314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af84150ff0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 -- 172.21.15.131:0/274511314 >> 172.21.15.131:0/274511314 conn(0x55af8414c240 msgr2=0x55af8414c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 -- 172.21.15.131:0/274511314 shutdown_connections 2024-04-26T20:22:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.325+0000 7ffa6398d740 1 -- 172.21.15.131:0/274511314 wait complete. 2024-04-26T20:22:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 -- start start 2024-04-26T20:22:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 Processor -- start 2024-04-26T20:22:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af8415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af8415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55af84159680 con 0x55af84150bd0 2024-04-26T20:22:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55af8403cc60 con 0x55af841515c0 2024-04-26T20:22:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55af84159800 con 0x55af84156730 2024-04-26T20:22:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.326+0000 7ffa6398d740 10 client.0 _ll_register_callbacks cb 0x55af84068e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af8415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa59400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af8415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af8415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42544/0 (socket says 172.21.15.131:42544) 2024-04-26T20:22:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa5a800640 1 -- 172.21.15.131:0/2017395987 learned_addr learned my addr 172.21.15.131:0/2017395987 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa5a800640 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa59400640 1 -- 172.21.15.131:0/2017395987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 msgr2=0x55af841622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa59400640 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa59400640 1 -- 172.21.15.131:0/2017395987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 msgr2=0x55af8415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa59400640 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af8415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa59400640 1 -- 172.21.15.131:0/2017395987 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8404b510 con 0x55af841515c0 2024-04-26T20:22:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa5a800640 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af8415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa59400640 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af8415b2d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c10b870 tx=0x7ffa48012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c031b8df001687f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.327+0000 7ffa3b400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:22:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.328+0000 7ffa3b400640 1 -- 172.21.15.131:0/2017395987 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800f890 con 0x55af841515c0 2024-04-26T20:22:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.328+0000 7ffa3b400640 1 -- 172.21.15.131:0/2017395987 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ffa4800fa30 con 0x55af841515c0 2024-04-26T20:22:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.328+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa54007410 con 0x55af841515c0 2024-04-26T20:22:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.328+0000 7ffa3b400640 1 -- 172.21.15.131:0/2017395987 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa4800fc30 con 0x55af841515c0 2024-04-26T20:22:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.328+0000 7ffa6398d740 10 client.26018 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.328+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af83dc7210 con 0x55af841515c0 2024-04-26T20:22:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.328+0000 7ffa3b400640 1 -- 172.21.15.131:0/2017395987 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa54007410 con 0x55af841515c0 2024-04-26T20:22:06.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa3b400640 1 -- 172.21.15.131:0/2017395987 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffa4801c420 con 0x55af841515c0 2024-04-26T20:22:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa6398d740 10 client.26018 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa6398d740 10 client.26018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa6398d740 20 client.26018 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841667b0 0x55af84168c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa6398d740 4 client.26018 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af840c2f90 con 0x55af841667b0 2024-04-26T20:22:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa5a800640 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841667b0 0x55af84168c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.330+0000 7ffa3b400640 10 client.26018 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.329+0000 7ffa5a800640 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841667b0 0x55af84168c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8571de39d7a3fd8b server_cookie=f3efa900a6c5e7ff in_seq=0 out_seq=0 2024-04-26T20:22:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.330+0000 7ffa3b400640 1 -- 172.21.15.131:0/2017395987 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af840c2f90 con 0x55af841667b0 2024-04-26T20:22:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.330+0000 7ffa3b400640 10 client.26018 handle_command_reply: tid=0 2024-04-26T20:22:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.330+0000 7ffa6398d740 1 client.26018 shutdown 2024-04-26T20:22:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.330+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 msgr2=0x55af8415b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.330+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af8415b2d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c10b870 tx=0x7ffa48012b60 comp rx=0 tx=0).stop 2024-04-26T20:22:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 shutdown_connections 2024-04-26T20:22:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841667b0 0x55af84168c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af841515c0 0x55af8415b2d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa3b400640 0 client.26018 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2017395987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af84150bd0 0x55af8415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 >> 172.21.15.131:0/2017395987 conn(0x55af8414c240 msgr2=0x55af8414b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 shutdown_connections 2024-04-26T20:22:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 1 -- 172.21.15.131:0/2017395987 wait complete. 2024-04-26T20:22:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.331+0000 7ffa6398d740 20 client.26018 trim_cache size 0 max 16384 2024-04-26T20:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.347+0000 7ffa6398d740 1 Processor -- start 2024-04-26T20:22:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.347+0000 7ffa6398d740 1 -- start start 2024-04-26T20:22:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8414c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af84162260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af8415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa6398d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b8860 con 0x55af841515c0 2024-04-26T20:22:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa6398d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b6130 con 0x55af84150bd0 2024-04-26T20:22:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa6398d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b3f80 con 0x55af84156730 2024-04-26T20:22:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af84162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af84162260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42560/0 (socket says 172.21.15.131:42560) 2024-04-26T20:22:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 -- 172.21.15.131:0/2043661769 learned_addr learned my addr 172.21.15.131:0/2043661769 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa5a800640 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8414c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa5a800640 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af8415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 -- 172.21.15.131:0/2043661769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 msgr2=0x55af8415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af8415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 -- 172.21.15.131:0/2043661769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 msgr2=0x55af8414c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8414c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 -- 172.21.15.131:0/2043661769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af83e93620 con 0x55af841515c0 2024-04-26T20:22:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa5a800640 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af8415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:22:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.348+0000 7ffa59400640 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af84162260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa48000c00 tx=0x7ffa480096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9670e803435f415f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa46600640 1 -- 172.21.15.131:0/2043661769 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4801cb50 con 0x55af841515c0 2024-04-26T20:22:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa46600640 1 -- 172.21.15.131:0/2043661769 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4801ccf0 con 0x55af841515c0 2024-04-26T20:22:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa46600640 1 -- 172.21.15.131:0/2043661769 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa480035f0 con 0x55af841515c0 2024-04-26T20:22:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa6398d740 1 -- 172.21.15.131:0/2043661769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 msgr2=0x55af84162260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af84162260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa48000c00 tx=0x7ffa480096c0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa6398d740 1 -- 172.21.15.131:0/2043661769 shutdown_connections 2024-04-26T20:22:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af8415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af84162260 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa6398d740 1 --2- 172.21.15.131:0/2043661769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8414c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.349+0000 7ffa6398d740 1 -- 172.21.15.131:0/2043661769 >> 172.21.15.131:0/2043661769 conn(0x55af8414c240 msgr2=0x55af84155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.350+0000 7ffa6398d740 1 -- 172.21.15.131:0/2043661769 shutdown_connections 2024-04-26T20:22:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.350+0000 7ffa6398d740 1 -- 172.21.15.131:0/2043661769 wait complete. 2024-04-26T20:22:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.350+0000 7ffa6398d740 1 -- start start 2024-04-26T20:22:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.350+0000 7ffa6398d740 1 Processor -- start 2024-04-26T20:22:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.350+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8415f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af8415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa6398d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa6398d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b45f0 con 0x55af841515c0 2024-04-26T20:22:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa6398d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b5a60 con 0x55af84150bd0 2024-04-26T20:22:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa6398d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b4cc0 con 0x55af84156730 2024-04-26T20:22:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa6398d740 10 client.0 _ll_register_callbacks cb 0x7ffa48009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af8415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af8415f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42566/0 (socket says 172.21.15.131:42566) 2024-04-26T20:22:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 -- 172.21.15.131:0/1813425518 learned_addr learned my addr 172.21.15.131:0/1813425518 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa5a800640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa5a800640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 -- 172.21.15.131:0/1813425518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 msgr2=0x55af841cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 -- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 msgr2=0x55af8415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8415f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 -- 172.21.15.131:0/1813425518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8406fcb0 con 0x55af841515c0 2024-04-26T20:22:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa5a800640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa59400640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af8415f5c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800fac0 tx=0x7ffa4800faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e5fca186c50397 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.351+0000 7ffa3b400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:22:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.352+0000 7ffa3b400640 1 -- 172.21.15.131:0/1813425518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48038060 con 0x55af841515c0 2024-04-26T20:22:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.352+0000 7ffa3b400640 1 -- 172.21.15.131:0/1813425518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ffa48005320 con 0x55af841515c0 2024-04-26T20:22:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.352+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4801ccf0 con 0x55af841515c0 2024-04-26T20:22:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.352+0000 7ffa3b400640 1 -- 172.21.15.131:0/1813425518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa48005520 con 0x55af841515c0 2024-04-26T20:22:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.352+0000 7ffa6398d740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.352+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af841cc230 con 0x55af841515c0 2024-04-26T20:22:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.352+0000 7ffa3b400640 1 -- 172.21.15.131:0/1813425518 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4801ccf0 con 0x55af841515c0 2024-04-26T20:22:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa3b400640 1 -- 172.21.15.131:0/1813425518 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffa4802a550 con 0x55af841515c0 2024-04-26T20:22:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa6398d740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa6398d740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa6398d740 20 client.15903 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa6398d740 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841cca70 0x55af841cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa6398d740 4 client.15903 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:22:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af840bce00 con 0x55af841cca70 2024-04-26T20:22:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa5a800640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841cca70 0x55af841cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa5a800640 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841cca70 0x55af841cee90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4146416051c44f4d server_cookie=65a3304c62e48a77 in_seq=0 out_seq=0 2024-04-26T20:22:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.353+0000 7ffa3b400640 10 client.15903 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa3b400640 1 -- 172.21.15.131:0/1813425518 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af840bce00 con 0x55af841cca70 2024-04-26T20:22:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa3b400640 10 client.15903 handle_command_reply: tid=0 2024-04-26T20:22:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 client.15903 shutdown 2024-04-26T20:22:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 msgr2=0x55af8415f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af8415f5c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800fac0 tx=0x7ffa4800faf0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 shutdown_connections 2024-04-26T20:22:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55af841cca70 0x55af841cee90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55af84156730 0x55af841cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af841515c0 0x55af8415f5c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 --2- 172.21.15.131:0/1813425518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af84150bd0 0x55af8415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 >> 172.21.15.131:0/1813425518 conn(0x55af8414c240 msgr2=0x55af84152840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.357+0000 7ffa3b400640 0 client.15903 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.358+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 shutdown_connections 2024-04-26T20:22:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.358+0000 7ffa6398d740 1 -- 172.21.15.131:0/1813425518 wait complete. 2024-04-26T20:22:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.358+0000 7ffa6398d740 20 client.15903 trim_cache size 0 max 16384 2024-04-26T20:22:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:22:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:22:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:22:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:22:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.358+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ffa400a3cf0 msgr2=0x7ffa400a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.358+0000 7ffa60c00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ffa400a3cf0 0x7ffa400a61b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55af84152030 tx=0x7ffa4c0d48f0 comp rx=0 tx=0).stop 2024-04-26T20:22:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.358+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 msgr2=0x7ffa5c10bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.358+0000 7ffa60c00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 0x7ffa5c10bb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9c0 tx=0x7ffa4c00ee80 comp rx=0 tx=0).stop 2024-04-26T20:22:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 shutdown_connections 2024-04-26T20:22:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa5c10f040 0x7ffa5c10a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ffa400a3cf0 0x7ffa400a61b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa5c10c560 0x7ffa5c10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 --2- 172.21.15.131:0/1319121982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa5c109ba0 0x7ffa5c10bb00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 >> 172.21.15.131:0/1319121982 conn(0x7ffa5c06fb60 msgr2=0x7ffa5c071f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 shutdown_connections 2024-04-26T20:22:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:36.359+0000 7ffa60c00640 1 -- 172.21.15.131:0/1319121982 wait complete. 2024-04-26T20:22:12.442 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:37 smithi131 ceph-mon[27919]: pgmap v698: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:12.450 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:39 smithi131 ceph-mon[27919]: pgmap v699: 129 pgs: 129 active+clean; 677 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:12.456 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:41 smithi131 ceph-mon[27919]: pgmap v700: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:22:12.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:43 smithi131 ceph-mon[27919]: pgmap v701: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:12.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:45 smithi131 ceph-mon[27919]: pgmap v702: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:12.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:45 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:12.647 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:12.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:12.661 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:46 smithi131 ceph-mon[27919]: pgmap v703: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:12.670 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:12.678 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:12.687 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:12.694 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:46 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:12.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:03 smithi148 ceph-mon[32281]: pgmap v771: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:12.799 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:05 smithi148 ceph-mon[32281]: pgmap v772: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:22:12.885 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:07 smithi148 ceph-mon[32281]: pgmap v773: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:12.938 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:09 smithi148 ceph-mon[32281]: pgmap v774: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:13.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:11 smithi148 ceph-mon[32281]: pgmap v775: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:13.085 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:03 smithi157 ceph-mon[32703]: pgmap v771: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:13.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:05 smithi157 ceph-mon[32703]: pgmap v772: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:22:13.099 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:07 smithi157 ceph-mon[32703]: pgmap v773: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:13.170 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:09 smithi157 ceph-mon[32703]: pgmap v774: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:13.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:11 smithi157 ceph-mon[32703]: pgmap v775: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 -- 172.21.15.131:0/3622045436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8108530 msgr2=0x7f14b810a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 --2- 172.21.15.131:0/3622045436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8108530 0x7f14b810a920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009ca0 tx=0x7f14b402a8b0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 -- 172.21.15.131:0/3622045436 shutdown_connections 2024-04-26T20:22:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 --2- 172.21.15.131:0/3622045436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8108530 0x7f14b810a920 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 --2- 172.21.15.131:0/3622045436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b8105c00 0x7f14b8107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 --2- 172.21.15.131:0/3622045436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 0x7f14b81056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 -- 172.21.15.131:0/3622045436 >> 172.21.15.131:0/3622045436 conn(0x7f14b80fcc20 msgr2=0x7f14b80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 -- 172.21.15.131:0/3622045436 shutdown_connections 2024-04-26T20:22:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.496+0000 7f14bf200640 1 -- 172.21.15.131:0/3622045436 wait complete. 2024-04-26T20:22:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.497+0000 7f14bf200640 1 Processor -- start 2024-04-26T20:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bf200640 1 -- start start 2024-04-26T20:22:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 0x7f14b8102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bf200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8105c00 0x7f14b8101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b8108530 0x7f14b81017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bf200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810e330 con 0x7f14b81032d0 2024-04-26T20:22:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bf200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810e1b0 con 0x7f14b8105c00 2024-04-26T20:22:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810e4b0 con 0x7f14b8108530 2024-04-26T20:22:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 0x7f14b8102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 0x7f14b8102c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35916/0 (socket says 172.21.15.131:35916) 2024-04-26T20:22:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14bde00640 1 -- 172.21.15.131:0/2925240086 learned_addr learned my addr 172.21.15.131:0/2925240086 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.498+0000 7f14be800640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b8108530 0x7f14b81017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14bd400640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8105c00 0x7f14b8101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14bde00640 1 -- 172.21.15.131:0/2925240086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b8108530 msgr2=0x7f14b81017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14bde00640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b8108530 0x7f14b81017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14bde00640 1 -- 172.21.15.131:0/2925240086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8105c00 msgr2=0x7f14b8101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14bde00640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8105c00 0x7f14b8101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14bde00640 1 -- 172.21.15.131:0/2925240086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b8102050 con 0x7f14b81032d0 2024-04-26T20:22:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14be800640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b8108530 0x7f14b81017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14bde00640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 0x7f14b8102c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00e9f0 tx=0x7f14ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.499+0000 7f14b2a00640 1 -- 172.21.15.131:0/2925240086 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00cd80 con 0x7f14b81032d0 2024-04-26T20:22:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.500+0000 7f14b2a00640 1 -- 172.21.15.131:0/2925240086 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac0108e0 con 0x7f14b81032d0 2024-04-26T20:22:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.500+0000 7f14b2a00640 1 -- 172.21.15.131:0/2925240086 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac018980 con 0x7f14b81032d0 2024-04-26T20:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.500+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b81a2ac0 con 0x7f14b81032d0 2024-04-26T20:22:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.500+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b81a3060 con 0x7f14b81032d0 2024-04-26T20:22:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.503+0000 7f14b2a00640 1 -- 172.21.15.131:0/2925240086 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f14ac010430 con 0x7f14b81032d0 2024-04-26T20:22:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 Processor -- start 2024-04-26T20:22:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 -- start start 2024-04-26T20:22:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a74200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a79940 0x55ba02a7bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ba029d8380 con 0x55ba02a79940 2024-04-26T20:22:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ba02974c30 con 0x55ba02a73de0 2024-04-26T20:22:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14c3782740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ba029d4340 con 0x55ba02a747d0 2024-04-26T20:22:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14bd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a79940 0x55ba02a7bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.504+0000 7f14be800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a74200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a74200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55002/0 (socket says 172.21.15.131:55002) 2024-04-26T20:22:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 -- 172.21.15.131:0/2599502073 learned_addr learned my addr 172.21.15.131:0/2599502073 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14bd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a79940 0x55ba02a7bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35924/0 (socket says 172.21.15.131:35924) 2024-04-26T20:22:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14bde00640 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 -- 172.21.15.131:0/2599502073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 msgr2=0x55ba02a792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a792d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 -- 172.21.15.131:0/2599502073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a79940 msgr2=0x55ba02a7bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a79940 0x55ba02a7bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 -- 172.21.15.131:0/2599502073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba02996940 con 0x55ba02a73de0 2024-04-26T20:22:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.505+0000 7f14be800640 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a74200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14b4005e30 tx=0x7f14b4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4ddcae7c6fd612 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.506+0000 7f14a7000640 1 -- 172.21.15.131:0/2599502073 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4030900 con 0x55ba02a73de0 2024-04-26T20:22:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.506+0000 7f14a7000640 1 -- 172.21.15.131:0/2599502073 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4036040 con 0x55ba02a73de0 2024-04-26T20:22:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.506+0000 7f14a7000640 1 -- 172.21.15.131:0/2599502073 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40043f0 con 0x55ba02a73de0 2024-04-26T20:22:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.506+0000 7f14c3782740 1 -- 172.21.15.131:0/2599502073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 msgr2=0x55ba02a74200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.506+0000 7f14c3782740 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a74200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14b4005e30 tx=0x7f14b4009a20 comp rx=0 tx=0).stop 2024-04-26T20:22:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.506+0000 7f14b2a00640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14a00a3cb0 0x7f14a00a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14c3782740 1 -- 172.21.15.131:0/2599502073 shutdown_connections 2024-04-26T20:22:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14c3782740 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a79940 0x55ba02a7bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14c3782740 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14c3782740 1 --2- 172.21.15.131:0/2599502073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a74200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14c3782740 1 -- 172.21.15.131:0/2599502073 >> 172.21.15.131:0/2599502073 conn(0x55ba02a6f450 msgr2=0x55ba02a6f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14b2a00640 1 -- 172.21.15.131:0/2925240086 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f14ac0d0db0 con 0x7f14b81032d0 2024-04-26T20:22:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14be800640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14a00a3cb0 0x7f14a00a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14c3782740 1 -- 172.21.15.131:0/2599502073 shutdown_connections 2024-04-26T20:22:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.507+0000 7f14c3782740 1 -- 172.21.15.131:0/2599502073 wait complete. 2024-04-26T20:22:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.508+0000 7f14c3782740 1 -- start start 2024-04-26T20:22:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.508+0000 7f14be800640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14a00a3cb0 0x7f14a00a6170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f14b402add0 tx=0x7f14b402a8b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.508+0000 7f14c3782740 1 Processor -- start 2024-04-26T20:22:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.508+0000 7f14c3782740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a7dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.508+0000 7f14c3782740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a747d0 0x55ba02a7e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14c3782740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a79940 0x55ba02a85540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14c3782740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ba02a7c890 con 0x55ba02a747d0 2024-04-26T20:22:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14c3782740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ba0295fc60 con 0x55ba02a73de0 2024-04-26T20:22:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14c3782740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ba02a7ca10 con 0x55ba02a79940 2024-04-26T20:22:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14c3782740 10 client.0 _ll_register_callbacks cb 0x55ba0298be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a7dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a7dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55004/0 (socket says 172.21.15.131:55004) 2024-04-26T20:22:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 -- 172.21.15.131:0/1846338942 learned_addr learned my addr 172.21.15.131:0/1846338942 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bd400640 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a747d0 0x55ba02a7e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14be800640 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a79940 0x55ba02a85540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 -- 172.21.15.131:0/1846338942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a79940 msgr2=0x55ba02a85540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a79940 0x55ba02a85540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 -- 172.21.15.131:0/1846338942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a747d0 msgr2=0x55ba02a7e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a747d0 0x55ba02a7e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.509+0000 7f14bde00640 1 -- 172.21.15.131:0/1846338942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba0296e510 con 0x55ba02a73de0 2024-04-26T20:22:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f14bde00640 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a7dfc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00e9c0 tx=0x7f14ac00cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4d282ef3e0177b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f149b400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:22:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f149b400640 1 -- 172.21.15.131:0/1846338942 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac010720 con 0x55ba02a73de0 2024-04-26T20:22:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b4036040 con 0x55ba02a73de0 2024-04-26T20:22:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f149b400640 1 -- 172.21.15.131:0/1846338942 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f14ac010a90 con 0x55ba02a73de0 2024-04-26T20:22:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f149b400640 1 -- 172.21.15.131:0/1846338942 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14ac09d1e0 con 0x55ba02a73de0 2024-04-26T20:22:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f14c3782740 10 client.26036 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba026ea210 con 0x55ba02a73de0 2024-04-26T20:22:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.510+0000 7f149b400640 1 -- 172.21.15.131:0/1846338942 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4036040 con 0x55ba02a73de0 2024-04-26T20:22:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f149b400640 1 -- 172.21.15.131:0/1846338942 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14ac018b20 con 0x55ba02a73de0 2024-04-26T20:22:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f14c3782740 10 client.26036 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f14c3782740 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f14c3782740 20 client.26036 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f14c3782740 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02a89a10 0x55ba02a8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f14c3782740 4 client.26036 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:22:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba029e5f90 con 0x55ba02a89a10 2024-04-26T20:22:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.511+0000 7f14bd400640 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02a89a10 0x55ba02a8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.512+0000 7f149b400640 10 client.26036 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.512+0000 7f14bd400640 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02a89a10 0x55ba02a8bea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b1496ad5d35aeb server_cookie=d393512f8b242fbe in_seq=0 out_seq=0 2024-04-26T20:22:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.512+0000 7f149b400640 1 -- 172.21.15.131:0/1846338942 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba029e5f90 con 0x55ba02a89a10 2024-04-26T20:22:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.512+0000 7f149b400640 10 client.26036 handle_command_reply: tid=0 2024-04-26T20:22:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 client.26036 shutdown 2024-04-26T20:22:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 msgr2=0x55ba02a7dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a7dfc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00e9c0 tx=0x7f14ac00cf20 comp rx=0 tx=0).stop 2024-04-26T20:22:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 shutdown_connections 2024-04-26T20:22:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02a89a10 0x55ba02a8bea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a79940 0x55ba02a85540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a747d0 0x55ba02a7e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f149b400640 0 client.26036 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 --2- 172.21.15.131:0/1846338942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a73de0 0x55ba02a7dfc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 >> 172.21.15.131:0/1846338942 conn(0x55ba02a6f450 msgr2=0x55ba02a6e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 shutdown_connections 2024-04-26T20:22:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 1 -- 172.21.15.131:0/1846338942 wait complete. 2024-04-26T20:22:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.513+0000 7f14c3782740 20 client.26036 trim_cache size 0 max 16384 2024-04-26T20:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.529+0000 7f14c3782740 1 Processor -- start 2024-04-26T20:22:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.529+0000 7f14c3782740 1 -- start start 2024-04-26T20:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14c3782740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a72400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14c3782740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14c3782740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a72940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14c3782740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81b0010 con 0x55ba02a73de0 2024-04-26T20:22:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14c3782740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81ad8e0 con 0x55ba02a79940 2024-04-26T20:22:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14c3782740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81ab730 con 0x55ba02a747d0 2024-04-26T20:22:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14bde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a72400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14bd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14bde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a72400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35944/0 (socket says 172.21.15.131:35944) 2024-04-26T20:22:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14bde00640 1 -- 172.21.15.131:0/3377287965 learned_addr learned my addr 172.21.15.131:0/3377287965 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.530+0000 7f14be800640 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a72940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14bd400640 1 -- 172.21.15.131:0/3377287965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 msgr2=0x55ba02a72940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14bd400640 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a72940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14bd400640 1 -- 172.21.15.131:0/3377287965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 msgr2=0x55ba02a72400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14bd400640 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a72400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14bd400640 1 -- 172.21.15.131:0/3377287965 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba027b6620 con 0x55ba02a747d0 2024-04-26T20:22:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14bde00640 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a72400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14bd400640 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a854d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ba02a8c750 tx=0x7f14a800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9870cb1bc8c938b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14a6600640 1 -- 172.21.15.131:0/3377287965 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8023070 con 0x55ba02a747d0 2024-04-26T20:22:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14a6600640 1 -- 172.21.15.131:0/3377287965 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a801d430 con 0x55ba02a747d0 2024-04-26T20:22:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.531+0000 7f14a6600640 1 -- 172.21.15.131:0/3377287965 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8003970 con 0x55ba02a747d0 2024-04-26T20:22:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 -- 172.21.15.131:0/3377287965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 msgr2=0x55ba02a854d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a854d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ba02a8c750 tx=0x7f14a800bf90 comp rx=0 tx=0).stop 2024-04-26T20:22:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 -- 172.21.15.131:0/3377287965 shutdown_connections 2024-04-26T20:22:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a72940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a854d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 --2- 172.21.15.131:0/3377287965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a72400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 -- 172.21.15.131:0/3377287965 >> 172.21.15.131:0/3377287965 conn(0x55ba02a6f450 msgr2=0x55ba02a6f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 -- 172.21.15.131:0/3377287965 shutdown_connections 2024-04-26T20:22:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.532+0000 7f14c3782740 1 -- 172.21.15.131:0/3377287965 wait complete. 2024-04-26T20:22:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 -- start start 2024-04-26T20:22:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 Processor -- start 2024-04-26T20:22:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a82e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a83da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81abda0 con 0x55ba02a73de0 2024-04-26T20:22:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81ad210 con 0x55ba02a79940 2024-04-26T20:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81ac470 con 0x55ba02a747d0 2024-04-26T20:22:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.533+0000 7f14c3782740 10 client.0 _ll_register_callbacks cb 0x7f14ac0d4550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a83da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14bde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a83da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55012/0 (socket says 172.21.15.131:55012) 2024-04-26T20:22:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 -- 172.21.15.131:0/774723182 learned_addr learned my addr 172.21.15.131:0/774723182 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14bd400640 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a82e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 -- 172.21.15.131:0/774723182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 msgr2=0x55ba02a82e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a82e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 -- 172.21.15.131:0/774723182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 msgr2=0x55ba02a828f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a828f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 -- 172.21.15.131:0/774723182 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba02992cb0 con 0x55ba02a79940 2024-04-26T20:22:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14bde00640 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f14be800640 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a83da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14b4000f80 tx=0x7f14b4005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73631d6e92b7cc0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.534+0000 7f149b400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f149b400640 1 -- 172.21.15.131:0/774723182 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4030a80 con 0x55ba02a79940 2024-04-26T20:22:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a801d430 con 0x55ba02a79940 2024-04-26T20:22:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f149b400640 1 -- 172.21.15.131:0/774723182 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f14b4058050 con 0x55ba02a79940 2024-04-26T20:22:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f149b400640 1 -- 172.21.15.131:0/774723182 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14b4069050 con 0x55ba02a79940 2024-04-26T20:22:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f14c3782740 10 client.26042 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba02a85520 con 0x55ba02a79940 2024-04-26T20:22:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f149b400640 1 -- 172.21.15.131:0/774723182 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a801d430 con 0x55ba02a79940 2024-04-26T20:22:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.535+0000 7f149b400640 1 -- 172.21.15.131:0/774723182 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14b4003dc0 con 0x55ba02a79940 2024-04-26T20:22:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14c3782740 10 client.26042 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14c3782740 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14c3782740 20 client.26042 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14c3782740 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02af0020 0x55ba02af2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14c3782740 4 client.26042 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:22:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba029dfe00 con 0x55ba02af0020 2024-04-26T20:22:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14bde00640 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02af0020 0x55ba02af2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f14bde00640 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02af0020 0x55ba02af2410 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d828267b08a827 server_cookie=6ed4b2b860f1ccc3 in_seq=0 out_seq=0 2024-04-26T20:22:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.536+0000 7f149b400640 10 client.26042 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f149b400640 1 -- 172.21.15.131:0/774723182 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba029dfe00 con 0x55ba02af0020 2024-04-26T20:22:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f149b400640 10 client.26042 handle_command_reply: tid=0 2024-04-26T20:22:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 client.26042 shutdown 2024-04-26T20:22:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 msgr2=0x55ba02a83da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a83da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14b4000f80 tx=0x7f14b4005da0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 shutdown_connections 2024-04-26T20:22:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ba02af0020 0x55ba02af2410 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba02a79940 0x55ba02a83da0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f149b400640 0 client.26042 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba02a747d0 0x55ba02a82e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.540+0000 7f14c3782740 1 --2- 172.21.15.131:0/774723182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba02a73de0 0x55ba02a828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.541+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 >> 172.21.15.131:0/774723182 conn(0x55ba02a6f450 msgr2=0x55ba02a72400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.541+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 shutdown_connections 2024-04-26T20:22:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.541+0000 7f14c3782740 1 -- 172.21.15.131:0/774723182 wait complete. 2024-04-26T20:22:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.541+0000 7f14c3782740 20 client.26042 trim_cache size 0 max 16384 2024-04-26T20:22:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:22:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:22:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:22:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:22:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.541+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14a00a3cb0 msgr2=0x7f14a00a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14a00a3cb0 0x7f14a00a6170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f14b402add0 tx=0x7f14b402a8b0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 msgr2=0x7f14b8102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 0x7f14b8102c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00e9f0 tx=0x7f14ac00eeb0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 shutdown_connections 2024-04-26T20:22:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f14a00a3cb0 0x7f14a00a6170 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b8108530 0x7f14b81017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8105c00 0x7f14b8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 --2- 172.21.15.131:0/2925240086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14b81032d0 0x7f14b8102c40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 >> 172.21.15.131:0/2925240086 conn(0x7f14b80fcc20 msgr2=0x7f14b8103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 shutdown_connections 2024-04-26T20:22:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:19:49.542+0000 7f14bf200640 1 -- 172.21.15.131:0/2925240086 wait complete. 2024-04-26T20:22:22.467 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:13 smithi148 ceph-mon[32281]: pgmap v776: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:22.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:15 smithi148 ceph-mon[32281]: pgmap v777: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:22.530 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:17 smithi148 ceph-mon[32281]: pgmap v778: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:22.538 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:19 smithi148 ceph-mon[32281]: pgmap v779: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:22:22.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:21 smithi148 ceph-mon[32281]: pgmap v780: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:22.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:13 smithi157 ceph-mon[32703]: pgmap v776: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:22.672 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:15 smithi157 ceph-mon[32703]: pgmap v777: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:22.694 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:17 smithi157 ceph-mon[32703]: pgmap v778: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:22.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:19 smithi157 ceph-mon[32703]: pgmap v779: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:22:22.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:21 smithi157 ceph-mon[32703]: pgmap v780: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:22.889 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:49 smithi131 ceph-mon[27919]: pgmap v704: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:22.898 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:22.904 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:49 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:22.911 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:22.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:22.925 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:50 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:22.946 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:51 smithi131 ceph-mon[27919]: pgmap v705: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:22.954 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:53 smithi131 ceph-mon[27919]: pgmap v706: 129 pgs: 129 active+clean; 678 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:22.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:55 smithi131 ceph-mon[27919]: pgmap v707: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:23.046 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:57 smithi131 ceph-mon[27919]: pgmap v708: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:23.108 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:19:59 smithi131 ceph-mon[27919]: pgmap v709: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:23.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:00 smithi131 ceph-mon[27919]: overall HEALTH_OK 2024-04-26T20:22:23.371 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:01 smithi131 ceph-mon[27919]: pgmap v710: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:24.101 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:23 smithi148 ceph-mon[32281]: pgmap v781: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:24.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:23 smithi157 ceph-mon[32703]: pgmap v781: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 -- 172.21.15.131:0/426907964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 msgr2=0x7f30c41093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 --2- 172.21.15.131:0/426907964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c41093e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009900 tx=0x7f30b402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 -- 172.21.15.131:0/426907964 shutdown_connections 2024-04-26T20:22:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 --2- 172.21.15.131:0/426907964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c410ab70 0x7f30c4111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 --2- 172.21.15.131:0/426907964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c410a1b0 0x7f30c410a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 --2- 172.21.15.131:0/426907964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c41093e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 -- 172.21.15.131:0/426907964 >> 172.21.15.131:0/426907964 conn(0x7f30c4077f80 msgr2=0x7f30c407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 -- 172.21.15.131:0/426907964 shutdown_connections 2024-04-26T20:22:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.682+0000 7f30c8a00640 1 -- 172.21.15.131:0/426907964 wait complete. 2024-04-26T20:22:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.683+0000 7f30c8a00640 1 Processor -- start 2024-04-26T20:22:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.683+0000 7f30c8a00640 1 -- start start 2024-04-26T20:22:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c8a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c410a1b0 0x7f30c419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c8a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c410ab70 0x7f30c41a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c8a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4114b50 con 0x7f30c4108fc0 2024-04-26T20:22:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c8a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41149d0 con 0x7f30c410ab70 2024-04-26T20:22:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c8a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4114cd0 con 0x7f30c410a1b0 2024-04-26T20:22:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44868/0 (socket says 172.21.15.131:44868) 2024-04-26T20:22:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c1e00640 1 -- 172.21.15.131:0/3615416942 learned_addr learned my addr 172.21.15.131:0/3615416942 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30bbe00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c410a1b0 0x7f30c419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.684+0000 7f30c2800640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c410ab70 0x7f30c41a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c1e00640 1 -- 172.21.15.131:0/3615416942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c410a1b0 msgr2=0x7f30c419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c1e00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c410a1b0 0x7f30c419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c1e00640 1 -- 172.21.15.131:0/3615416942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c410ab70 msgr2=0x7f30c41a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c1e00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c410ab70 0x7f30c41a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c1e00640 1 -- 172.21.15.131:0/3615416942 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c41a6d60 con 0x7f30c4108fc0 2024-04-26T20:22:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c2800640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c410ab70 0x7f30c41a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c1e00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c419f140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f30b4031500 tx=0x7f30b4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30baa00640 1 -- 172.21.15.131:0/3615416942 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b403f070 con 0x7f30c4108fc0 2024-04-26T20:22:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30baa00640 1 -- 172.21.15.131:0/3615416942 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b4031c50 con 0x7f30c4108fc0 2024-04-26T20:22:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c41a6ff0 con 0x7f30c4108fc0 2024-04-26T20:22:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30baa00640 1 -- 172.21.15.131:0/3615416942 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4035030 con 0x7f30c4108fc0 2024-04-26T20:22:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.685+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c41a7450 con 0x7f30c4108fc0 2024-04-26T20:22:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 Processor -- start 2024-04-26T20:22:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30baa00640 1 -- 172.21.15.131:0/3615416942 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f30b4042070 con 0x7f30c4108fc0 2024-04-26T20:22:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 -- start start 2024-04-26T20:22:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef02ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef0abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562caee67380 con 0x562caef08730 2024-04-26T20:22:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562caee03c30 con 0x562caef035c0 2024-04-26T20:22:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562caee63340 con 0x562caef02bd0 2024-04-26T20:22:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30c2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef02ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.689+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef0abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef0abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44878/0 (socket says 172.21.15.131:44878) 2024-04-26T20:22:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 -- 172.21.15.131:0/1251021205 learned_addr learned my addr 172.21.15.131:0/1251021205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30c2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef02ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60364/0 (socket says 172.21.15.131:60364) 2024-04-26T20:22:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30c1e00640 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 -- 172.21.15.131:0/1251021205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 msgr2=0x562caef02ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef02ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 -- 172.21.15.131:0/1251021205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 msgr2=0x562caef080c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef080c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 -- 172.21.15.131:0/1251021205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562caee25940 con 0x562caef08730 2024-04-26T20:22:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30bbe00640 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef0abc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009ca0 tx=0x7f30ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31db8d8f42e8106c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.690+0000 7f30baa00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f30a40a3d70 0x7f30a40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30b3000640 1 -- 172.21.15.131:0/1251021205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac01e070 con 0x562caef08730 2024-04-26T20:22:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30b3000640 1 -- 172.21.15.131:0/1251021205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ac019ae0 con 0x562caef08730 2024-04-26T20:22:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30b3000640 1 -- 172.21.15.131:0/1251021205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00e9b0 con 0x562caef08730 2024-04-26T20:22:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30c2800640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f30a40a3d70 0x7f30a40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30baa00640 1 -- 172.21.15.131:0/3615416942 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f30b40f10c0 con 0x7f30c4108fc0 2024-04-26T20:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30cb7e4740 1 -- 172.21.15.131:0/1251021205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 msgr2=0x562caef0abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30c2800640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f30a40a3d70 0x7f30a40a6230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x562caef03e20 tx=0x7f30bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.691+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef0abc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009ca0 tx=0x7f30ac011810 comp rx=0 tx=0).stop 2024-04-26T20:22:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 -- 172.21.15.131:0/1251021205 shutdown_connections 2024-04-26T20:22:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef0abc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/1251021205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef02ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 -- 172.21.15.131:0/1251021205 >> 172.21.15.131:0/1251021205 conn(0x562caeefe240 msgr2=0x562caeefe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 -- 172.21.15.131:0/1251021205 shutdown_connections 2024-04-26T20:22:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 -- 172.21.15.131:0/1251021205 wait complete. 2024-04-26T20:22:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 -- start start 2024-04-26T20:22:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.692+0000 7f30cb7e4740 1 Processor -- start 2024-04-26T20:22:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef02bd0 0x562caef0ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef035c0 0x562caef0d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562caef0b6e0 con 0x562caef08730 2024-04-26T20:22:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562caedeec60 con 0x562caef02bd0 2024-04-26T20:22:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562caef0b860 con 0x562caef035c0 2024-04-26T20:22:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30cb7e4740 10 client.0 _ll_register_callbacks cb 0x562caee1ae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef035c0 0x562caef0d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef035c0 0x562caef0d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60374/0 (socket says 172.21.15.131:60374) 2024-04-26T20:22:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 -- 172.21.15.131:0/2979862186 learned_addr learned my addr 172.21.15.131:0/2979862186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef02bd0 0x562caef0ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30c2800640 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 -- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef02bd0 msgr2=0x562caef0ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef02bd0 0x562caef0ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 -- 172.21.15.131:0/2979862186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 msgr2=0x562caef143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30bbe00640 1 -- 172.21.15.131:0/2979862186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562caedfd510 con 0x562caef035c0 2024-04-26T20:22:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.693+0000 7f30c1e00640 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef02bd0 0x562caef0ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.694+0000 7f30bbe00640 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef035c0 0x562caef0d3b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009f50 tx=0x7f30ac019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5461585b345bba1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.694+0000 7f30ab400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:22:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.694+0000 7f30ab400640 1 -- 172.21.15.131:0/2979862186 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac028020 con 0x562caef035c0 2024-04-26T20:22:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.694+0000 7f30ab400640 1 -- 172.21.15.131:0/2979862186 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f30ac005050 con 0x562caef035c0 2024-04-26T20:22:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.694+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30ac019ae0 con 0x562caef035c0 2024-04-26T20:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30ab400640 1 -- 172.21.15.131:0/2979862186 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30ac033050 con 0x562caef035c0 2024-04-26T20:22:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 10 client.26035 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562caeb79210 con 0x562caef035c0 2024-04-26T20:22:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30ab400640 1 -- 172.21.15.131:0/2979862186 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ac019ae0 con 0x562caef035c0 2024-04-26T20:22:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30ab400640 1 -- 172.21.15.131:0/2979862186 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f30ac004cd0 con 0x562caef035c0 2024-04-26T20:22:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 10 client.26035 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 10 client.26035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 20 client.26035 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef18650 0x562caef1aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 4 client.26035 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:22:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.695+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562caee74f90 con 0x562caef18650 2024-04-26T20:22:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.696+0000 7f30c1e00640 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef18650 0x562caef1aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.696+0000 7f30c1e00640 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef18650 0x562caef1aa80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ffd1327c01a0fd server_cookie=d1ccb44a6571a4ba in_seq=0 out_seq=0 2024-04-26T20:22:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.696+0000 7f30ab400640 10 client.26035 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30ab400640 1 -- 172.21.15.131:0/2979862186 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562caee74f90 con 0x562caef18650 2024-04-26T20:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30ab400640 10 client.26035 handle_command_reply: tid=0 2024-04-26T20:22:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 client.26035 shutdown 2024-04-26T20:22:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef035c0 msgr2=0x562caef0d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef035c0 0x562caef0d3b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009f50 tx=0x7f30ac019660 comp rx=0 tx=0).stop 2024-04-26T20:22:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 shutdown_connections 2024-04-26T20:22:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef18650 0x562caef1aa80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef035c0 0x562caef0d3b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/2979862186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef02bd0 0x562caef0ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 >> 172.21.15.131:0/2979862186 conn(0x562caeefe240 msgr2=0x562caeefd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.697+0000 7f30ab400640 0 client.26035 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.698+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 shutdown_connections 2024-04-26T20:22:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.698+0000 7f30cb7e4740 1 -- 172.21.15.131:0/2979862186 wait complete. 2024-04-26T20:22:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.698+0000 7f30cb7e4740 20 client.26035 trim_cache size 0 max 16384 2024-04-26T20:22:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 Processor -- start 2024-04-26T20:22:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 -- start start 2024-04-26T20:22:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef02bd0 0x562caef010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef08730 0x562caef01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41b44b0 con 0x562caef02bd0 2024-04-26T20:22:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41b1d80 con 0x562caef035c0 2024-04-26T20:22:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41afbd0 con 0x562caef08730 2024-04-26T20:22:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef02bd0 0x562caef010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef02bd0 0x562caef010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44898/0 (socket says 172.21.15.131:44898) 2024-04-26T20:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.714+0000 7f30c1e00640 1 -- 172.21.15.131:0/808603153 learned_addr learned my addr 172.21.15.131:0/808603153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30c2800640 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef08730 0x562caef01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30bbe00640 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30c1e00640 1 -- 172.21.15.131:0/808603153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef08730 msgr2=0x562caef01600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30c1e00640 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef08730 0x562caef01600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30c1e00640 1 -- 172.21.15.131:0/808603153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 msgr2=0x562caef143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30c1e00640 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30c1e00640 1 -- 172.21.15.131:0/808603153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562caec45620 con 0x562caef02bd0 2024-04-26T20:22:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30bbe00640 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:22:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30c1e00640 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef02bd0 0x562caef010c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f30b40bcd80 tx=0x7f30b40f3580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be5a241d26d04f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30b2600640 1 -- 172.21.15.131:0/808603153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4049040 con 0x562caef02bd0 2024-04-26T20:22:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30b2600640 1 -- 172.21.15.131:0/808603153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b4031e00 con 0x562caef02bd0 2024-04-26T20:22:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.715+0000 7f30b2600640 1 -- 172.21.15.131:0/808603153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40c11a0 con 0x562caef02bd0 2024-04-26T20:22:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 -- 172.21.15.131:0/808603153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef02bd0 msgr2=0x562caef010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef02bd0 0x562caef010c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f30b40bcd80 tx=0x7f30b40f3580 comp rx=0 tx=0).stop 2024-04-26T20:22:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 -- 172.21.15.131:0/808603153 shutdown_connections 2024-04-26T20:22:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef08730 0x562caef01600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/808603153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef02bd0 0x562caef010c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 -- 172.21.15.131:0/808603153 >> 172.21.15.131:0/808603153 conn(0x562caeefe240 msgr2=0x562caef04840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 -- 172.21.15.131:0/808603153 shutdown_connections 2024-04-26T20:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.716+0000 7f30cb7e4740 1 -- 172.21.15.131:0/808603153 wait complete. 2024-04-26T20:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 -- start start 2024-04-26T20:22:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 Processor -- start 2024-04-26T20:22:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef11de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef12df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41b0240 con 0x562caef08730 2024-04-26T20:22:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41b16b0 con 0x562caef035c0 2024-04-26T20:22:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41b0910 con 0x562caef02bd0 2024-04-26T20:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30cb7e4740 10 client.0 _ll_register_callbacks cb 0x7f30ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30c2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef12df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30c2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef12df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44914/0 (socket says 172.21.15.131:44914) 2024-04-26T20:22:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.717+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c2800640 1 -- 172.21.15.131:0/895184609 learned_addr learned my addr 172.21.15.131:0/895184609 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30bbe00640 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef11de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c2800640 1 -- 172.21.15.131:0/895184609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 msgr2=0x562caef118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c2800640 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef118a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c2800640 1 -- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 msgr2=0x562caef11de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c2800640 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef11de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c2800640 1 -- 172.21.15.131:0/895184609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562caee21cb0 con 0x562caef08730 2024-04-26T20:22:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c1e00640 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30c2800640 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef12df0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f30bc010d40 tx=0x7f30bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f602a164757166f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30ab400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:22:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.718+0000 7f30ab400640 1 -- 172.21.15.131:0/895184609 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc02d060 con 0x562caef08730 2024-04-26T20:22:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562caee6ee00 con 0x562caef08730 2024-04-26T20:22:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30ab400640 1 -- 172.21.15.131:0/895184609 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f30bc01eb30 con 0x562caef08730 2024-04-26T20:22:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30ab400640 1 -- 172.21.15.131:0/895184609 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30bc034050 con 0x562caef08730 2024-04-26T20:22:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 10 client.15939 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562caef14640 con 0x562caef08730 2024-04-26T20:22:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30ab400640 1 -- 172.21.15.131:0/895184609 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562caee6ee00 con 0x562caef08730 2024-04-26T20:22:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30ab400640 1 -- 172.21.15.131:0/895184609 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f30bc01d410 con 0x562caef08730 2024-04-26T20:22:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 10 client.15939 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 20 client.15939 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef7f380 0x562caef817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 4 client.15939 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:22:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.719+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562caedea4b0 con 0x562caef7f380 2024-04-26T20:22:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.720+0000 7f30c1e00640 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef7f380 0x562caef817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.720+0000 7f30c1e00640 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef7f380 0x562caef817a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a733e427e45b65 server_cookie=f30556917adb45f9 in_seq=0 out_seq=0 2024-04-26T20:22:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.720+0000 7f30ab400640 10 client.15939 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30ab400640 1 -- 172.21.15.131:0/895184609 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562caedea4b0 con 0x562caef7f380 2024-04-26T20:22:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30ab400640 10 client.15939 handle_command_reply: tid=0 2024-04-26T20:22:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 client.15939 shutdown 2024-04-26T20:22:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 msgr2=0x562caef12df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef12df0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f30bc010d40 tx=0x7f30bc004a40 comp rx=0 tx=0).stop 2024-04-26T20:22:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 shutdown_connections 2024-04-26T20:22:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562caef7f380 0x562caef817a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562caef08730 0x562caef12df0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caef035c0 0x562caef11de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 --2- 172.21.15.131:0/895184609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562caef02bd0 0x562caef118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30ab400640 0 client.15939 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 >> 172.21.15.131:0/895184609 conn(0x562caeefe240 msgr2=0x562caef010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 shutdown_connections 2024-04-26T20:22:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.724+0000 7f30cb7e4740 1 -- 172.21.15.131:0/895184609 wait complete. 2024-04-26T20:22:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.725+0000 7f30cb7e4740 20 client.15939 trim_cache size 0 max 16384 2024-04-26T20:22:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:22:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:22:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:22:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:22:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.725+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f30a40a3d70 msgr2=0x7f30a40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.725+0000 7f30c8a00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f30a40a3d70 0x7f30a40a6230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x562caef03e20 tx=0x7f30bc009500 comp rx=0 tx=0).stop 2024-04-26T20:22:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.725+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 msgr2=0x7f30c419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.725+0000 7f30c8a00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c419f140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f30b4031500 tx=0x7f30b4003c10 comp rx=0 tx=0).stop 2024-04-26T20:22:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 shutdown_connections 2024-04-26T20:22:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c410ab70 0x7f30c41a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f30a40a3d70 0x7f30a40a6230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c410a1b0 0x7f30c419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 --2- 172.21.15.131:0/3615416942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30c4108fc0 0x7f30c419f140 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 >> 172.21.15.131:0/3615416942 conn(0x7f30c4077f80 msgr2=0x7f30c410d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 shutdown_connections 2024-04-26T20:22:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:02.726+0000 7f30c8a00640 1 -- 172.21.15.131:0/3615416942 wait complete. 2024-04-26T20:22:35.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:03 smithi131 ceph-mon[27919]: pgmap v711: 129 pgs: 129 active+clean; 679 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:35.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:05 smithi131 ceph-mon[27919]: pgmap v712: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:22:35.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:07 smithi131 ceph-mon[27919]: pgmap v713: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:35.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:09 smithi131 ceph-mon[27919]: pgmap v714: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:35.317 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:11 smithi131 ceph-mon[27919]: pgmap v715: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:35.325 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:13 smithi131 ceph-mon[27919]: pgmap v716: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:35.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:25 smithi148 ceph-mon[32281]: pgmap v782: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:35.415 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:27 smithi148 ceph-mon[32281]: pgmap v783: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:35.465 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:29 smithi148 ceph-mon[32281]: pgmap v784: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:35.535 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:31 smithi148 ceph-mon[32281]: pgmap v785: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:22:35.542 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:33 smithi148 ceph-mon[32281]: pgmap v786: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:35.548 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:25 smithi157 ceph-mon[32703]: pgmap v782: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:35.648 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:27 smithi157 ceph-mon[32703]: pgmap v783: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:35.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:29 smithi157 ceph-mon[32703]: pgmap v784: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:35.848 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:31 smithi157 ceph-mon[32703]: pgmap v785: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:22:35.917 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:33 smithi157 ceph-mon[32703]: pgmap v786: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 -- 172.21.15.131:0/1951619558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f6010a990 msgr2=0x7f8f60111220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1951619558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f6010a990 0x7f8f60111220 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009be0 tx=0x7f8f5802a6d0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 -- 172.21.15.131:0/1951619558 shutdown_connections 2024-04-26T20:22:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1951619558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f6010a990 0x7f8f60111220 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1951619558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f60109fd0 0x7f8f6010a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1951619558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60108de0 0x7f8f60109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 -- 172.21.15.131:0/1951619558 >> 172.21.15.131:0/1951619558 conn(0x7f8f60077dc0 msgr2=0x7f8f6007a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 -- 172.21.15.131:0/1951619558 shutdown_connections 2024-04-26T20:22:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.867+0000 7f8f5f200640 1 -- 172.21.15.131:0/1951619558 wait complete. 2024-04-26T20:22:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.868+0000 7f8f5f200640 1 Processor -- start 2024-04-26T20:22:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.868+0000 7f8f5f200640 1 -- start start 2024-04-26T20:22:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60108de0 0x7f8f6019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f60109fd0 0x7f8f6019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f6010a990 0x7f8f601a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601148f0 con 0x7f8f60109fd0 2024-04-26T20:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60114770 con 0x7f8f60108de0 2024-04-26T20:22:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60114a70 con 0x7f8f6010a990 2024-04-26T20:22:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f60109fd0 0x7f8f6019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f60109fd0 0x7f8f6019f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37390/0 (socket says 172.21.15.131:37390) 2024-04-26T20:22:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f57e00640 1 -- 172.21.15.131:0/1711366326 learned_addr learned my addr 172.21.15.131:0/1711366326 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60108de0 0x7f8f6019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5e800640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f6010a990 0x7f8f601a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5de00640 1 -- 172.21.15.131:0/1711366326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f6010a990 msgr2=0x7f8f601a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5de00640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f6010a990 0x7f8f601a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5de00640 1 -- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f60109fd0 msgr2=0x7f8f6019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5de00640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f60109fd0 0x7f8f6019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.869+0000 7f8f5de00640 1 -- 172.21.15.131:0/1711366326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f601a6bb0 con 0x7f8f60108de0 2024-04-26T20:22:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.870+0000 7f8f5de00640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60108de0 0x7f8f6019ef90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e9c0 tx=0x7f8f5000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.870+0000 7f8f56a00640 1 -- 172.21.15.131:0/1711366326 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000cd80 con 0x7f8f60108de0 2024-04-26T20:22:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.870+0000 7f8f56a00640 1 -- 172.21.15.131:0/1711366326 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f500108e0 con 0x7f8f60108de0 2024-04-26T20:22:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.870+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f601a6ea0 con 0x7f8f60108de0 2024-04-26T20:22:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.870+0000 7f8f56a00640 1 -- 172.21.15.131:0/1711366326 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50018900 con 0x7f8f60108de0 2024-04-26T20:22:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.870+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f601a7300 con 0x7f8f60108de0 2024-04-26T20:22:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.873+0000 7f8f67a0a740 1 Processor -- start 2024-04-26T20:22:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.873+0000 7f8f56a00640 1 -- 172.21.15.131:0/1711366326 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8f5001f020 con 0x7f8f60108de0 2024-04-26T20:22:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.873+0000 7f8f67a0a740 1 -- start start 2024-04-26T20:22:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6ed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55942c64c380 con 0x55942c6e87d0 2024-04-26T20:22:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55942c5e8c30 con 0x55942c6e7de0 2024-04-26T20:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55942c648340 con 0x55942c6ed940 2024-04-26T20:22:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6ed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f5e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f5e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:39840/0 (socket says 172.21.15.131:39840) 2024-04-26T20:22:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f5e800640 1 -- 172.21.15.131:0/2302506856 learned_addr learned my addr 172.21.15.131:0/2302506856 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.874+0000 7f8f57e00640 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f5e800640 1 -- 172.21.15.131:0/2302506856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 msgr2=0x55942c6efdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f5e800640 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6efdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f5e800640 1 -- 172.21.15.131:0/2302506856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 msgr2=0x55942c6ed2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f5e800640 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6ed2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f5e800640 1 -- 172.21.15.131:0/2302506856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942c60a940 con 0x55942c6e7de0 2024-04-26T20:22:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f5e800640 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6e8200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5802acc0 tx=0x7f8f58005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84df4fa82e525266 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f56a00640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8f440a3c30 0x7f8f440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f4f000640 1 -- 172.21.15.131:0/2302506856 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58041030 con 0x55942c6e7de0 2024-04-26T20:22:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f4f000640 1 -- 172.21.15.131:0/2302506856 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f58030660 con 0x55942c6e7de0 2024-04-26T20:22:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f4f000640 1 -- 172.21.15.131:0/2302506856 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58036040 con 0x55942c6e7de0 2024-04-26T20:22:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f56a00640 1 -- 172.21.15.131:0/1711366326 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8f500d0960 con 0x7f8f60108de0 2024-04-26T20:22:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.875+0000 7f8f5de00640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8f440a3c30 0x7f8f440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 -- 172.21.15.131:0/2302506856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 msgr2=0x55942c6e8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6e8200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5802acc0 tx=0x7f8f58005ec0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 -- 172.21.15.131:0/2302506856 shutdown_connections 2024-04-26T20:22:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6ed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/2302506856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6e8200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 -- 172.21.15.131:0/2302506856 >> 172.21.15.131:0/2302506856 conn(0x55942c6e3450 msgr2=0x55942c6e3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f5de00640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8f440a3c30 0x7f8f440a60f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55942c6e8bd0 tx=0x7f8f500d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 -- 172.21.15.131:0/2302506856 shutdown_connections 2024-04-26T20:22:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.876+0000 7f8f67a0a740 1 -- 172.21.15.131:0/2302506856 wait complete. 2024-04-26T20:22:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 -- start start 2024-04-26T20:22:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 Processor -- start 2024-04-26T20:22:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55942c6f0890 con 0x55942c6e87d0 2024-04-26T20:22:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55942c5d3c60 con 0x55942c6e7de0 2024-04-26T20:22:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55942c6f0a10 con 0x55942c6ed940 2024-04-26T20:22:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.877+0000 7f8f67a0a740 10 client.0 _ll_register_callbacks cb 0x55942c5ffe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f5e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37406/0 (socket says 172.21.15.131:37406) 2024-04-26T20:22:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f5e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6f1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:39846/0 (socket says 172.21.15.131:39846) 2024-04-26T20:22:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 -- 172.21.15.131:0/3783289993 learned_addr learned my addr 172.21.15.131:0/3783289993 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f5e800640 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 -- 172.21.15.131:0/3783289993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 msgr2=0x55942c6f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6f9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 -- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 msgr2=0x55942c6f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 -- 172.21.15.131:0/3783289993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942c5e2510 con 0x55942c6e87d0 2024-04-26T20:22:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f5e800640 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f57e00640 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f24e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8f48012700 tx=0x7f8f48012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa930e8c8ef6d645 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.878+0000 7f8f43400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:22:39.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f43400640 1 -- 172.21.15.131:0/3783289993 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800f8c0 con 0x55942c6e87d0 2024-04-26T20:22:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f43400640 1 -- 172.21.15.131:0/3783289993 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8f4800fa60 con 0x55942c6e87d0 2024-04-26T20:22:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f58030660 con 0x55942c6e87d0 2024-04-26T20:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f43400640 1 -- 172.21.15.131:0/3783289993 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f4800fc60 con 0x55942c6e87d0 2024-04-26T20:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f67a0a740 10 client.15951 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55942c35e210 con 0x55942c6e87d0 2024-04-26T20:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f43400640 1 -- 172.21.15.131:0/3783289993 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f58030660 con 0x55942c6e87d0 2024-04-26T20:22:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.879+0000 7f8f43400640 1 -- 172.21.15.131:0/3783289993 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8f4801c420 con 0x55942c6e87d0 2024-04-26T20:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f67a0a740 10 client.15951 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f67a0a740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f67a0a740 20 client.15951 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c6fdb40 0x55942c6fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f67a0a740 4 client.15951 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:22:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55942c659f90 con 0x55942c6fdb40 2024-04-26T20:22:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f5e800640 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c6fdb40 0x55942c6fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.880+0000 7f8f5e800640 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c6fdb40 0x55942c6fff90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56b3ff5d2bf2cfc server_cookie=d34594b82d3a8e63 in_seq=0 out_seq=0 2024-04-26T20:22:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.881+0000 7f8f43400640 10 client.15951 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.881+0000 7f8f43400640 1 -- 172.21.15.131:0/3783289993 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55942c659f90 con 0x55942c6fdb40 2024-04-26T20:22:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.881+0000 7f8f43400640 10 client.15951 handle_command_reply: tid=0 2024-04-26T20:22:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.881+0000 7f8f67a0a740 1 client.15951 shutdown 2024-04-26T20:22:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 msgr2=0x55942c6f24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f24e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8f48012700 tx=0x7f8f48012bc0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 shutdown_connections 2024-04-26T20:22:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c6fdb40 0x55942c6fff90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6ed940 0x55942c6f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f24e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/3783289993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6e7de0 0x55942c6f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f43400640 0 client.15951 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 >> 172.21.15.131:0/3783289993 conn(0x55942c6e3450 msgr2=0x55942c6e2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 shutdown_connections 2024-04-26T20:22:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 1 -- 172.21.15.131:0/3783289993 wait complete. 2024-04-26T20:22:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.882+0000 7f8f67a0a740 20 client.15951 trim_cache size 0 max 16384 2024-04-26T20:22:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 Processor -- start 2024-04-26T20:22:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 -- start start 2024-04-26T20:22:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e7de0 0x55942c6e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e87d0 0x55942c6f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c704890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b4260 con 0x55942c6e7de0 2024-04-26T20:22:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b1b30 con 0x55942c6ed940 2024-04-26T20:22:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.898+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601af980 con 0x55942c6e87d0 2024-04-26T20:22:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e7de0 0x55942c6e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e7de0 0x55942c6e6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37418/0 (socket says 172.21.15.131:37418) 2024-04-26T20:22:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 -- 172.21.15.131:0/1779775936 learned_addr learned my addr 172.21.15.131:0/1779775936 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f57e00640 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e87d0 0x55942c6f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c704890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 -- 172.21.15.131:0/1779775936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e87d0 msgr2=0x55942c6f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e87d0 0x55942c6f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 -- 172.21.15.131:0/1779775936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 msgr2=0x55942c704890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c704890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 -- 172.21.15.131:0/1779775936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942c42a620 con 0x55942c6e7de0 2024-04-26T20:22:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f57e00640 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e87d0 0x55942c6f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.899+0000 7f8f5e800640 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e7de0 0x55942c6e6ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006010 tx=0x7f8f58030970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e7c1f7e14250a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f4e600640 1 -- 172.21.15.131:0/1779775936 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58036040 con 0x55942c6e7de0 2024-04-26T20:22:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f4e600640 1 -- 172.21.15.131:0/1779775936 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f58003da0 con 0x55942c6e7de0 2024-04-26T20:22:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f4e600640 1 -- 172.21.15.131:0/1779775936 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58041040 con 0x55942c6e7de0 2024-04-26T20:22:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 -- 172.21.15.131:0/1779775936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e7de0 msgr2=0x55942c6e6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e7de0 0x55942c6e6ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006010 tx=0x7f8f58030970 comp rx=0 tx=0).stop 2024-04-26T20:22:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 -- 172.21.15.131:0/1779775936 shutdown_connections 2024-04-26T20:22:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c704890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e87d0 0x55942c6f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/1779775936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e7de0 0x55942c6e6ae0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 -- 172.21.15.131:0/1779775936 >> 172.21.15.131:0/1779775936 conn(0x55942c6e3450 msgr2=0x55942c6e63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.900+0000 7f8f67a0a740 1 -- 172.21.15.131:0/1779775936 shutdown_connections 2024-04-26T20:22:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 -- 172.21.15.131:0/1779775936 wait complete. 2024-04-26T20:22:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 -- start start 2024-04-26T20:22:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 Processor -- start 2024-04-26T20:22:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e7de0 0x55942c6f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c6f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601afff0 con 0x55942c6e87d0 2024-04-26T20:22:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b1460 con 0x55942c6ed940 2024-04-26T20:22:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.901+0000 7f8f67a0a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b06c0 con 0x55942c6e7de0 2024-04-26T20:22:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f67a0a740 10 client.0 _ll_register_callbacks cb 0x7f8f48005b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37434/0 (socket says 172.21.15.131:37434) 2024-04-26T20:22:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 -- 172.21.15.131:0/4247233742 learned_addr learned my addr 172.21.15.131:0/4247233742 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f5e800640 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e7de0 0x55942c6f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f5e800640 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c6f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 -- 172.21.15.131:0/4247233742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e7de0 msgr2=0x55942c6f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e7de0 0x55942c6f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 -- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 msgr2=0x55942c6f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c6f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 -- 172.21.15.131:0/4247233742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942c606cb0 con 0x55942c6e87d0 2024-04-26T20:22:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f57e00640 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f7550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f4801cad0 tx=0x7f8f48009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f631b73d1b26fd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.902+0000 7f8f43400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:22:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f43400640 1 -- 172.21.15.131:0/4247233742 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4801c630 con 0x55942c6e87d0 2024-04-26T20:22:44.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f43400640 1 -- 172.21.15.131:0/4247233742 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8f48018040 con 0x55942c6e87d0 2024-04-26T20:22:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f58003da0 con 0x55942c6e87d0 2024-04-26T20:22:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f43400640 1 -- 172.21.15.131:0/4247233742 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f48037050 con 0x55942c6e87d0 2024-04-26T20:22:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 10 client.15957 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55942c7643b0 con 0x55942c6e87d0 2024-04-26T20:22:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f43400640 1 -- 172.21.15.131:0/4247233742 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f58003da0 con 0x55942c6e87d0 2024-04-26T20:22:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f43400640 1 -- 172.21.15.131:0/4247233742 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8f48004660 con 0x55942c6e87d0 2024-04-26T20:22:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 20 client.15957 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c764e30 0x55942c767250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 4 client.15957 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:22:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.903+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55942c653e00 con 0x55942c764e30 2024-04-26T20:22:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.904+0000 7f8f5e800640 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c764e30 0x55942c767250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.904+0000 7f8f43400640 10 client.15957 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.904+0000 7f8f5e800640 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c764e30 0x55942c767250 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99515bcbf38055c7 server_cookie=e8a86a82ef3393a7 in_seq=0 out_seq=0 2024-04-26T20:22:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f43400640 1 -- 172.21.15.131:0/4247233742 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55942c653e00 con 0x55942c764e30 2024-04-26T20:22:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f43400640 10 client.15957 handle_command_reply: tid=0 2024-04-26T20:22:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 client.15957 shutdown 2024-04-26T20:22:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 msgr2=0x55942c6f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f7550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f4801cad0 tx=0x7f8f48009d80 comp rx=0 tx=0).stop 2024-04-26T20:22:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 shutdown_connections 2024-04-26T20:22:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55942c764e30 0x55942c767250 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55942c6ed940 0x55942c6f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55942c6e87d0 0x55942c6f7550 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f43400640 0 client.15957 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 --2- 172.21.15.131:0/4247233742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942c6e7de0 0x55942c6f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 >> 172.21.15.131:0/4247233742 conn(0x55942c6e3450 msgr2=0x55942c6e1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 shutdown_connections 2024-04-26T20:22:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 1 -- 172.21.15.131:0/4247233742 wait complete. 2024-04-26T20:22:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.908+0000 7f8f67a0a740 20 client.15957 trim_cache size 0 max 16384 2024-04-26T20:22:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:22:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:22:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:22:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:22:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8f440a3c30 msgr2=0x7f8f440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8f440a3c30 0x7f8f440a60f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55942c6e8bd0 tx=0x7f8f500d6530 comp rx=0 tx=0).stop 2024-04-26T20:22:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60108de0 msgr2=0x7f8f6019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60108de0 0x7f8f6019ef90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e9c0 tx=0x7f8f5000ee80 comp rx=0 tx=0).stop 2024-04-26T20:22:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 shutdown_connections 2024-04-26T20:22:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8f440a3c30 0x7f8f440a60f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f6010a990 0x7f8f601a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f60109fd0 0x7f8f6019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 --2- 172.21.15.131:0/1711366326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60108de0 0x7f8f6019ef90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 >> 172.21.15.131:0/1711366326 conn(0x7f8f60077dc0 msgr2=0x7f8f6010d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 shutdown_connections 2024-04-26T20:22:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:15.909+0000 7f8f5f200640 1 -- 172.21.15.131:0/1711366326 wait complete. 2024-04-26T20:22:46.584 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:35 smithi157 ceph-mon[32703]: pgmap v787: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:46.591 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:37 smithi157 ceph-mon[32703]: pgmap v788: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:46.682 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:39 smithi157 ceph-mon[32703]: pgmap v789: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:46.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:41 smithi157 ceph-mon[32703]: pgmap v790: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:46.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:43 smithi157 ceph-mon[32703]: pgmap v791: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:46.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:45 smithi157 ceph-mon[32703]: pgmap v792: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:46.788 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:35 smithi148 ceph-mon[32281]: pgmap v787: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:46.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:37 smithi148 ceph-mon[32281]: pgmap v788: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:46.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:39 smithi148 ceph-mon[32281]: pgmap v789: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:46.909 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:41 smithi148 ceph-mon[32281]: pgmap v790: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:46.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:43 smithi148 ceph-mon[32281]: pgmap v791: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:47.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:45 smithi148 ceph-mon[32281]: pgmap v792: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:47.024 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:15 smithi131 ceph-mon[27919]: pgmap v717: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:47.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:16 smithi131 ceph-mon[27919]: pgmap v718: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:47.101 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:19 smithi131 ceph-mon[27919]: pgmap v719: 129 pgs: 129 active+clean; 680 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-26T20:22:47.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:21 smithi131 ceph-mon[27919]: pgmap v720: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:47.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:23 smithi131 ceph-mon[27919]: pgmap v721: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:47.222 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:25 smithi131 ceph-mon[27919]: pgmap v722: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:47.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:27 smithi131 ceph-mon[27919]: pgmap v723: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 -- 172.21.15.131:0/2286318028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8103b90 msgr2=0x7fdee8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 --2- 172.21.15.131:0/2286318028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8103b90 0x7fdee8104010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fded4009900 tx=0x7fded402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 -- 172.21.15.131:0/2286318028 shutdown_connections 2024-04-26T20:22:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 --2- 172.21.15.131:0/2286318028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee8104550 0x7fdee810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 --2- 172.21.15.131:0/2286318028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8103b90 0x7fdee8104010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 --2- 172.21.15.131:0/2286318028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee81029a0 0x7fdee8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 -- 172.21.15.131:0/2286318028 >> 172.21.15.131:0/2286318028 conn(0x7fdee80fdef0 msgr2=0x7fdee8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 -- 172.21.15.131:0/2286318028 shutdown_connections 2024-04-26T20:22:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.050+0000 7fdeef200640 1 -- 172.21.15.131:0/2286318028 wait complete. 2024-04-26T20:22:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.051+0000 7fdeef200640 1 Processor -- start 2024-04-26T20:22:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.051+0000 7fdeef200640 1 -- start start 2024-04-26T20:22:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.051+0000 7fdeef200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee81029a0 0x7fdee819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.051+0000 7fdeef200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee8103b90 0x7fdee819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdeef200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8104550 0x7fdee81a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdeef200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdee810e520 con 0x7fdee8103b90 2024-04-26T20:22:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdeef200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdee810e3a0 con 0x7fdee81029a0 2024-04-26T20:22:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdeef200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdee810e6a0 con 0x7fdee8104550 2024-04-26T20:22:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee8103b90 0x7fdee819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee81029a0 0x7fdee819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee81029a0 0x7fdee819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50100/0 (socket says 172.21.15.131:50100) 2024-04-26T20:22:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 -- 172.21.15.131:0/4075957776 learned_addr learned my addr 172.21.15.131:0/4075957776 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdeed200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8104550 0x7fdee81a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 -- 172.21.15.131:0/4075957776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8104550 msgr2=0x7fdee81a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8104550 0x7fdee81a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 -- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee8103b90 msgr2=0x7fdee819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee8103b90 0x7fdee819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.052+0000 7fdee7e00640 1 -- 172.21.15.131:0/4075957776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee81a2950 con 0x7fdee81029a0 2024-04-26T20:22:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.053+0000 7fdee7e00640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee81029a0 0x7fdee819ad40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fded800e9c0 tx=0x7fded800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.053+0000 7fdee4c00640 1 -- 172.21.15.131:0/4075957776 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded800cd80 con 0x7fdee81029a0 2024-04-26T20:22:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.053+0000 7fdee4c00640 1 -- 172.21.15.131:0/4075957776 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded80035f0 con 0x7fdee81029a0 2024-04-26T20:22:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.053+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee81a2c40 con 0x7fdee81029a0 2024-04-26T20:22:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.053+0000 7fdee4c00640 1 -- 172.21.15.131:0/4075957776 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8010690 con 0x7fdee81029a0 2024-04-26T20:22:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.053+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee81a3190 con 0x7fdee81029a0 2024-04-26T20:22:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.056+0000 7fdee4c00640 1 -- 172.21.15.131:0/4075957776 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fded80037a0 con 0x7fdee81029a0 2024-04-26T20:22:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.056+0000 7fdef21c2740 1 Processor -- start 2024-04-26T20:22:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.056+0000 7fdef21c2740 1 -- start start 2024-04-26T20:22:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96abd0 0x5600da96aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da970730 0x5600da972bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdef21c2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5600da8cf380 con 0x5600da96abd0 2024-04-26T20:22:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdef21c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5600da86bc30 con 0x5600da96b5c0 2024-04-26T20:22:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdef21c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5600da8cb340 con 0x5600da970730 2024-04-26T20:22:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da970730 0x5600da972bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdeed200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96abd0 0x5600da96aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da970730 0x5600da972bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59476/0 (socket says 172.21.15.131:59476) 2024-04-26T20:22:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 -- 172.21.15.131:0/987359177 learned_addr learned my addr 172.21.15.131:0/987359177 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 -- 172.21.15.131:0/987359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 msgr2=0x5600da9700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 -- 172.21.15.131:0/987359177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96abd0 msgr2=0x5600da96aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96abd0 0x5600da96aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7400640 1 -- 172.21.15.131:0/987359177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600da88d940 con 0x5600da970730 2024-04-26T20:22:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.057+0000 7fdee7e00640 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.058+0000 7fdee7400640 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da970730 0x5600da972bc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fded4006010 tx=0x7fded4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bef5adebeb06db49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.058+0000 7fdee4c00640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdecc0a3d70 0x7fdecc0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.058+0000 7fdedd600640 1 -- 172.21.15.131:0/987359177 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4035030 con 0x5600da970730 2024-04-26T20:22:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.058+0000 7fdedd600640 1 -- 172.21.15.131:0/987359177 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded4040070 con 0x5600da970730 2024-04-26T20:22:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.058+0000 7fdedd600640 1 -- 172.21.15.131:0/987359177 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4031890 con 0x5600da970730 2024-04-26T20:22:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.058+0000 7fdee4c00640 1 -- 172.21.15.131:0/4075957776 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fded80d0d70 con 0x7fdee81029a0 2024-04-26T20:22:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.058+0000 7fdeed200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdecc0a3d70 0x7fdecc0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 -- 172.21.15.131:0/987359177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da970730 msgr2=0x5600da972bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da970730 0x5600da972bc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fded4006010 tx=0x7fded4004040 comp rx=0 tx=0).stop 2024-04-26T20:22:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 -- 172.21.15.131:0/987359177 shutdown_connections 2024-04-26T20:22:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da970730 0x5600da972bc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 --2- 172.21.15.131:0/987359177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96abd0 0x5600da96aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 -- 172.21.15.131:0/987359177 >> 172.21.15.131:0/987359177 conn(0x5600da966240 msgr2=0x5600da966640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 -- 172.21.15.131:0/987359177 shutdown_connections 2024-04-26T20:22:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdeed200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdecc0a3d70 0x7fdecc0a6230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5600da96be20 tx=0x7fdee0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.059+0000 7fdef21c2740 1 -- 172.21.15.131:0/987359177 wait complete. 2024-04-26T20:22:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 -- start start 2024-04-26T20:22:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 Processor -- start 2024-04-26T20:22:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da974e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da970730 0x5600da97c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5600da973680 con 0x5600da970730 2024-04-26T20:22:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5600da856c60 con 0x5600da96b5c0 2024-04-26T20:22:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5600da973800 con 0x5600da96abd0 2024-04-26T20:22:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.060+0000 7fdef21c2740 10 client.0 _ll_register_callbacks cb 0x5600da882e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdeed200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da970730 0x5600da97c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50116/0 (socket says 172.21.15.131:50116) 2024-04-26T20:22:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da974e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 -- 172.21.15.131:0/3228214726 learned_addr learned my addr 172.21.15.131:0/3228214726 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 -- 172.21.15.131:0/3228214726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 msgr2=0x5600da974e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da974e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 -- 172.21.15.131:0/3228214726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da970730 msgr2=0x5600da97c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da970730 0x5600da97c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 -- 172.21.15.131:0/3228214726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600da865510 con 0x5600da96b5c0 2024-04-26T20:22:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdeed200640 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da970730 0x5600da97c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7e00640 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da974e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:22:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fdee7400640 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9753b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fded40092a0 tx=0x7fded4004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237e3b74daea74e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.061+0000 7fded2000640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:22:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fded2000640 1 -- 172.21.15.131:0/3228214726 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4035050 con 0x5600da96b5c0 2024-04-26T20:22:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded4040070 con 0x5600da96b5c0 2024-04-26T20:22:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fded2000640 1 -- 172.21.15.131:0/3228214726 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fded4003a00 con 0x5600da96b5c0 2024-04-26T20:22:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fded2000640 1 -- 172.21.15.131:0/3228214726 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fded405b050 con 0x5600da96b5c0 2024-04-26T20:22:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fdef21c2740 10 client.26090 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600da5e1210 con 0x5600da96b5c0 2024-04-26T20:22:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fded2000640 1 -- 172.21.15.131:0/3228214726 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded4040070 con 0x5600da96b5c0 2024-04-26T20:22:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fded2000640 1 -- 172.21.15.131:0/3228214726 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fded4056030 con 0x5600da96b5c0 2024-04-26T20:22:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.062+0000 7fdef21c2740 10 client.26090 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fdef21c2740 10 client.26090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fdef21c2740 20 client.26090 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da9804e0 0x5600da9828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fdef21c2740 4 client.26090 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:22:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600da8dcf90 con 0x5600da9804e0 2024-04-26T20:22:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fdee7e00640 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da9804e0 0x5600da9828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fdee7e00640 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da9804e0 0x5600da9828d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea88a6e038e89f75 server_cookie=a5b5ae8c5ed8cfa2 in_seq=0 out_seq=0 2024-04-26T20:22:52.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.063+0000 7fded2000640 10 client.26090 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fded2000640 1 -- 172.21.15.131:0/3228214726 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600da8dcf90 con 0x5600da9804e0 2024-04-26T20:22:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fded2000640 10 client.26090 handle_command_reply: tid=0 2024-04-26T20:22:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 client.26090 shutdown 2024-04-26T20:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 msgr2=0x5600da9753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9753b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fded40092a0 tx=0x7fded4004460 comp rx=0 tx=0).stop 2024-04-26T20:22:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 shutdown_connections 2024-04-26T20:22:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da9804e0 0x5600da9828d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da970730 0x5600da97c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96b5c0 0x5600da9753b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3228214726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da974e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fded2000640 0 client.26090 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.064+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 >> 172.21.15.131:0/3228214726 conn(0x5600da966240 msgr2=0x5600da965440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.065+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 shutdown_connections 2024-04-26T20:22:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.065+0000 7fdef21c2740 1 -- 172.21.15.131:0/3228214726 wait complete. 2024-04-26T20:22:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.065+0000 7fdef21c2740 20 client.26090 trim_cache size 0 max 16384 2024-04-26T20:22:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 Processor -- start 2024-04-26T20:22:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 -- start start 2024-04-26T20:22:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da966240 0x5600da9690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da97c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da969600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81b00b0 con 0x5600da96b5c0 2024-04-26T20:22:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81ad980 con 0x5600da966240 2024-04-26T20:22:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdef21c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81ab7d0 con 0x5600da96abd0 2024-04-26T20:22:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdeed200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da969600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdeed200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da969600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55332/0 (socket says 172.21.15.131:55332) 2024-04-26T20:22:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdeed200640 1 -- 172.21.15.131:0/1760724161 learned_addr learned my addr 172.21.15.131:0/1760724161 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.081+0000 7fdee7e00640 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da966240 0x5600da9690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdee7e00640 1 -- 172.21.15.131:0/1760724161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 msgr2=0x5600da97c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdee7e00640 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da97c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdee7e00640 1 -- 172.21.15.131:0/1760724161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 msgr2=0x5600da969600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdee7e00640 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da969600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdee7e00640 1 -- 172.21.15.131:0/1760724161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600da6ad620 con 0x5600da966240 2024-04-26T20:22:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdee7e00640 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da966240 0x5600da9690c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fded80ce2d0 tx=0x7fded80d31c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f6a41da548dcf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdedcc00640 1 -- 172.21.15.131:0/1760724161 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded809c400 con 0x5600da966240 2024-04-26T20:22:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdedcc00640 1 -- 172.21.15.131:0/1760724161 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded809d1a0 con 0x5600da966240 2024-04-26T20:22:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.082+0000 7fdedcc00640 1 -- 172.21.15.131:0/1760724161 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded809abe0 con 0x5600da966240 2024-04-26T20:22:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 -- 172.21.15.131:0/1760724161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da966240 msgr2=0x5600da9690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da966240 0x5600da9690c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fded80ce2d0 tx=0x7fded80d31c0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 -- 172.21.15.131:0/1760724161 shutdown_connections 2024-04-26T20:22:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da969600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da96abd0 0x5600da97c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 --2- 172.21.15.131:0/1760724161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da966240 0x5600da9690c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 -- 172.21.15.131:0/1760724161 >> 172.21.15.131:0/1760724161 conn(0x5600da970200 msgr2=0x5600da96c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 -- 172.21.15.131:0/1760724161 shutdown_connections 2024-04-26T20:22:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 -- 172.21.15.131:0/1760724161 wait complete. 2024-04-26T20:22:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.083+0000 7fdef21c2740 1 -- start start 2024-04-26T20:22:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 1 Processor -- start 2024-04-26T20:22:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da966240 0x5600da979480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96abd0 0x5600da9799c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da97a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81abe40 con 0x5600da96b5c0 2024-04-26T20:22:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81ad2b0 con 0x5600da96abd0 2024-04-26T20:22:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81ac510 con 0x5600da966240 2024-04-26T20:22:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdef21c2740 10 client.0 _ll_register_callbacks cb 0x7fdee0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:22:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdeed200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da97a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdeed200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da97a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55334/0 (socket says 172.21.15.131:55334) 2024-04-26T20:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdeed200640 1 -- 172.21.15.131:0/3807825865 learned_addr learned my addr 172.21.15.131:0/3807825865 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdee7e00640 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da966240 0x5600da979480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.084+0000 7fdee7400640 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96abd0 0x5600da9799c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fdeed200640 1 -- 172.21.15.131:0/3807825865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da966240 msgr2=0x5600da979480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fdeed200640 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da966240 0x5600da979480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fdeed200640 1 -- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96abd0 msgr2=0x5600da9799c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:22:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fdeed200640 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96abd0 0x5600da9799c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fdeed200640 1 -- 172.21.15.131:0/3807825865 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600da889cb0 con 0x5600da96b5c0 2024-04-26T20:22:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fdeed200640 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da97a980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdee000f990 tx=0x7fdee0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76ec299139b4429 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:22:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fded2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:22:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.085+0000 7fded2000640 1 -- 172.21.15.131:0/3807825865 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee002d020 con 0x5600da96b5c0 2024-04-26T20:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded809d1a0 con 0x5600da96b5c0 2024-04-26T20:22:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fded2000640 1 -- 172.21.15.131:0/3807825865 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fdee0004310 con 0x5600da96b5c0 2024-04-26T20:22:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fded2000640 1 -- 172.21.15.131:0/3807825865 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdee0038050 con 0x5600da96b5c0 2024-04-26T20:22:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-04-26T20:22:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600da97c1d0 con 0x5600da96b5c0 2024-04-26T20:22:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fded2000640 1 -- 172.21.15.131:0/3807825865 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded809d1a0 con 0x5600da96b5c0 2024-04-26T20:22:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fded2000640 1 -- 172.21.15.131:0/3807825865 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdee0029c00 con 0x5600da96b5c0 2024-04-26T20:22:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:22:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:22:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 20 client.15975 populate_metadata read hostname 'smithi131' 2024-04-26T20:22:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da970730 0x5600da9e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:22:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 4 client.15975 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:22:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.086+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600da8d6e00 con 0x5600da970730 2024-04-26T20:22:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.087+0000 7fdee7e00640 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da970730 0x5600da9e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:22:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.087+0000 7fdee7e00640 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da970730 0x5600da9e8db0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a57668116b8a514 server_cookie=ce6fc47adddd20cc in_seq=0 out_seq=0 2024-04-26T20:22:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.087+0000 7fded2000640 10 client.15975 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fded2000640 1 -- 172.21.15.131:0/3807825865 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600da8d6e00 con 0x5600da970730 2024-04-26T20:22:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fded2000640 10 client.15975 handle_command_reply: tid=0 2024-04-26T20:22:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 client.15975 shutdown 2024-04-26T20:22:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 msgr2=0x5600da97a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da97a980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdee000f990 tx=0x7fdee0004ab0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 shutdown_connections 2024-04-26T20:22:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5600da970730 0x5600da9e8db0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600da96b5c0 0x5600da97a980 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600da96abd0 0x5600da9799c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fded2000640 0 client.15975 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:22:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 --2- 172.21.15.131:0/3807825865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600da966240 0x5600da979480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 >> 172.21.15.131:0/3807825865 conn(0x5600da970200 msgr2=0x5600da96c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 shutdown_connections 2024-04-26T20:22:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 1 -- 172.21.15.131:0/3807825865 wait complete. 2024-04-26T20:22:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.091+0000 7fdef21c2740 20 client.15975 trim_cache size 0 max 16384 2024-04-26T20:22:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:22:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:22:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdecc0a3d70 msgr2=0x7fdecc0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdecc0a3d70 0x7fdecc0a6230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5600da96be20 tx=0x7fdee0009500 comp rx=0 tx=0).stop 2024-04-26T20:22:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee81029a0 msgr2=0x7fdee819ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee81029a0 0x7fdee819ad40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fded800e9c0 tx=0x7fded800ee80 comp rx=0 tx=0).stop 2024-04-26T20:22:57.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 shutdown_connections 2024-04-26T20:22:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fdecc0a3d70 0x7fdecc0a6230 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdee8104550 0x7fdee81a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee8103b90 0x7fdee819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 --2- 172.21.15.131:0/4075957776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee81029a0 0x7fdee819ad40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 >> 172.21.15.131:0/4075957776 conn(0x7fdee80fdef0 msgr2=0x7fdee8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:22:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 shutdown_connections 2024-04-26T20:22:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:29.092+0000 7fdeef200640 1 -- 172.21.15.131:0/4075957776 wait complete. 2024-04-26T20:22:57.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:29 smithi131 ceph-mon[27919]: pgmap v724: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:57.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:31 smithi131 ceph-mon[27919]: pgmap v725: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:57.780 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:33 smithi131 ceph-mon[27919]: pgmap v726: 129 pgs: 129 active+clean; 681 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:57.787 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:35 smithi131 ceph-mon[27919]: pgmap v727: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:57.797 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:37 smithi131 ceph-mon[27919]: pgmap v728: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:57.896 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:39 smithi131 ceph-mon[27919]: pgmap v729: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:57.903 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:41 smithi131 ceph-mon[27919]: pgmap v730: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:57.973 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:47 smithi157 ceph-mon[32703]: pgmap v793: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:57.985 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:49 smithi157 ceph-mon[32703]: pgmap v794: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:57.997 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:51 smithi157 ceph-mon[32703]: pgmap v795: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:58.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:53 smithi157 ceph-mon[32703]: pgmap v796: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:58.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:53 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:58.208 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:58.270 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:58.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:54 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:58.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:55 smithi157 ceph-mon[32703]: pgmap v797: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:58.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:56 smithi157 ceph-mon[32703]: pgmap v798: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:58.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:47 smithi148 ceph-mon[32281]: pgmap v793: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:58.521 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:49 smithi148 ceph-mon[32281]: pgmap v794: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:58.602 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:51 smithi148 ceph-mon[32281]: pgmap v795: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:22:58.709 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:53 smithi148 ceph-mon[32281]: pgmap v796: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:22:58.775 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:53 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:58.864 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:58.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:58.881 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:54 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:22:58.896 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:55 smithi148 ceph-mon[32281]: pgmap v797: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:22:58.907 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:56 smithi148 ceph-mon[32281]: pgmap v798: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:22:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 -- 172.21.15.131:0/4280695672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8105e00 msgr2=0x7f0eb81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:22:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 --2- 172.21.15.131:0/4280695672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8105e00 0x7f0eb81081f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009900 tx=0x7f0ea402a400 comp rx=0 tx=0).stop 2024-04-26T20:22:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 -- 172.21.15.131:0/4280695672 shutdown_connections 2024-04-26T20:22:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 --2- 172.21.15.131:0/4280695672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb8108730 0x7f0eb810ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:22:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 --2- 172.21.15.131:0/4280695672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8105e00 0x7f0eb81081f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 --2- 172.21.15.131:0/4280695672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81034d0 0x7f0eb81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 -- 172.21.15.131:0/4280695672 >> 172.21.15.131:0/4280695672 conn(0x7f0eb80fce40 msgr2=0x7f0eb80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 -- 172.21.15.131:0/4280695672 shutdown_connections 2024-04-26T20:23:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.234+0000 7f0ebf800640 1 -- 172.21.15.131:0/4280695672 wait complete. 2024-04-26T20:23:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 Processor -- start 2024-04-26T20:23:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 -- start start 2024-04-26T20:23:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb81034d0 0x7f0eb819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb8105e00 0x7f0eb819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8108730 0x7f0eb81a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810e500 con 0x7f0eb81034d0 2024-04-26T20:23:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810e380 con 0x7f0eb8105e00 2024-04-26T20:23:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.235+0000 7f0ebf800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810e680 con 0x7f0eb8108730 2024-04-26T20:23:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb81034d0 0x7f0eb819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb81034d0 0x7f0eb819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39018/0 (socket says 172.21.15.131:39018) 2024-04-26T20:23:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 -- 172.21.15.131:0/1824058577 learned_addr learned my addr 172.21.15.131:0/1824058577 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebd800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8108730 0x7f0eb81a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb8105e00 0x7f0eb819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 -- 172.21.15.131:0/1824058577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8108730 msgr2=0x7f0eb81a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8108730 0x7f0eb81a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 -- 172.21.15.131:0/1824058577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb8105e00 msgr2=0x7f0eb819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb8105e00 0x7f0eb819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 -- 172.21.15.131:0/1824058577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb81a2970 con 0x7f0eb81034d0 2024-04-26T20:23:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebd800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8108730 0x7f0eb81a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.236+0000 7f0ebce00640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb81034d0 0x7f0eb819ad50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00ea60 tx=0x7f0eac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.237+0000 7f0eb5600640 1 -- 172.21.15.131:0/1824058577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00cde0 con 0x7f0eb81034d0 2024-04-26T20:23:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.237+0000 7f0eb5600640 1 -- 172.21.15.131:0/1824058577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eac0035f0 con 0x7f0eb81034d0 2024-04-26T20:23:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.237+0000 7f0eb5600640 1 -- 172.21.15.131:0/1824058577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac010690 con 0x7f0eb81034d0 2024-04-26T20:23:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.237+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81a2c60 con 0x7f0eb81034d0 2024-04-26T20:23:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.237+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a3130 con 0x7f0eb81034d0 2024-04-26T20:23:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.240+0000 7f0ec25fb740 1 Processor -- start 2024-04-26T20:23:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.240+0000 7f0eb5600640 1 -- 172.21.15.131:0/1824058577 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0eac0037a0 con 0x7f0eb81034d0 2024-04-26T20:23:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.240+0000 7f0ec25fb740 1 -- start start 2024-04-26T20:23:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8413ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a84190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a841bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d3a8378380 con 0x55d3a84145c0 2024-04-26T20:23:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d3a8314c30 con 0x55d3a8413bd0 2024-04-26T20:23:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d3a8374340 con 0x55d3a8419730 2024-04-26T20:23:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a84190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a84190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39034/0 (socket says 172.21.15.131:39034) 2024-04-26T20:23:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebce00640 1 -- 172.21.15.131:0/425432471 learned_addr learned my addr 172.21.15.131:0/425432471 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a841bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebd800640 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8413ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebce00640 1 -- 172.21.15.131:0/425432471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 msgr2=0x55d3a841bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebce00640 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a841bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebce00640 1 -- 172.21.15.131:0/425432471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 msgr2=0x55d3a8413ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.241+0000 7f0ebce00640 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8413ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0ebce00640 1 -- 172.21.15.131:0/425432471 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a8336940 con 0x55d3a84145c0 2024-04-26T20:23:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0ebd800640 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8413ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0ebce00640 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a84190c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0d4b30 tx=0x7f0eac0d9240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f849ac8d789c1a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0eb5600640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0ea00a3e30 0x7f0ea00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0eaa000640 1 -- 172.21.15.131:0/425432471 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac098b20 con 0x55d3a84145c0 2024-04-26T20:23:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0ebd800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0ea00a3e30 0x7f0ea00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0eaa000640 1 -- 172.21.15.131:0/425432471 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eac0e2070 con 0x55d3a84145c0 2024-04-26T20:23:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0eaa000640 1 -- 172.21.15.131:0/425432471 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac09bcd0 con 0x55d3a84145c0 2024-04-26T20:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.242+0000 7f0eb5600640 1 -- 172.21.15.131:0/1824058577 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0eac0d0b50 con 0x7f0eb81034d0 2024-04-26T20:23:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 -- 172.21.15.131:0/425432471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 msgr2=0x55d3a84190c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ebd800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0ea00a3e30 0x7f0ea00a62f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d3a8414e20 tx=0x7f0eb0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a84190c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0d4b30 tx=0x7f0eac0d9240 comp rx=0 tx=0).stop 2024-04-26T20:23:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 -- 172.21.15.131:0/425432471 shutdown_connections 2024-04-26T20:23:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a841bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a84190c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/425432471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8413ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 -- 172.21.15.131:0/425432471 >> 172.21.15.131:0/425432471 conn(0x55d3a840f240 msgr2=0x55d3a840f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 -- 172.21.15.131:0/425432471 shutdown_connections 2024-04-26T20:23:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.243+0000 7f0ec25fb740 1 -- 172.21.15.131:0/425432471 wait complete. 2024-04-26T20:23:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.244+0000 7f0ec25fb740 1 -- start start 2024-04-26T20:23:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.244+0000 7f0ec25fb740 1 Processor -- start 2024-04-26T20:23:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.244+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a841dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.244+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a84145c0 0x55d3a841e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a8419730 0x55d3a8425260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d3a841c680 con 0x55d3a8419730 2024-04-26T20:23:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d3a82ffc60 con 0x55d3a84145c0 2024-04-26T20:23:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d3a841c800 con 0x55d3a8413bd0 2024-04-26T20:23:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ec25fb740 10 client.0 _ll_register_callbacks cb 0x55d3a832be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ebd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a8419730 0x55d3a8425260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a84145c0 0x55d3a841e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ebd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a8419730 0x55d3a8425260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39048/0 (socket says 172.21.15.131:39048) 2024-04-26T20:23:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ebd800640 1 -- 172.21.15.131:0/1168569379 learned_addr learned my addr 172.21.15.131:0/1168569379 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0ebce00640 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a841dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0eb7e00640 1 -- 172.21.15.131:0/1168569379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 msgr2=0x55d3a841dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a841dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0eb7e00640 1 -- 172.21.15.131:0/1168569379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a8419730 msgr2=0x55d3a8425260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a8419730 0x55d3a8425260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.245+0000 7f0eb7e00640 1 -- 172.21.15.131:0/1168569379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a830e510 con 0x55d3a84145c0 2024-04-26T20:23:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a84145c0 0x55d3a841e290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ea402a920 tx=0x7f0ea40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9932f95361cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0e9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:23:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0e9ea00640 1 -- 172.21.15.131:0/1168569379 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4035040 con 0x55d3a84145c0 2024-04-26T20:23:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eac0e2070 con 0x55d3a84145c0 2024-04-26T20:23:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0e9ea00640 1 -- 172.21.15.131:0/1168569379 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0ea4003a00 con 0x55d3a84145c0 2024-04-26T20:23:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0e9ea00640 1 -- 172.21.15.131:0/1168569379 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ea404d050 con 0x55d3a84145c0 2024-04-26T20:23:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0ec25fb740 10 client.26108 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3a808a210 con 0x55d3a84145c0 2024-04-26T20:23:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.246+0000 7f0e9ea00640 1 -- 172.21.15.131:0/1168569379 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eac0e2070 con 0x55d3a84145c0 2024-04-26T20:23:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0e9ea00640 1 -- 172.21.15.131:0/1168569379 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ea4044070 con 0x55d3a84145c0 2024-04-26T20:23:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0ec25fb740 10 client.26108 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0ec25fb740 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0ec25fb740 20 client.26108 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a84297c0 0x55d3a842bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0ec25fb740 4 client.26108 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:23:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a8385f90 con 0x55d3a84297c0 2024-04-26T20:23:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.247+0000 7f0ebce00640 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a84297c0 0x55d3a842bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.248+0000 7f0ebce00640 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a84297c0 0x55d3a842bbb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c536a6bbf931a76 server_cookie=283f9e6f307745ab in_seq=0 out_seq=0 2024-04-26T20:23:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.248+0000 7f0e9ea00640 10 client.26108 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.248+0000 7f0e9ea00640 1 -- 172.21.15.131:0/1168569379 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3a8385f90 con 0x55d3a84297c0 2024-04-26T20:23:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.248+0000 7f0e9ea00640 10 client.26108 handle_command_reply: tid=0 2024-04-26T20:23:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 client.26108 shutdown 2024-04-26T20:23:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a84145c0 msgr2=0x55d3a841e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a84145c0 0x55d3a841e290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ea402a920 tx=0x7f0ea40043b0 comp rx=0 tx=0).stop 2024-04-26T20:23:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 shutdown_connections 2024-04-26T20:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a84297c0 0x55d3a842bbb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a8419730 0x55d3a8425260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a84145c0 0x55d3a841e290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/1168569379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a841dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 >> 172.21.15.131:0/1168569379 conn(0x55d3a840f240 msgr2=0x55d3a840e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0e9ea00640 0 client.26108 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 shutdown_connections 2024-04-26T20:23:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 1 -- 172.21.15.131:0/1168569379 wait complete. 2024-04-26T20:23:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.249+0000 7f0ec25fb740 20 client.26108 trim_cache size 0 max 16384 2024-04-26T20:23:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 Processor -- start 2024-04-26T20:23:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 -- start start 2024-04-26T20:23:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8411fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8425220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a8412500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81b0070 con 0x55d3a84145c0 2024-04-26T20:23:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81ad940 con 0x55d3a8413bd0 2024-04-26T20:23:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.265+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81ab790 con 0x55d3a8419730 2024-04-26T20:23:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8411fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8411fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:52974/0 (socket says 172.21.15.131:52974) 2024-04-26T20:23:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 -- 172.21.15.131:0/3420754543 learned_addr learned my addr 172.21.15.131:0/3420754543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8425220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebd800640 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a8412500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 -- 172.21.15.131:0/3420754543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 msgr2=0x55d3a8412500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a8412500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 -- 172.21.15.131:0/3420754543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 msgr2=0x55d3a8425220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8425220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 -- 172.21.15.131:0/3420754543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a8156620 con 0x55d3a8413bd0 2024-04-26T20:23:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.266+0000 7f0ebce00640 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8411fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0eac09c580 tx=0x7f0eac0d4600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3313ba30dbfb9cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ea9600640 1 -- 172.21.15.131:0/3420754543 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac09c000 con 0x55d3a8413bd0 2024-04-26T20:23:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ea9600640 1 -- 172.21.15.131:0/3420754543 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eac0e2070 con 0x55d3a8413bd0 2024-04-26T20:23:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ea9600640 1 -- 172.21.15.131:0/3420754543 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac09c1e0 con 0x55d3a8413bd0 2024-04-26T20:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 -- 172.21.15.131:0/3420754543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 msgr2=0x55d3a8411fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8411fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0eac09c580 tx=0x7f0eac0d4600 comp rx=0 tx=0).stop 2024-04-26T20:23:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 -- 172.21.15.131:0/3420754543 shutdown_connections 2024-04-26T20:23:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8419730 0x55d3a8412500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8425220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/3420754543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8413bd0 0x55d3a8411fc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 -- 172.21.15.131:0/3420754543 >> 172.21.15.131:0/3420754543 conn(0x55d3a840f240 msgr2=0x55d3a84157a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 -- 172.21.15.131:0/3420754543 shutdown_connections 2024-04-26T20:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.267+0000 7f0ec25fb740 1 -- 172.21.15.131:0/3420754543 wait complete. 2024-04-26T20:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 -- start start 2024-04-26T20:23:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 Processor -- start 2024-04-26T20:23:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a8422330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8422870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8419730 0x55d3a8423880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81abe00 con 0x55d3a84145c0 2024-04-26T20:23:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81ad270 con 0x55d3a8419730 2024-04-26T20:23:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81ac4d0 con 0x55d3a8413bd0 2024-04-26T20:23:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.268+0000 7f0ec25fb740 10 client.0 _ll_register_callbacks cb 0x7f0ea4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8422870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8422870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39064/0 (socket says 172.21.15.131:39064) 2024-04-26T20:23:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 -- 172.21.15.131:0/171813108 learned_addr learned my addr 172.21.15.131:0/171813108 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0ebce00640 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a8422330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0ebd800640 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8419730 0x55d3a8423880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 -- 172.21.15.131:0/171813108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 msgr2=0x55d3a8422330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a8422330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 -- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8419730 msgr2=0x55d3a8423880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8419730 0x55d3a8423880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 -- 172.21.15.131:0/171813108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a8332cb0 con 0x55d3a84145c0 2024-04-26T20:23:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0eb7e00640 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8422870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009900 tx=0x7f0ea403b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b21d17e263edf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.269+0000 7f0e9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:23:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0e9ea00640 1 -- 172.21.15.131:0/171813108 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4035040 con 0x55d3a84145c0 2024-04-26T20:23:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0e9ea00640 1 -- 172.21.15.131:0/171813108 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0ea403bcc0 con 0x55d3a84145c0 2024-04-26T20:23:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eac0e2070 con 0x55d3a84145c0 2024-04-26T20:23:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0e9ea00640 1 -- 172.21.15.131:0/171813108 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ea4005d50 con 0x55d3a84145c0 2024-04-26T20:23:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 10 client.15993 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3a8425050 con 0x55d3a84145c0 2024-04-26T20:23:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0e9ea00640 1 -- 172.21.15.131:0/171813108 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eac0e2070 con 0x55d3a84145c0 2024-04-26T20:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0e9ea00640 1 -- 172.21.15.131:0/171813108 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ea4044040 con 0x55d3a84145c0 2024-04-26T20:23:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 20 client.15993 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a8490070 0x55d3a8492460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 4 client.15993 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:23:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.270+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3a837fe00 con 0x55d3a8490070 2024-04-26T20:23:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.271+0000 7f0ebce00640 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a8490070 0x55d3a8492460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.271+0000 7f0ebce00640 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a8490070 0x55d3a8492460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54531802654f9551 server_cookie=3cda0510a79deba2 in_seq=0 out_seq=0 2024-04-26T20:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.271+0000 7f0e9ea00640 10 client.15993 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0e9ea00640 1 -- 172.21.15.131:0/171813108 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3a837fe00 con 0x55d3a8490070 2024-04-26T20:23:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0e9ea00640 10 client.15993 handle_command_reply: tid=0 2024-04-26T20:23:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 client.15993 shutdown 2024-04-26T20:23:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 msgr2=0x55d3a8422870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8422870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009900 tx=0x7f0ea403b6f0 comp rx=0 tx=0).stop 2024-04-26T20:23:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 shutdown_connections 2024-04-26T20:23:10.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d3a8490070 0x55d3a8492460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d3a8419730 0x55d3a8423880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3a84145c0 0x55d3a8422870 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0e9ea00640 0 client.15993 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 --2- 172.21.15.131:0/171813108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3a8413bd0 0x55d3a8422330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.275+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 >> 172.21.15.131:0/171813108 conn(0x55d3a840f240 msgr2=0x55d3a8411fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 shutdown_connections 2024-04-26T20:23:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ec25fb740 1 -- 172.21.15.131:0/171813108 wait complete. 2024-04-26T20:23:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ec25fb740 20 client.15993 trim_cache size 0 max 16384 2024-04-26T20:23:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:23:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:23:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:23:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:23:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0ea00a3e30 msgr2=0x7f0ea00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0ea00a3e30 0x7f0ea00a62f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d3a8414e20 tx=0x7f0eb0009500 comp rx=0 tx=0).stop 2024-04-26T20:23:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb81034d0 msgr2=0x7f0eb819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb81034d0 0x7f0eb819ad50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00ea60 tx=0x7f0eac00ef20 comp rx=0 tx=0).stop 2024-04-26T20:23:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 shutdown_connections 2024-04-26T20:23:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0ea00a3e30 0x7f0ea00a62f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb8108730 0x7f0eb81a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb8105e00 0x7f0eb819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 --2- 172.21.15.131:0/1824058577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0eb81034d0 0x7f0eb819ad50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.276+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 >> 172.21.15.131:0/1824058577 conn(0x7f0eb80fce40 msgr2=0x7f0eb8102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.277+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 shutdown_connections 2024-04-26T20:23:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:42.277+0000 7f0ebf800640 1 -- 172.21.15.131:0/1824058577 wait complete. 2024-04-26T20:23:11.508 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:22:59 smithi148 ceph-mon[32281]: pgmap v799: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:23:11.523 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:01 smithi148 ceph-mon[32281]: pgmap v800: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:11.529 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:03 smithi148 ceph-mon[32281]: pgmap v801: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:11.535 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:05 smithi148 ceph-mon[32281]: pgmap v802: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:11.541 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:07 smithi148 ceph-mon[32281]: pgmap v803: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:11.593 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:09 smithi148 ceph-mon[32281]: pgmap v804: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:23:11.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:22:59 smithi157 ceph-mon[32703]: pgmap v799: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:23:11.743 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:01 smithi157 ceph-mon[32703]: pgmap v800: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:11.841 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:03 smithi157 ceph-mon[32703]: pgmap v801: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:11.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:05 smithi157 ceph-mon[32703]: pgmap v802: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:11.959 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:07 smithi157 ceph-mon[32703]: pgmap v803: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:11.965 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:09 smithi157 ceph-mon[32703]: pgmap v804: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:23:11.972 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:11 smithi157 ceph-mon[32703]: pgmap v805: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:23:11.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:43 smithi131 ceph-mon[27919]: pgmap v731: 129 pgs: 129 active+clean; 682 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:11.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:45 smithi131 ceph-mon[27919]: pgmap v732: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:23:11.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:47 smithi131 ceph-mon[27919]: pgmap v733: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:12.061 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:49 smithi131 ceph-mon[27919]: pgmap v734: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:12.137 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:51 smithi131 ceph-mon[27919]: pgmap v735: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:12.146 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:51 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:23:12.154 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:51 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:23:12.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:51 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:23:12.265 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:51 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:23:12.325 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:51 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:23:12.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:23:12.451 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:23:12.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:23:12.638 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:53 smithi131 ceph-mon[27919]: pgmap v736: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:12.646 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:11 smithi148 ceph-mon[32281]: pgmap v805: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:23:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 -- 172.21.15.131:0/4078036588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 msgr2=0x7f6fb4103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 --2- 172.21.15.131:0/4078036588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 0x7f6fb4103e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009900 tx=0x7f6fa002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:23:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 -- 172.21.15.131:0/4078036588 shutdown_connections 2024-04-26T20:23:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 --2- 172.21.15.131:0/4078036588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb4104370 0x7f6fb410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 --2- 172.21.15.131:0/4078036588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 0x7f6fb4103e30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 --2- 172.21.15.131:0/4078036588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb41027c0 0x7f6fb4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 -- 172.21.15.131:0/4078036588 >> 172.21.15.131:0/4078036588 conn(0x7f6fb40fdd30 msgr2=0x7f6fb4100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 -- 172.21.15.131:0/4078036588 shutdown_connections 2024-04-26T20:23:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.419+0000 7f6fbac00640 1 -- 172.21.15.131:0/4078036588 wait complete. 2024-04-26T20:23:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.420+0000 7f6fbac00640 1 Processor -- start 2024-04-26T20:23:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fbac00640 1 -- start start 2024-04-26T20:23:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fbac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb41027c0 0x7f6fb419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fbac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 0x7f6fb419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fbac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb4104370 0x7f6fb41a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fbac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb410e2f0 con 0x7f6fb4104370 2024-04-26T20:23:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fbac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb410e170 con 0x7f6fb41039b0 2024-04-26T20:23:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fbac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb410e470 con 0x7f6fb41027c0 2024-04-26T20:23:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fba200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb4104370 0x7f6fb41a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fba200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb4104370 0x7f6fb41a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35382/0 (socket says 172.21.15.131:35382) 2024-04-26T20:23:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb41027c0 0x7f6fb419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fba200640 1 -- 172.21.15.131:0/1475783896 learned_addr learned my addr 172.21.15.131:0/1475783896 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.421+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 0x7f6fb419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fba200640 1 -- 172.21.15.131:0/1475783896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb41027c0 msgr2=0x7f6fb419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fba200640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb41027c0 0x7f6fb419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fba200640 1 -- 172.21.15.131:0/1475783896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 msgr2=0x7f6fb419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fba200640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 0x7f6fb419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fba200640 1 -- 172.21.15.131:0/1475783896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb41a27f0 con 0x7f6fb4104370 2024-04-26T20:23:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fb9800640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb41027c0 0x7f6fb419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fba200640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb4104370 0x7f6fb41a2180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e970 tx=0x7f6fac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fb2000640 1 -- 172.21.15.131:0/1475783896 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00ccc0 con 0x7f6fb4104370 2024-04-26T20:23:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fb2000640 1 -- 172.21.15.131:0/1475783896 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fac00ce60 con 0x7f6fb4104370 2024-04-26T20:23:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb41a2ae0 con 0x7f6fb4104370 2024-04-26T20:23:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fb2000640 1 -- 172.21.15.131:0/1475783896 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac010430 con 0x7f6fb4104370 2024-04-26T20:23:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.422+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb41a3000 con 0x7f6fb4104370 2024-04-26T20:23:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 Processor -- start 2024-04-26T20:23:14.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fb2000640 1 -- 172.21.15.131:0/1475783896 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6fac004f90 con 0x7f6fb4104370 2024-04-26T20:23:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 -- start start 2024-04-26T20:23:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbda9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f8cbd0d380 con 0x55f8cbda8de0 2024-04-26T20:23:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f8cbca9c30 con 0x55f8cbda97d0 2024-04-26T20:23:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.426+0000 7f6fbef99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f8cbd09340 con 0x55f8cbdae940 2024-04-26T20:23:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbda9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbda9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35384/0 (socket says 172.21.15.131:35384) 2024-04-26T20:23:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb8e00640 1 -- 172.21.15.131:0/1212430915 learned_addr learned my addr 172.21.15.131:0/1212430915 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb9800640 1 -- 172.21.15.131:0/1212430915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 msgr2=0x55f8cbdb0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb9800640 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb9800640 1 -- 172.21.15.131:0/1212430915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 msgr2=0x55f8cbda9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb9800640 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbda9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb9800640 1 -- 172.21.15.131:0/1212430915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cbccb940 con 0x55f8cbda97d0 2024-04-26T20:23:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb2000640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f9c0a3c30 0x7f6f9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.427+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.428+0000 7f6fb2000640 1 -- 172.21.15.131:0/1475783896 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f6fac0d0d80 con 0x7f6fb4104370 2024-04-26T20:23:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.428+0000 7f6fb9800640 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdae2d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4103810 tx=0x7f6fa8011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d971a202021b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.428+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f9c0a3c30 0x7f6f9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.428+0000 7f6fa6a00640 1 -- 172.21.15.131:0/1212430915 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8003c30 con 0x55f8cbda97d0 2024-04-26T20:23:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.428+0000 7f6fa6a00640 1 -- 172.21.15.131:0/1212430915 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa800fe20 con 0x55f8cbda97d0 2024-04-26T20:23:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.428+0000 7f6fa6a00640 1 -- 172.21.15.131:0/1212430915 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80193e0 con 0x55f8cbda97d0 2024-04-26T20:23:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 -- 172.21.15.131:0/1212430915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 msgr2=0x55f8cbdae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdae2d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4103810 tx=0x7f6fa8011c70 comp rx=0 tx=0).stop 2024-04-26T20:23:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f9c0a3c30 0x7f6f9c0a60f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55f8cbdaa030 tx=0x7f6fa003f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 -- 172.21.15.131:0/1212430915 shutdown_connections 2024-04-26T20:23:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdae2d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1212430915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbda9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 -- 172.21.15.131:0/1212430915 >> 172.21.15.131:0/1212430915 conn(0x55f8cbda4450 msgr2=0x55f8cbda4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 -- 172.21.15.131:0/1212430915 shutdown_connections 2024-04-26T20:23:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.429+0000 7f6fbef99740 1 -- 172.21.15.131:0/1212430915 wait complete. 2024-04-26T20:23:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.430+0000 7f6fbef99740 1 -- start start 2024-04-26T20:23:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.430+0000 7f6fbef99740 1 Processor -- start 2024-04-26T20:23:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.430+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.430+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fbef99740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f8cbdb18f0 con 0x55f8cbda97d0 2024-04-26T20:23:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fbef99740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f8cbc94c60 con 0x55f8cbda8de0 2024-04-26T20:23:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fbef99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f8cbdb1a70 con 0x55f8cbdae940 2024-04-26T20:23:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fbef99740 10 client.0 _ll_register_callbacks cb 0x55f8cbcc0e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:35634/0 (socket says 172.21.15.131:35634) 2024-04-26T20:23:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb9800640 1 -- 172.21.15.131:0/2888203926 learned_addr learned my addr 172.21.15.131:0/2888203926 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fba200640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb8e00640 1 -- 172.21.15.131:0/2888203926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 msgr2=0x55f8cbdba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb8e00640 1 -- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda8de0 msgr2=0x55f8cbdb2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:17.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb8e00640 1 -- 172.21.15.131:0/2888203926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cbca3510 con 0x55f8cbda97d0 2024-04-26T20:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fba200640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.431+0000 7f6fb9800640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb34b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0005ab0 tx=0x7f6fa00314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a6db5dd27dece9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6f9b400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:23:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6f9b400640 1 -- 172.21.15.131:0/2888203926 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0035040 con 0x55f8cbda97d0 2024-04-26T20:23:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa800fe20 con 0x55f8cbda97d0 2024-04-26T20:23:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6f9b400640 1 -- 172.21.15.131:0/2888203926 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f6fa0003a00 con 0x55f8cbda97d0 2024-04-26T20:23:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6f9b400640 1 -- 172.21.15.131:0/2888203926 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fa005a050 con 0x55f8cbda97d0 2024-04-26T20:23:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6fbef99740 10 client.16005 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.432+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8cba1f210 con 0x55f8cbda97d0 2024-04-26T20:23:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6f9b400640 1 -- 172.21.15.131:0/2888203926 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa800fe20 con 0x55f8cbda97d0 2024-04-26T20:23:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6f9b400640 1 -- 172.21.15.131:0/2888203926 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6fa003bbb0 con 0x55f8cbda97d0 2024-04-26T20:23:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6fbef99740 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6fbef99740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6fbef99740 20 client.16005 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6fbef99740 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbdbeab0 0x55f8cbdc0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6fbef99740 4 client.16005 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:23:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8cbd1af90 con 0x55f8cbdbeab0 2024-04-26T20:23:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.433+0000 7f6fb9800640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbdbeab0 0x55f8cbdc0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.434+0000 7f6fb9800640 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbdbeab0 0x55f8cbdc0f00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5da69cdd57e594 server_cookie=5e0f39ec7b0c2af1 in_seq=0 out_seq=0 2024-04-26T20:23:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.434+0000 7f6f9b400640 10 client.16005 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6f9b400640 1 -- 172.21.15.131:0/2888203926 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8cbd1af90 con 0x55f8cbdbeab0 2024-04-26T20:23:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6f9b400640 10 client.16005 handle_command_reply: tid=0 2024-04-26T20:23:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 client.16005 shutdown 2024-04-26T20:23:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda97d0 msgr2=0x55f8cbdb34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb34b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0005ab0 tx=0x7f6fa00314a0 comp rx=0 tx=0).stop 2024-04-26T20:23:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 shutdown_connections 2024-04-26T20:23:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbdbeab0 0x55f8cbdc0f00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb34b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6f9b400640 0 client.16005 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 --2- 172.21.15.131:0/2888203926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 >> 172.21.15.131:0/2888203926 conn(0x55f8cbda4450 msgr2=0x55f8cbda3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 shutdown_connections 2024-04-26T20:23:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.435+0000 7f6fbef99740 1 -- 172.21.15.131:0/2888203926 wait complete. 2024-04-26T20:23:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.436+0000 7f6fbef99740 20 client.16005 trim_cache size 0 max 16384 2024-04-26T20:23:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 Processor -- start 2024-04-26T20:23:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 -- start start 2024-04-26T20:23:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdaa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbda97d0 0x55f8cbdaae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41aff20 con 0x55f8cbda8de0 2024-04-26T20:23:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41ad7f0 con 0x55f8cbdae940 2024-04-26T20:23:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fbef99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41ab640 con 0x55f8cbda97d0 2024-04-26T20:23:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdaa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdaa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35414/0 (socket says 172.21.15.131:35414) 2024-04-26T20:23:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fb9800640 1 -- 172.21.15.131:0/18533991 learned_addr learned my addr 172.21.15.131:0/18533991 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fba200640 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.452+0000 7f6fb8e00640 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbda97d0 0x55f8cbdaae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fb9800640 1 -- 172.21.15.131:0/18533991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbda97d0 msgr2=0x55f8cbdaae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fb9800640 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbda97d0 0x55f8cbdaae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fb9800640 1 -- 172.21.15.131:0/18533991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbdae940 msgr2=0x55f8cbdba4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fb9800640 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fb9800640 1 -- 172.21.15.131:0/18533991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cbaeb620 con 0x55f8cbda8de0 2024-04-26T20:23:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fba200640 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fb9800640 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdaa8f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009a00 tx=0x7f6fa8009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1177006d719e7bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fa6000640 1 -- 172.21.15.131:0/18533991 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004780 con 0x55f8cbda8de0 2024-04-26T20:23:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fa6000640 1 -- 172.21.15.131:0/18533991 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa8029070 con 0x55f8cbda8de0 2024-04-26T20:23:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.453+0000 7f6fa6000640 1 -- 172.21.15.131:0/18533991 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8003c90 con 0x55f8cbda8de0 2024-04-26T20:23:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 -- 172.21.15.131:0/18533991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 msgr2=0x55f8cbdaa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdaa8f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009a00 tx=0x7f6fa8009e20 comp rx=0 tx=0).stop 2024-04-26T20:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 -- 172.21.15.131:0/18533991 shutdown_connections 2024-04-26T20:23:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbdae940 0x55f8cbdba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbda97d0 0x55f8cbdaae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 --2- 172.21.15.131:0/18533991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdaa8f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 -- 172.21.15.131:0/18533991 >> 172.21.15.131:0/18533991 conn(0x55f8cbda4450 msgr2=0x55f8cbdb2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 -- 172.21.15.131:0/18533991 shutdown_connections 2024-04-26T20:23:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.454+0000 7f6fbef99740 1 -- 172.21.15.131:0/18533991 wait complete. 2024-04-26T20:23:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 -- start start 2024-04-26T20:23:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 Processor -- start 2024-04-26T20:23:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41abcb0 con 0x55f8cbda8de0 2024-04-26T20:23:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41ad120 con 0x55f8cbda97d0 2024-04-26T20:23:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41ac380 con 0x55f8cbdae940 2024-04-26T20:23:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fbef99740 10 client.0 _ll_register_callbacks cb 0x7f6fa0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.455+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46154/0 (socket says 172.21.15.131:46154) 2024-04-26T20:23:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 -- 172.21.15.131:0/1567457348 learned_addr learned my addr 172.21.15.131:0/1567457348 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35428/0 (socket says 172.21.15.131:35428) 2024-04-26T20:23:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 -- 172.21.15.131:0/1567457348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 msgr2=0x55f8cbdb7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 -- 172.21.15.131:0/1567457348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 msgr2=0x55f8cbdb7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 -- 172.21.15.131:0/1567457348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cbcc7cb0 con 0x55f8cbdae940 2024-04-26T20:23:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fb9800640 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fba200640 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb8db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e940 tx=0x7f6fac005340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50edc1490ed760c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6f9b400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:23:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6f9b400640 1 -- 172.21.15.131:0/1567457348 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac09cab0 con 0x55f8cbdae940 2024-04-26T20:23:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa8029070 con 0x55f8cbdae940 2024-04-26T20:23:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.456+0000 7f6f9b400640 1 -- 172.21.15.131:0/1567457348 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f6fac0ce530 con 0x55f8cbdae940 2024-04-26T20:23:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6f9b400640 1 -- 172.21.15.131:0/1567457348 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fac083050 con 0x55f8cbdae940 2024-04-26T20:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 10 client.26110 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8cbdba600 con 0x55f8cbdae940 2024-04-26T20:23:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6f9b400640 1 -- 172.21.15.131:0/1567457348 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa8029070 con 0x55f8cbdae940 2024-04-26T20:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6f9b400640 1 -- 172.21.15.131:0/1567457348 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6fac098990 con 0x55f8cbdae940 2024-04-26T20:23:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 10 client.26110 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 10 client.26110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 20 client.26110 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbe25650 0x55f8cbe27a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 4 client.26110 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:23:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.457+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8cbd14e00 con 0x55f8cbe25650 2024-04-26T20:23:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.458+0000 7f6fb9800640 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbe25650 0x55f8cbe27a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.458+0000 7f6fb9800640 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbe25650 0x55f8cbe27a70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bcf9758447881b server_cookie=96643d61c4654a23 in_seq=0 out_seq=0 2024-04-26T20:23:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.458+0000 7f6f9b400640 10 client.26110 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.461+0000 7f6f9b400640 1 -- 172.21.15.131:0/1567457348 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8cbd14e00 con 0x55f8cbe25650 2024-04-26T20:23:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.461+0000 7f6f9b400640 10 client.26110 handle_command_reply: tid=0 2024-04-26T20:23:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 client.26110 shutdown 2024-04-26T20:23:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 msgr2=0x55f8cbdb8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb8db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e940 tx=0x7f6fac005340 comp rx=0 tx=0).stop 2024-04-26T20:23:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 shutdown_connections 2024-04-26T20:23:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f8cbe25650 0x55f8cbe27a70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8cbdae940 0x55f8cbdb8db0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f8cbda97d0 0x55f8cbdb7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 --2- 172.21.15.131:0/1567457348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8cbda8de0 0x55f8cbdb7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6f9b400640 0 client.26110 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 >> 172.21.15.131:0/1567457348 conn(0x55f8cbda4450 msgr2=0x55f8cbdb2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 shutdown_connections 2024-04-26T20:23:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 1 -- 172.21.15.131:0/1567457348 wait complete. 2024-04-26T20:23:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.462+0000 7f6fbef99740 20 client.26110 trim_cache size 0 max 16384 2024-04-26T20:23:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:23:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:23:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:23:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:23:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f9c0a3c30 msgr2=0x7f6f9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f9c0a3c30 0x7f6f9c0a60f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55f8cbdaa030 tx=0x7f6fa003f040 comp rx=0 tx=0).stop 2024-04-26T20:23:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb4104370 msgr2=0x7f6fb41a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb4104370 0x7f6fb41a2180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e970 tx=0x7f6fac00ed70 comp rx=0 tx=0).stop 2024-04-26T20:23:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 shutdown_connections 2024-04-26T20:23:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fb4104370 0x7f6fb41a2180 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fb41039b0 0x7f6fb419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f6f9c0a3c30 0x7f6f9c0a60f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 --2- 172.21.15.131:0/1475783896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb41027c0 0x7f6fb419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 >> 172.21.15.131:0/1475783896 conn(0x7f6fb40fdd30 msgr2=0x7f6fb4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 shutdown_connections 2024-04-26T20:23:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:20:55.463+0000 7f6fbac00640 1 -- 172.21.15.131:0/1475783896 wait complete. 2024-04-26T20:23:23.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:55 smithi131 ceph-mon[27919]: pgmap v737: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:57 smithi131 ceph-mon[27919]: pgmap v738: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:24.153 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:20:59 smithi131 ceph-mon[27919]: pgmap v739: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:23:24.262 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:01 smithi131 ceph-mon[27919]: pgmap v740: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:24.322 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:03 smithi131 ceph-mon[27919]: pgmap v741: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:24.440 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:05 smithi131 ceph-mon[27919]: pgmap v742: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:24.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:07 smithi131 ceph-mon[27919]: pgmap v743: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:24.604 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:13 smithi148 ceph-mon[32281]: pgmap v806: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:24.693 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:15 smithi148 ceph-mon[32281]: pgmap v807: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:24.700 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:17 smithi148 ceph-mon[32281]: pgmap v808: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:24.707 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:19 smithi148 ceph-mon[32281]: pgmap v809: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:24.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:21 smithi148 ceph-mon[32281]: pgmap v810: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:24.827 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:23 smithi148 ceph-mon[32281]: pgmap v811: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:24.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:13 smithi157 ceph-mon[32703]: pgmap v806: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:24.843 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:15 smithi157 ceph-mon[32703]: pgmap v807: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:24.849 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:17 smithi157 ceph-mon[32703]: pgmap v808: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:24.855 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:19 smithi157 ceph-mon[32703]: pgmap v809: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:24.957 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:21 smithi157 ceph-mon[32703]: pgmap v810: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:24.969 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:23 smithi157 ceph-mon[32703]: pgmap v811: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.605+0000 7fa34b200640 1 -- 172.21.15.131:0/2359883245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 msgr2=0x7fa34c10ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.605+0000 7fa34b200640 1 --2- 172.21.15.131:0/2359883245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c10ce30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa344009ca0 tx=0x7fa34402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.605+0000 7fa34b200640 1 -- 172.21.15.131:0/2359883245 shutdown_connections 2024-04-26T20:23:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.605+0000 7fa34b200640 1 --2- 172.21.15.131:0/2359883245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c10ce30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.605+0000 7fa34b200640 1 --2- 172.21.15.131:0/2359883245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa34c105be0 0x7fa34c106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.605+0000 7fa34b200640 1 --2- 172.21.15.131:0/2359883245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa34c1049f0 0x7fa34c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.605+0000 7fa34b200640 1 -- 172.21.15.131:0/2359883245 >> 172.21.15.131:0/2359883245 conn(0x7fa34c0fffa0 msgr2=0x7fa34c1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.606+0000 7fa34b200640 1 -- 172.21.15.131:0/2359883245 shutdown_connections 2024-04-26T20:23:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.606+0000 7fa34b200640 1 -- 172.21.15.131:0/2359883245 wait complete. 2024-04-26T20:23:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 Processor -- start 2024-04-26T20:23:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 -- start start 2024-04-26T20:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa34c1049f0 0x7fa34c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa34c105be0 0x7fa34c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c1a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c110570 con 0x7fa34c1065a0 2024-04-26T20:23:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1103f0 con 0x7fa34c1049f0 2024-04-26T20:23:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1106f0 con 0x7fa34c105be0 2024-04-26T20:23:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.607+0000 7fa34a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c1a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c1a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45688/0 (socket says 172.21.15.131:45688) 2024-04-26T20:23:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 -- 172.21.15.131:0/268075361 learned_addr learned my addr 172.21.15.131:0/268075361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa349e00640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa34c1049f0 0x7fa34c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa343e00640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa34c105be0 0x7fa34c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 -- 172.21.15.131:0/268075361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa34c105be0 msgr2=0x7fa34c19b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa34c105be0 0x7fa34c19b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 -- 172.21.15.131:0/268075361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa34c1049f0 msgr2=0x7fa34c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa34c1049f0 0x7fa34c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 -- 172.21.15.131:0/268075361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c1a27d0 con 0x7fa34c1065a0 2024-04-26T20:23:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa349e00640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa34c1049f0 0x7fa34c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.608+0000 7fa34a800640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c1a2160 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa344033040 tx=0x7fa3440316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.609+0000 7fa342a00640 1 -- 172.21.15.131:0/268075361 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344003b70 con 0x7fa34c1065a0 2024-04-26T20:23:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.609+0000 7fa342a00640 1 -- 172.21.15.131:0/268075361 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa344003d10 con 0x7fa34c1065a0 2024-04-26T20:23:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.609+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c1a2a60 con 0x7fa34c1065a0 2024-04-26T20:23:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.609+0000 7fa342a00640 1 -- 172.21.15.131:0/268075361 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34402fb40 con 0x7fa34c1065a0 2024-04-26T20:23:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.609+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c1a2fb0 con 0x7fa34c1065a0 2024-04-26T20:23:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.613+0000 7fa353d7c740 1 Processor -- start 2024-04-26T20:23:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.613+0000 7fa342a00640 1 -- 172.21.15.131:0/268075361 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa344004610 con 0x7fa34c1065a0 2024-04-26T20:23:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.613+0000 7fa353d7c740 1 -- start start 2024-04-26T20:23:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216763200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x5652167682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x56521676add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa353d7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5652166c7380 con 0x5652167637d0 2024-04-26T20:23:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa353d7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x565216663c30 con 0x565216762de0 2024-04-26T20:23:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa353d7c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5652166c3340 con 0x565216768940 2024-04-26T20:23:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x5652167682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x5652167682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45696/0 (socket says 172.21.15.131:45696) 2024-04-26T20:23:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 -- 172.21.15.131:0/1796796757 learned_addr learned my addr 172.21.15.131:0/1796796757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa343e00640 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216763200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa343e00640 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x56521676add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 -- 172.21.15.131:0/1796796757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 msgr2=0x56521676add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x56521676add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 -- 172.21.15.131:0/1796796757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 msgr2=0x565216763200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216763200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.614+0000 7fa349e00640 1 -- 172.21.15.131:0/1796796757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565216685940 con 0x5652167637d0 2024-04-26T20:23:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.615+0000 7fa349e00640 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x5652167682d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0114c0 tx=0x7fa33c011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c75d36d0567c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.615+0000 7fa33b400640 1 -- 172.21.15.131:0/1796796757 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c004020 con 0x5652167637d0 2024-04-26T20:23:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.615+0000 7fa342a00640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa32c0a3c30 0x7fa32c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.615+0000 7fa33b400640 1 -- 172.21.15.131:0/1796796757 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c0054d0 con 0x5652167637d0 2024-04-26T20:23:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.615+0000 7fa33b400640 1 -- 172.21.15.131:0/1796796757 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c004cb0 con 0x5652167637d0 2024-04-26T20:23:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.615+0000 7fa342a00640 1 -- 172.21.15.131:0/268075361 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa3440f1c80 con 0x7fa34c1065a0 2024-04-26T20:23:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.615+0000 7fa343e00640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa32c0a3c30 0x7fa32c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 -- 172.21.15.131:0/1796796757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 msgr2=0x5652167682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x5652167682d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0114c0 tx=0x7fa33c011980 comp rx=0 tx=0).stop 2024-04-26T20:23:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 -- 172.21.15.131:0/1796796757 shutdown_connections 2024-04-26T20:23:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x56521676add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x5652167682d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1796796757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216763200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 -- 172.21.15.131:0/1796796757 >> 172.21.15.131:0/1796796757 conn(0x56521675e450 msgr2=0x56521675e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 -- 172.21.15.131:0/1796796757 shutdown_connections 2024-04-26T20:23:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa343e00640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa32c0a3c30 0x7fa32c0a60f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x565216764030 tx=0x7fa3340096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.616+0000 7fa353d7c740 1 -- 172.21.15.131:0/1796796757 wait complete. 2024-04-26T20:23:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 -- start start 2024-04-26T20:23:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 Processor -- start 2024-04-26T20:23:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216762de0 0x56521676cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x56521676d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216768940 0x5652167744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56521676b8f0 con 0x5652167637d0 2024-04-26T20:23:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56521664ec60 con 0x565216768940 2024-04-26T20:23:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56521676ba70 con 0x565216762de0 2024-04-26T20:23:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa353d7c740 10 client.0 _ll_register_callbacks cb 0x56521667ae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.617+0000 7fa343e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x56521676d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x56521676d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45706/0 (socket says 172.21.15.131:45706) 2024-04-26T20:23:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 -- 172.21.15.131:0/1533194265 learned_addr learned my addr 172.21.15.131:0/1533194265 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa349e00640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216762de0 0x56521676cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa34a800640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216768940 0x5652167744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 -- 172.21.15.131:0/1533194265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216762de0 msgr2=0x56521676cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216762de0 0x56521676cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 -- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216768940 msgr2=0x5652167744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216768940 0x5652167744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 -- 172.21.15.131:0/1533194265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521665d510 con 0x5652167637d0 2024-04-26T20:23:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa349e00640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216762de0 0x56521676cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa343e00640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x56521676d4a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa33401e9e0 tx=0x7fa334004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de86f52d62869cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.618+0000 7fa333e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:23:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.619+0000 7fa333e00640 1 -- 172.21.15.131:0/1533194265 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33402e060 con 0x5652167637d0 2024-04-26T20:23:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.619+0000 7fa333e00640 1 -- 172.21.15.131:0/1533194265 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa334019030 con 0x5652167637d0 2024-04-26T20:23:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.619+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33c0054d0 con 0x5652167637d0 2024-04-26T20:23:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.619+0000 7fa333e00640 1 -- 172.21.15.131:0/1533194265 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa334035050 con 0x5652167637d0 2024-04-26T20:23:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.619+0000 7fa353d7c740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.619+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652163d9210 con 0x5652167637d0 2024-04-26T20:23:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.619+0000 7fa333e00640 1 -- 172.21.15.131:0/1533194265 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c0054d0 con 0x5652167637d0 2024-04-26T20:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa333e00640 1 -- 172.21.15.131:0/1533194265 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa33401d3a0 con 0x5652167637d0 2024-04-26T20:23:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa353d7c740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa353d7c740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa353d7c740 20 client.16023 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x565216778ab0 0x56521677af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa353d7c740 4 client.16023 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:23:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652166d4f90 con 0x565216778ab0 2024-04-26T20:23:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.620+0000 7fa349e00640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x565216778ab0 0x56521677af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.621+0000 7fa349e00640 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x565216778ab0 0x56521677af40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4f48715efdc98c server_cookie=3969b97a327a0b1a in_seq=0 out_seq=0 2024-04-26T20:23:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.621+0000 7fa333e00640 10 client.16023 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.621+0000 7fa333e00640 1 -- 172.21.15.131:0/1533194265 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652166d4f90 con 0x565216778ab0 2024-04-26T20:23:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.621+0000 7fa333e00640 10 client.16023 handle_command_reply: tid=0 2024-04-26T20:23:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 client.16023 shutdown 2024-04-26T20:23:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 msgr2=0x56521676d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x56521676d4a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa33401e9e0 tx=0x7fa334004730 comp rx=0 tx=0).stop 2024-04-26T20:23:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 shutdown_connections 2024-04-26T20:23:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x565216778ab0 0x56521677af40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216768940 0x5652167744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x56521676d4a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa333e00640 0 client.16023 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 --2- 172.21.15.131:0/1533194265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216762de0 0x56521676cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 >> 172.21.15.131:0/1533194265 conn(0x56521675e450 msgr2=0x56521675d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 shutdown_connections 2024-04-26T20:23:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 1 -- 172.21.15.131:0/1533194265 wait complete. 2024-04-26T20:23:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.622+0000 7fa353d7c740 20 client.16023 trim_cache size 0 max 16384 2024-04-26T20:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.638+0000 7fa353d7c740 1 Processor -- start 2024-04-26T20:23:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa353d7c740 1 -- start start 2024-04-26T20:23:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216780850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216780d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x5652167744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa353d7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1aff80 con 0x5652167637d0 2024-04-26T20:23:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa353d7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ad850 con 0x565216762de0 2024-04-26T20:23:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa353d7c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ab6a0 con 0x565216768940 2024-04-26T20:23:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa343e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216780d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa343e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216780d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45720/0 (socket says 172.21.15.131:45720) 2024-04-26T20:23:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa343e00640 1 -- 172.21.15.131:0/2826936960 learned_addr learned my addr 172.21.15.131:0/2826936960 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa349e00640 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216780850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.639+0000 7fa34a800640 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x5652167744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.640+0000 7fa343e00640 1 -- 172.21.15.131:0/2826936960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 msgr2=0x5652167744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.640+0000 7fa343e00640 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x5652167744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.640+0000 7fa343e00640 1 -- 172.21.15.131:0/2826936960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 msgr2=0x565216780850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.640+0000 7fa343e00640 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216780850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.640+0000 7fa343e00640 1 -- 172.21.15.131:0/2826936960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652164a5620 con 0x5652167637d0 2024-04-26T20:23:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.640+0000 7fa343e00640 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216780d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa334006010 tx=0x7fa334009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65eaaaad0b2d8dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.642+0000 7fa33aa00640 1 -- 172.21.15.131:0/2826936960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33401ddd0 con 0x5652167637d0 2024-04-26T20:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa33aa00640 1 -- 172.21.15.131:0/2826936960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33401d3a0 con 0x5652167637d0 2024-04-26T20:23:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa33aa00640 1 -- 172.21.15.131:0/2826936960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334039b80 con 0x5652167637d0 2024-04-26T20:23:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa353d7c740 1 -- 172.21.15.131:0/2826936960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 msgr2=0x565216780d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216780d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa334006010 tx=0x7fa334009be0 comp rx=0 tx=0).stop 2024-04-26T20:23:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa353d7c740 1 -- 172.21.15.131:0/2826936960 shutdown_connections 2024-04-26T20:23:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x5652167744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216780d90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2826936960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216780850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.643+0000 7fa353d7c740 1 -- 172.21.15.131:0/2826936960 >> 172.21.15.131:0/2826936960 conn(0x56521675e450 msgr2=0x56521675e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.644+0000 7fa353d7c740 1 -- 172.21.15.131:0/2826936960 shutdown_connections 2024-04-26T20:23:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.644+0000 7fa353d7c740 1 -- 172.21.15.131:0/2826936960 wait complete. 2024-04-26T20:23:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.644+0000 7fa353d7c740 1 -- start start 2024-04-26T20:23:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.644+0000 7fa353d7c740 1 Processor -- start 2024-04-26T20:23:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216771b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216772050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa353d7c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x565216773010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa353d7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1abd10 con 0x5652167637d0 2024-04-26T20:23:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa353d7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ad180 con 0x565216762de0 2024-04-26T20:23:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa353d7c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ac3e0 con 0x565216768940 2024-04-26T20:23:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa353d7c740 10 client.0 _ll_register_callbacks cb 0x7fa334009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216771b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa343e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216772050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa34a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x565216773010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216771b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:35816/0 (socket says 172.21.15.131:35816) 2024-04-26T20:23:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 -- 172.21.15.131:0/2617850848 learned_addr learned my addr 172.21.15.131:0/2617850848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 -- 172.21.15.131:0/2617850848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 msgr2=0x565216773010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x565216773010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 -- 172.21.15.131:0/2617850848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 msgr2=0x565216772050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216772050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.645+0000 7fa349e00640 1 -- 172.21.15.131:0/2617850848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565216681cb0 con 0x565216762de0 2024-04-26T20:23:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa349e00640 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216771b10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009460 tx=0x7fa33c0110f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b586cc947654ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa333e00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:23:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa333e00640 1 -- 172.21.15.131:0/2617850848 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c01d040 con 0x565216762de0 2024-04-26T20:23:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33401d3a0 con 0x565216762de0 2024-04-26T20:23:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa333e00640 1 -- 172.21.15.131:0/2617850848 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa33c011260 con 0x565216762de0 2024-04-26T20:23:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa333e00640 1 -- 172.21.15.131:0/2617850848 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa33c03a050 con 0x565216762de0 2024-04-26T20:23:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa353d7c740 10 client.26150 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652167747b0 con 0x565216762de0 2024-04-26T20:23:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.646+0000 7fa333e00640 1 -- 172.21.15.131:0/2617850848 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33401d3a0 con 0x565216762de0 2024-04-26T20:23:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa333e00640 1 -- 172.21.15.131:0/2617850848 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa33c02b1d0 con 0x565216762de0 2024-04-26T20:23:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa353d7c740 10 client.26150 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa353d7c740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa353d7c740 20 client.26150 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5652167e7df0 0x5652167ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa353d7c740 4 client.26150 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:23:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652166cee00 con 0x5652167e7df0 2024-04-26T20:23:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa343e00640 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5652167e7df0 0x5652167ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa333e00640 10 client.26150 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.647+0000 7fa343e00640 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5652167e7df0 0x5652167ea1e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1c96a3cb5a3964 server_cookie=45e761789bc424a2 in_seq=0 out_seq=0 2024-04-26T20:23:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa333e00640 1 -- 172.21.15.131:0/2617850848 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652166cee00 con 0x5652167e7df0 2024-04-26T20:23:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa333e00640 10 client.26150 handle_command_reply: tid=0 2024-04-26T20:23:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 client.26150 shutdown 2024-04-26T20:23:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 msgr2=0x565216771b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216771b10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009460 tx=0x7fa33c0110f0 comp rx=0 tx=0).stop 2024-04-26T20:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 shutdown_connections 2024-04-26T20:23:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5652167e7df0 0x5652167ea1e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565216768940 0x565216773010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa333e00640 0 client.26150 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652167637d0 0x565216772050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 --2- 172.21.15.131:0/2617850848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565216762de0 0x565216771b10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.651+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 >> 172.21.15.131:0/2617850848 conn(0x56521675e450 msgr2=0x56521676cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 shutdown_connections 2024-04-26T20:23:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa353d7c740 1 -- 172.21.15.131:0/2617850848 wait complete. 2024-04-26T20:23:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa353d7c740 20 client.26150 trim_cache size 0 max 16384 2024-04-26T20:23:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:23:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:23:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:23:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:23:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa32c0a3c30 msgr2=0x7fa32c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa34b200640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa32c0a3c30 0x7fa32c0a60f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x565216764030 tx=0x7fa3340096a0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 msgr2=0x7fa34c1a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa34b200640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c1a2160 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa344033040 tx=0x7fa3440316f0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 shutdown_connections 2024-04-26T20:23:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.652+0000 7fa34b200640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1065a0 0x7fa34c1a2160 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.653+0000 7fa34b200640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa32c0a3c30 0x7fa32c0a60f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.653+0000 7fa34b200640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa34c105be0 0x7fa34c19b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.653+0000 7fa34b200640 1 --2- 172.21.15.131:0/268075361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa34c1049f0 0x7fa34c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.653+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 >> 172.21.15.131:0/268075361 conn(0x7fa34c0fffa0 msgr2=0x7fa34c108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.653+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 shutdown_connections 2024-04-26T20:23:34.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:08.653+0000 7fa34b200640 1 -- 172.21.15.131:0/268075361 wait complete. 2024-04-26T20:23:34.736 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:25 smithi148 ceph-mon[32281]: pgmap v812: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:34.745 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:27 smithi148 ceph-mon[32281]: pgmap v813: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:34.789 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:29 smithi148 ceph-mon[32281]: pgmap v814: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:34.819 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:31 smithi148 ceph-mon[32281]: pgmap v815: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:34.827 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:33 smithi148 ceph-mon[32281]: pgmap v816: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:34.834 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:25 smithi157 ceph-mon[32703]: pgmap v812: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:34.844 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:27 smithi157 ceph-mon[32703]: pgmap v813: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:34.911 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:29 smithi157 ceph-mon[32703]: pgmap v814: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:34.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:31 smithi157 ceph-mon[32703]: pgmap v815: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:34.924 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:33 smithi157 ceph-mon[32703]: pgmap v816: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:34.932 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:09 smithi131 ceph-mon[27919]: pgmap v744: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:34.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:11 smithi131 ceph-mon[27919]: pgmap v745: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:35.005 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:13 smithi131 ceph-mon[27919]: pgmap v746: 129 pgs: 129 active+clean; 683 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:35.012 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:15 smithi131 ceph-mon[27919]: pgmap v747: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:35.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:17 smithi131 ceph-mon[27919]: pgmap v748: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:35.115 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:19 smithi131 ceph-mon[27919]: pgmap v749: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 -- 172.21.15.131:0/736952957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4108de0 msgr2=0x7f3dd4109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 --2- 172.21.15.131:0/736952957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4108de0 0x7f3dd4109200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009900 tx=0x7f3dc802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:23:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 -- 172.21.15.131:0/736952957 shutdown_connections 2024-04-26T20:23:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 --2- 172.21.15.131:0/736952957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dd410a990 0x7f3dd4111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 --2- 172.21.15.131:0/736952957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4109fd0 0x7f3dd410a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 --2- 172.21.15.131:0/736952957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4108de0 0x7f3dd4109200 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 -- 172.21.15.131:0/736952957 >> 172.21.15.131:0/736952957 conn(0x7f3dd4077dc0 msgr2=0x7f3dd407a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 -- 172.21.15.131:0/736952957 shutdown_connections 2024-04-26T20:23:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.795+0000 7f3dda800640 1 -- 172.21.15.131:0/736952957 wait complete. 2024-04-26T20:23:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.796+0000 7f3dda800640 1 Processor -- start 2024-04-26T20:23:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.796+0000 7f3dda800640 1 -- start start 2024-04-26T20:23:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dda800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4108de0 0x7f3dd419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dda800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4109fd0 0x7f3dd419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dda800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dd410a990 0x7f3dd41a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dda800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4114910 con 0x7f3dd4109fd0 2024-04-26T20:23:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dda800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4114790 con 0x7f3dd4108de0 2024-04-26T20:23:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dda800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4114a90 con 0x7f3dd410a990 2024-04-26T20:23:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4109fd0 0x7f3dd419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd9400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4108de0 0x7f3dd419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dd410a990 0x7f3dd41a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4109fd0 0x7f3dd419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42076/0 (socket says 172.21.15.131:42076) 2024-04-26T20:23:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2751929419 learned_addr learned my addr 172.21.15.131:0/2751929419 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2751929419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dd410a990 msgr2=0x7f3dd41a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dd410a990 0x7f3dd41a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2751929419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4108de0 msgr2=0x7f3dd419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.797+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4108de0 0x7f3dd419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2751929419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd41a6c10 con 0x7f3dd4109fd0 2024-04-26T20:23:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dd9e00640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dd410a990 0x7f3dd41a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4109fd0 0x7f3dd419f580 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000ea60 tx=0x7f3dc000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dd2000640 1 -- 172.21.15.131:0/2751929419 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000cde0 con 0x7f3dd4109fd0 2024-04-26T20:23:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dd2000640 1 -- 172.21.15.131:0/2751929419 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc00035f0 con 0x7f3dd4109fd0 2024-04-26T20:23:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41a6f00 con 0x7f3dd4109fd0 2024-04-26T20:23:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dd2000640 1 -- 172.21.15.131:0/2751929419 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0010690 con 0x7f3dd4109fd0 2024-04-26T20:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.798+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd41a73c0 con 0x7f3dd4109fd0 2024-04-26T20:23:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.801+0000 7f3ddeda0740 1 Processor -- start 2024-04-26T20:23:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.801+0000 7f3dd2000640 1 -- 172.21.15.131:0/2751929419 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3dc00037a0 con 0x7f3dd4109fd0 2024-04-26T20:23:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.801+0000 7f3ddeda0740 1 -- start start 2024-04-26T20:23:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x564114131200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641141317d0 0x5641141362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114136940 0x564114138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564114095380 con 0x564114136940 2024-04-26T20:23:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564114031c30 con 0x564114130de0 2024-04-26T20:23:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564114091340 con 0x5641141317d0 2024-04-26T20:23:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114136940 0x564114138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3dd9e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x564114131200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3dd9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641141317d0 0x5641141362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3dd9e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x564114131200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:58738/0 (socket says 172.21.15.131:58738) 2024-04-26T20:23:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3dd9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641141317d0 0x5641141362d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45704/0 (socket says 172.21.15.131:45704) 2024-04-26T20:23:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.802+0000 7f3dd9e00640 1 -- 172.21.15.131:0/1257014053 learned_addr learned my addr 172.21.15.131:0/1257014053 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dd9e00640 1 -- 172.21.15.131:0/1257014053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641141317d0 msgr2=0x5641141362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dd9e00640 1 --2- 172.21.15.131:0/1257014053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641141317d0 0x5641141362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dd9e00640 1 -- 172.21.15.131:0/1257014053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114136940 msgr2=0x564114138dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dd9e00640 1 --2- 172.21.15.131:0/1257014053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114136940 0x564114138dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dd9e00640 1 -- 172.21.15.131:0/1257014053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564114053940 con 0x564114130de0 2024-04-26T20:23:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dd9e00640 1 --2- 172.21.15.131:0/1257014053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x564114131200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0117e0 tx=0x7f3dcc011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29d6c5a0ff71a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dc6600640 1 -- 172.21.15.131:0/1257014053 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011310 con 0x564114130de0 2024-04-26T20:23:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dd2000640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3dbc0a3c30 0x7f3dbc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dc6600640 1 -- 172.21.15.131:0/1257014053 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dcc003e20 con 0x564114130de0 2024-04-26T20:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.803+0000 7f3dc6600640 1 -- 172.21.15.131:0/1257014053 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc004490 con 0x564114130de0 2024-04-26T20:23:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3dd2000640 1 -- 172.21.15.131:0/2751929419 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3dc00d0c20 con 0x7f3dd4109fd0 2024-04-26T20:23:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3dd9400640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3dbc0a3c30 0x7f3dbc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 -- 172.21.15.131:0/1257014053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 msgr2=0x564114131200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/1257014053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x564114131200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0117e0 tx=0x7f3dcc011ca0 comp rx=0 tx=0).stop 2024-04-26T20:23:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 -- 172.21.15.131:0/1257014053 shutdown_connections 2024-04-26T20:23:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/1257014053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114136940 0x564114138dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/1257014053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641141317d0 0x5641141362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/1257014053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x564114131200 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 -- 172.21.15.131:0/1257014053 >> 172.21.15.131:0/1257014053 conn(0x56411412c450 msgr2=0x56411412c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 -- 172.21.15.131:0/1257014053 shutdown_connections 2024-04-26T20:23:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.804+0000 7f3ddeda0740 1 -- 172.21.15.131:0/1257014053 wait complete. 2024-04-26T20:23:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.805+0000 7f3dd9400640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3dbc0a3c30 0x7f3dbc0a60f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009230 tx=0x7f3dc80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.805+0000 7f3ddeda0740 1 -- start start 2024-04-26T20:23:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.805+0000 7f3ddeda0740 1 Processor -- start 2024-04-26T20:23:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.805+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114130de0 0x56411413b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641141317d0 0x56411413b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x564114142540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5641141398f0 con 0x564114130de0 2024-04-26T20:23:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56411401cc60 con 0x5641141317d0 2024-04-26T20:23:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564114139a70 con 0x564114136940 2024-04-26T20:23:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3ddeda0740 10 client.0 _ll_register_callbacks cb 0x564114048e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114130de0 0x56411413b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114130de0 0x56411413b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42090/0 (socket says 172.21.15.131:42090) 2024-04-26T20:23:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641141317d0 0x56411413b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd9400640 1 -- 172.21.15.131:0/4267302648 learned_addr learned my addr 172.21.15.131:0/4267302648 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd9e00640 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x564114142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd8a00640 1 -- 172.21.15.131:0/4267302648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 msgr2=0x564114142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x564114142540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd8a00640 1 -- 172.21.15.131:0/4267302648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114130de0 msgr2=0x56411413b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114130de0 0x56411413b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.806+0000 7f3dd8a00640 1 -- 172.21.15.131:0/4267302648 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411402b510 con 0x5641141317d0 2024-04-26T20:23:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641141317d0 0x56411413b570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ce1e0 tx=0x7f3dc00d4360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e818aca6b77edbd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3dbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:23:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3dbaa00640 1 -- 172.21.15.131:0/4267302648 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc009c4f0 con 0x5641141317d0 2024-04-26T20:23:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dcc003e20 con 0x5641141317d0 2024-04-26T20:23:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3dbaa00640 1 -- 172.21.15.131:0/4267302648 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3dc0004000 con 0x5641141317d0 2024-04-26T20:23:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3dbaa00640 1 -- 172.21.15.131:0/4267302648 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3dc009c310 con 0x5641141317d0 2024-04-26T20:23:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3ddeda0740 10 client.26162 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564113da7210 con 0x5641141317d0 2024-04-26T20:23:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.807+0000 7f3dbaa00640 1 -- 172.21.15.131:0/4267302648 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dcc003e20 con 0x5641141317d0 2024-04-26T20:23:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3dbaa00640 1 -- 172.21.15.131:0/4267302648 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3dc009dce0 con 0x5641141317d0 2024-04-26T20:23:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3ddeda0740 10 client.26162 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3ddeda0740 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3ddeda0740 20 client.26162 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564114146a90 0x564114148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3ddeda0740 4 client.26162 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:23:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641140a2f90 con 0x564114146a90 2024-04-26T20:23:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.808+0000 7f3dd9400640 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564114146a90 0x564114148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.809+0000 7f3dd9400640 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564114146a90 0x564114148ec0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31be09299f9f98d1 server_cookie=ca03471f125acbaa in_seq=0 out_seq=0 2024-04-26T20:23:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.809+0000 7f3dbaa00640 10 client.26162 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3dbaa00640 1 -- 172.21.15.131:0/4267302648 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641140a2f90 con 0x564114146a90 2024-04-26T20:23:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3dbaa00640 10 client.26162 handle_command_reply: tid=0 2024-04-26T20:23:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 client.26162 shutdown 2024-04-26T20:23:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641141317d0 msgr2=0x56411413b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641141317d0 0x56411413b570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ce1e0 tx=0x7f3dc00d4360 comp rx=0 tx=0).stop 2024-04-26T20:23:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 shutdown_connections 2024-04-26T20:23:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x564114146a90 0x564114148ec0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x564114142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641141317d0 0x56411413b570 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/4267302648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564114130de0 0x56411413b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3dbaa00640 0 client.26162 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 >> 172.21.15.131:0/4267302648 conn(0x56411412c450 msgr2=0x56411412b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 shutdown_connections 2024-04-26T20:23:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 1 -- 172.21.15.131:0/4267302648 wait complete. 2024-04-26T20:23:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.810+0000 7f3ddeda0740 20 client.26162 trim_cache size 0 max 16384 2024-04-26T20:23:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.826+0000 7f3ddeda0740 1 Processor -- start 2024-04-26T20:23:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.826+0000 7f3ddeda0740 1 -- start start 2024-04-26T20:23:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x56411412f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114142500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x56411412fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41b4290 con 0x5641141317d0 2024-04-26T20:23:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41b1b60 con 0x564114130de0 2024-04-26T20:23:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41af9b0 con 0x564114136940 2024-04-26T20:23:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114142500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42100/0 (socket says 172.21.15.131:42100) 2024-04-26T20:23:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2396924124 learned_addr learned my addr 172.21.15.131:0/2396924124 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3dd9e00640 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x56411412fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3dd9400640 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x56411412f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.827+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2396924124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 msgr2=0x56411412fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x56411412fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2396924124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 msgr2=0x56411412f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x56411412f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dd8a00640 1 -- 172.21.15.131:0/2396924124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564113e73620 con 0x5641141317d0 2024-04-26T20:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114142500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ce210 tx=0x7f3dc0009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3ef7ac2cf2439c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dc5c00640 1 -- 172.21.15.131:0/2396924124 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc009c4f0 con 0x5641141317d0 2024-04-26T20:23:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dc5c00640 1 -- 172.21.15.131:0/2396924124 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc009d0b0 con 0x5641141317d0 2024-04-26T20:23:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.828+0000 7f3dc5c00640 1 -- 172.21.15.131:0/2396924124 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00bbb70 con 0x5641141317d0 2024-04-26T20:23:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 -- 172.21.15.131:0/2396924124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 msgr2=0x564114142500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114142500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ce210 tx=0x7f3dc0009550 comp rx=0 tx=0).stop 2024-04-26T20:23:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 -- 172.21.15.131:0/2396924124 shutdown_connections 2024-04-26T20:23:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114136940 0x56411412fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114142500 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/2396924124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114130de0 0x56411412f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 -- 172.21.15.131:0/2396924124 >> 172.21.15.131:0/2396924124 conn(0x56411412c450 msgr2=0x5641141329f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 -- 172.21.15.131:0/2396924124 shutdown_connections 2024-04-26T20:23:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 -- 172.21.15.131:0/2396924124 wait complete. 2024-04-26T20:23:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.829+0000 7f3ddeda0740 1 -- start start 2024-04-26T20:23:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 1 Processor -- start 2024-04-26T20:23:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114130de0 0x56411413fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114140090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114136940 0x564114141050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41b0020 con 0x5641141317d0 2024-04-26T20:23:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41b1490 con 0x564114136940 2024-04-26T20:23:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41b06f0 con 0x564114130de0 2024-04-26T20:23:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3ddeda0740 10 client.0 _ll_register_callbacks cb 0x7f3dc00d4460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3dd9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114130de0 0x56411413fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3dd8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42102/0 (socket says 172.21.15.131:42102) 2024-04-26T20:23:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3dd8a00640 1 -- 172.21.15.131:0/984483488 learned_addr learned my addr 172.21.15.131:0/984483488 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3dd9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114130de0 0x56411413fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45746/0 (socket says 172.21.15.131:45746) 2024-04-26T20:23:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.830+0000 7f3dd9e00640 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114136940 0x564114141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dd8a00640 1 -- 172.21.15.131:0/984483488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114130de0 msgr2=0x56411413fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114130de0 0x56411413fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dd8a00640 1 -- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114136940 msgr2=0x564114141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114136940 0x564114141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dd8a00640 1 -- 172.21.15.131:0/984483488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411404fcb0 con 0x5641141317d0 2024-04-26T20:23:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dd9400640 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114130de0 0x56411413fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dd8a00640 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114140090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ce760 tx=0x7f3dc00ce1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709dddd356271566 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:23:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dbaa00640 1 -- 172.21.15.131:0/984483488 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc009c4f0 con 0x5641141317d0 2024-04-26T20:23:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dbaa00640 1 -- 172.21.15.131:0/984483488 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3dc00870e0 con 0x5641141317d0 2024-04-26T20:23:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc009d0b0 con 0x5641141317d0 2024-04-26T20:23:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3dbaa00640 1 -- 172.21.15.131:0/984483488 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3dc0098c50 con 0x5641141317d0 2024-04-26T20:23:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3ddeda0740 10 client.16047 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.831+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564114142780 con 0x5641141317d0 2024-04-26T20:23:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3dbaa00640 1 -- 172.21.15.131:0/984483488 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc009d0b0 con 0x5641141317d0 2024-04-26T20:23:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3dbaa00640 1 -- 172.21.15.131:0/984483488 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3dc009dce0 con 0x5641141317d0 2024-04-26T20:23:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3ddeda0740 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3ddeda0740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3ddeda0740 20 client.16047 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5641141ad4f0 0x5641141af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3ddeda0740 4 client.16047 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:23:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56411409ce00 con 0x5641141ad4f0 2024-04-26T20:23:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.832+0000 7f3dd9400640 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5641141ad4f0 0x5641141af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.833+0000 7f3dd9400640 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5641141ad4f0 0x5641141af910 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67580b19ed2335c4 server_cookie=870910eeac4b467f in_seq=0 out_seq=0 2024-04-26T20:23:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.833+0000 7f3dbaa00640 10 client.16047 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3dbaa00640 1 -- 172.21.15.131:0/984483488 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56411409ce00 con 0x5641141ad4f0 2024-04-26T20:23:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3dbaa00640 10 client.16047 handle_command_reply: tid=0 2024-04-26T20:23:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 client.16047 shutdown 2024-04-26T20:23:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 msgr2=0x564114140090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114140090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ce760 tx=0x7f3dc00ce1a0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 shutdown_connections 2024-04-26T20:23:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5641141ad4f0 0x5641141af910 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564114136940 0x564114141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641141317d0 0x564114140090 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3dbaa00640 0 client.16047 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 --2- 172.21.15.131:0/984483488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564114130de0 0x56411413fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 >> 172.21.15.131:0/984483488 conn(0x56411412c450 msgr2=0x56411412f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 shutdown_connections 2024-04-26T20:23:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 1 -- 172.21.15.131:0/984483488 wait complete. 2024-04-26T20:23:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.837+0000 7f3ddeda0740 20 client.16047 trim_cache size 0 max 16384 2024-04-26T20:23:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:23:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:23:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:23:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:23:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3dbc0a3c30 msgr2=0x7f3dbc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3dbc0a3c30 0x7f3dbc0a60f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009230 tx=0x7f3dc80314a0 comp rx=0 tx=0).stop 2024-04-26T20:23:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4109fd0 msgr2=0x7f3dd419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4109fd0 0x7f3dd419f580 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000ea60 tx=0x7f3dc000ef20 comp rx=0 tx=0).stop 2024-04-26T20:23:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 shutdown_connections 2024-04-26T20:23:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3dbc0a3c30 0x7f3dbc0a60f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dd410a990 0x7f3dd41a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dd4109fd0 0x7f3dd419f580 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 --2- 172.21.15.131:0/2751929419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4108de0 0x7f3dd419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 >> 172.21.15.131:0/2751929419 conn(0x7f3dd4077dc0 msgr2=0x7f3dd410d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 shutdown_connections 2024-04-26T20:23:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:21.838+0000 7f3dda800640 1 -- 172.21.15.131:0/2751929419 wait complete. 2024-04-26T20:23:45.740 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:21 smithi131 ceph-mon[27919]: pgmap v750: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:45.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:23 smithi131 ceph-mon[27919]: pgmap v751: 129 pgs: 129 active+clean; 684 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:45.923 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:25 smithi131 ceph-mon[27919]: pgmap v752: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:23:45.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:27 smithi131 ceph-mon[27919]: pgmap v753: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:46.108 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:29 smithi131 ceph-mon[27919]: pgmap v754: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:46.154 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:31 smithi131 ceph-mon[27919]: pgmap v755: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:46.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:33 smithi131 ceph-mon[27919]: pgmap v756: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:46.182 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:35 smithi148 ceph-mon[32281]: pgmap v817: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:46.235 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:37 smithi148 ceph-mon[32281]: pgmap v818: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:46.246 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:39 smithi148 ceph-mon[32281]: pgmap v819: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:46.253 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:41 smithi148 ceph-mon[32281]: pgmap v820: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:46.331 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:43 smithi148 ceph-mon[32281]: pgmap v821: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:46.497 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:45 smithi148 ceph-mon[32281]: pgmap v822: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:46.563 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:35 smithi157 ceph-mon[32703]: pgmap v817: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:46.640 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:37 smithi157 ceph-mon[32703]: pgmap v818: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:46.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:39 smithi157 ceph-mon[32703]: pgmap v819: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:46.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:41 smithi157 ceph-mon[32703]: pgmap v820: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:46.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:43 smithi157 ceph-mon[32703]: pgmap v821: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:46.745 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:45 smithi157 ceph-mon[32703]: pgmap v822: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 -- 172.21.15.131:0/2764412998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8103250 msgr2=0x7facb8105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 --2- 172.21.15.131:0/2764412998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8103250 0x7facb8105640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faca8009900 tx=0x7faca802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:23:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 -- 172.21.15.131:0/2764412998 shutdown_connections 2024-04-26T20:23:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 --2- 172.21.15.131:0/2764412998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb81084b0 0x7facb810a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 --2- 172.21.15.131:0/2764412998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb8105b80 0x7facb8107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 --2- 172.21.15.131:0/2764412998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8103250 0x7facb8105640 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 -- 172.21.15.131:0/2764412998 >> 172.21.15.131:0/2764412998 conn(0x7facb80fcc60 msgr2=0x7facb80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 -- 172.21.15.131:0/2764412998 shutdown_connections 2024-04-26T20:23:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.981+0000 7facbcc00640 1 -- 172.21.15.131:0/2764412998 wait complete. 2024-04-26T20:23:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 Processor -- start 2024-04-26T20:23:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 -- start start 2024-04-26T20:23:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb8103250 0x7facb819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8105b80 0x7facb819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb81084b0 0x7facb81a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facb810e370 con 0x7facb8103250 2024-04-26T20:23:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7facb810e1f0 con 0x7facb8105b80 2024-04-26T20:23:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.982+0000 7facbcc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facb810e4f0 con 0x7facb81084b0 2024-04-26T20:23:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb8103250 0x7facb819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8105b80 0x7facb819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8105b80 0x7facb819b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42092/0 (socket says 172.21.15.131:42092) 2024-04-26T20:23:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 -- 172.21.15.131:0/1026227728 learned_addr learned my addr 172.21.15.131:0/1026227728 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb8103250 0x7facb819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56984/0 (socket says 172.21.15.131:56984) 2024-04-26T20:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb7e00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb81084b0 0x7facb81a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 -- 172.21.15.131:0/1026227728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb81084b0 msgr2=0x7facb81a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb81084b0 0x7facb81a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 -- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb8103250 msgr2=0x7facb819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb8103250 0x7facb819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 -- 172.21.15.131:0/1026227728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb81a2870 con 0x7facb8105b80 2024-04-26T20:23:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.983+0000 7facb6a00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8105b80 0x7facb819b1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faca400e9c0 tx=0x7faca400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.984+0000 7facafe00640 1 -- 172.21.15.131:0/1026227728 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca400cd80 con 0x7facb8105b80 2024-04-26T20:23:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.984+0000 7facafe00640 1 -- 172.21.15.131:0/1026227728 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca40054d0 con 0x7facb8105b80 2024-04-26T20:23:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.984+0000 7facafe00640 1 -- 172.21.15.131:0/1026227728 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4010430 con 0x7facb8105b80 2024-04-26T20:23:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.984+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb81a2b60 con 0x7facb8105b80 2024-04-26T20:23:48.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.984+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb81a3080 con 0x7facb8105b80 2024-04-26T20:23:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.987+0000 7facc0f73740 1 Processor -- start 2024-04-26T20:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.987+0000 7facc0f73740 1 -- start start 2024-04-26T20:23:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facc0f73740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facc0f73740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facc0f73740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278b1940 0x559a278b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facc0f73740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559a27810380 con 0x559a278ac7d0 2024-04-26T20:23:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facc0f73740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559a277acc30 con 0x559a278b1940 2024-04-26T20:23:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facc0f73740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a2780c340 con 0x559a278abde0 2024-04-26T20:23:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40782/0 (socket says 172.21.15.131:40782) 2024-04-26T20:23:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 -- 172.21.15.131:0/59231334 learned_addr learned my addr 172.21.15.131:0/59231334 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 -- 172.21.15.131:0/59231334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278b1940 msgr2=0x559a278b3dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:23:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 --2- 172.21.15.131:0/59231334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278b1940 0x559a278b3dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 -- 172.21.15.131:0/59231334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 msgr2=0x559a278b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 --2- 172.21.15.131:0/59231334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facb7e00640 1 -- 172.21.15.131:0/59231334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a277ce940 con 0x559a278abde0 2024-04-26T20:23:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.988+0000 7facafe00640 1 -- 172.21.15.131:0/1026227728 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7faca4004fd0 con 0x7facb8105b80 2024-04-26T20:23:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.989+0000 7facb7e00640 1 --2- 172.21.15.131:0/59231334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278ac200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7facb0009fd0 tx=0x7facb00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d9610e2180d3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.989+0000 7faca3e00640 1 -- 172.21.15.131:0/59231334 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0004d20 con 0x559a278abde0 2024-04-26T20:23:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.989+0000 7faca3e00640 1 -- 172.21.15.131:0/59231334 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb0004050 con 0x559a278abde0 2024-04-26T20:23:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.989+0000 7faca3e00640 1 -- 172.21.15.131:0/59231334 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0005280 con 0x559a278abde0 2024-04-26T20:23:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 -- 172.21.15.131:0/59231334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 msgr2=0x559a278ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 --2- 172.21.15.131:0/59231334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278ac200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7facb0009fd0 tx=0x7facb00118d0 comp rx=0 tx=0).stop 2024-04-26T20:23:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 -- 172.21.15.131:0/59231334 shutdown_connections 2024-04-26T20:23:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 --2- 172.21.15.131:0/59231334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278b1940 0x559a278b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 --2- 172.21.15.131:0/59231334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 --2- 172.21.15.131:0/59231334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278ac200 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 -- 172.21.15.131:0/59231334 >> 172.21.15.131:0/59231334 conn(0x559a278a7450 msgr2=0x559a278a7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 -- 172.21.15.131:0/59231334 shutdown_connections 2024-04-26T20:23:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 -- 172.21.15.131:0/59231334 wait complete. 2024-04-26T20:23:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facafe00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fac9c0a3b40 0x7fac9c0a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facafe00640 1 -- 172.21.15.131:0/1026227728 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7faca40d0e60 con 0x7facb8105b80 2024-04-26T20:23:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facc0f73740 1 -- start start 2024-04-26T20:23:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.990+0000 7facb7400640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fac9c0a3b40 0x7fac9c0a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 1 Processor -- start 2024-04-26T20:23:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facb7400640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fac9c0a3b40 0x7fac9c0a6000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faca8031500 tx=0x7faca8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278b1940 0x559a278bd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559a278b48f0 con 0x559a278b1940 2024-04-26T20:23:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559a27797c60 con 0x559a278ac7d0 2024-04-26T20:23:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a278b4a70 con 0x559a278abde0 2024-04-26T20:23:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facc0f73740 10 client.0 _ll_register_callbacks cb 0x559a277c3e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facb7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278b1940 0x559a278bd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.991+0000 7facb7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40794/0 (socket says 172.21.15.131:40794) 2024-04-26T20:23:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb7400640 1 -- 172.21.15.131:0/3077526642 learned_addr learned my addr 172.21.15.131:0/3077526642 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb6a00640 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb6a00640 1 -- 172.21.15.131:0/3077526642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 msgr2=0x559a278b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb6a00640 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb6a00640 1 -- 172.21.15.131:0/3077526642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278b1940 msgr2=0x559a278bd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb6a00640 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278b1940 0x559a278bd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb6a00640 1 -- 172.21.15.131:0/3077526642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a277a6510 con 0x559a278ac7d0 2024-04-26T20:23:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facb6a00640 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278b65a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faca40ce3c0 tx=0x7faca409c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbdd8ff0aeb8468 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7faca0c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:23:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7faca0c00640 1 -- 172.21.15.131:0/3077526642 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca409d290 con 0x559a278ac7d0 2024-04-26T20:23:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7faca0c00640 1 -- 172.21.15.131:0/3077526642 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7faca40a5370 con 0x559a278ac7d0 2024-04-26T20:23:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.992+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb0004050 con 0x559a278ac7d0 2024-04-26T20:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7faca0c00640 1 -- 172.21.15.131:0/3077526642 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faca409cb90 con 0x559a278ac7d0 2024-04-26T20:23:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 10 client.26177 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a27522210 con 0x559a278ac7d0 2024-04-26T20:23:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7faca0c00640 1 -- 172.21.15.131:0/3077526642 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb0004050 con 0x559a278ac7d0 2024-04-26T20:23:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7faca0c00640 1 -- 172.21.15.131:0/3077526642 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faca4098850 con 0x559a278ac7d0 2024-04-26T20:23:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 10 client.26177 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 20 client.26177 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a278c1b40 0x559a278c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 4 client.26177 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:23:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.993+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a2781df90 con 0x559a278c1b40 2024-04-26T20:23:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.994+0000 7facb7400640 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a278c1b40 0x559a278c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.994+0000 7facb7400640 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a278c1b40 0x559a278c3f30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1c6dd6cc6e3aa4 server_cookie=11188726fe405976 in_seq=0 out_seq=0 2024-04-26T20:23:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.994+0000 7faca0c00640 10 client.26177 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7faca0c00640 1 -- 172.21.15.131:0/3077526642 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a2781df90 con 0x559a278c1b40 2024-04-26T20:23:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7faca0c00640 10 client.26177 handle_command_reply: tid=0 2024-04-26T20:23:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 client.26177 shutdown 2024-04-26T20:23:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 msgr2=0x559a278b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278b65a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faca40ce3c0 tx=0x7faca409c750 comp rx=0 tx=0).stop 2024-04-26T20:23:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 shutdown_connections 2024-04-26T20:23:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a278c1b40 0x559a278c3f30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278b1940 0x559a278bd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278b65a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7faca0c00640 0 client.26177 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 --2- 172.21.15.131:0/3077526642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278abde0 0x559a278b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 >> 172.21.15.131:0/3077526642 conn(0x559a278a7450 msgr2=0x559a278a6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.995+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 shutdown_connections 2024-04-26T20:23:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.996+0000 7facc0f73740 1 -- 172.21.15.131:0/3077526642 wait complete. 2024-04-26T20:23:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:34.996+0000 7facc0f73740 20 client.26177 trim_cache size 0 max 16384 2024-04-26T20:23:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.011+0000 7facc0f73740 1 Processor -- start 2024-04-26T20:23:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facc0f73740 1 -- start start 2024-04-26T20:23:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facc0f73740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278abde0 0x559a278aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facc0f73740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facc0f73740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facc0f73740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facb81b0020 con 0x559a278ac7d0 2024-04-26T20:23:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facc0f73740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7facb81ad8f0 con 0x559a278abde0 2024-04-26T20:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facc0f73740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facb81ab740 con 0x559a278b1940 2024-04-26T20:23:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facb7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278abde0 0x559a278aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facb7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278abde0 0x559a278aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42110/0 (socket says 172.21.15.131:42110) 2024-04-26T20:23:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facb7400640 1 -- 172.21.15.131:0/1294871270 learned_addr learned my addr 172.21.15.131:0/1294871270 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facb6a00640 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.012+0000 7facb7e00640 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7facb7400640 1 -- 172.21.15.131:0/1294871270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 msgr2=0x559a278bd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7facb7400640 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7facb7400640 1 -- 172.21.15.131:0/1294871270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 msgr2=0x559a278aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7facb7400640 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7facb7400640 1 -- 172.21.15.131:0/1294871270 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a275ee620 con 0x559a278abde0 2024-04-26T20:23:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7facb7e00640 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7facb7400640 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278abde0 0x559a278aa8e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faca8005ab0 tx=0x7faca8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92829da9b071b152 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7faca3400640 1 -- 172.21.15.131:0/1294871270 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8035040 con 0x559a278abde0 2024-04-26T20:23:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7faca3400640 1 -- 172.21.15.131:0/1294871270 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca8031a50 con 0x559a278abde0 2024-04-26T20:23:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.013+0000 7faca3400640 1 -- 172.21.15.131:0/1294871270 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80546e0 con 0x559a278abde0 2024-04-26T20:23:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 -- 172.21.15.131:0/1294871270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278abde0 msgr2=0x559a278aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278abde0 0x559a278aa8e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faca8005ab0 tx=0x7faca8040000 comp rx=0 tx=0).stop 2024-04-26T20:23:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 -- 172.21.15.131:0/1294871270 shutdown_connections 2024-04-26T20:23:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278ac7d0 0x559a278aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 --2- 172.21.15.131:0/1294871270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278abde0 0x559a278aa8e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 -- 172.21.15.131:0/1294871270 >> 172.21.15.131:0/1294871270 conn(0x559a278a7450 msgr2=0x559a278aa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 -- 172.21.15.131:0/1294871270 shutdown_connections 2024-04-26T20:23:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 -- 172.21.15.131:0/1294871270 wait complete. 2024-04-26T20:23:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.014+0000 7facc0f73740 1 -- start start 2024-04-26T20:23:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 1 Processor -- start 2024-04-26T20:23:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278abde0 0x559a278bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facb81abdb0 con 0x559a278abde0 2024-04-26T20:23:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7facb81ad220 con 0x559a278ac7d0 2024-04-26T20:23:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facb81ac480 con 0x559a278b1940 2024-04-26T20:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facc0f73740 10 client.0 _ll_register_callbacks cb 0x7faca40048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:23:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facb7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278abde0 0x559a278bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facb6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facb6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278bb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42120/0 (socket says 172.21.15.131:42120) 2024-04-26T20:23:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facb6a00640 1 -- 172.21.15.131:0/1554746724 learned_addr learned my addr 172.21.15.131:0/1554746724 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.015+0000 7facb7e00640 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facb6a00640 1 -- 172.21.15.131:0/1554746724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 msgr2=0x559a278bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facb6a00640 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facb6a00640 1 -- 172.21.15.131:0/1554746724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278abde0 msgr2=0x559a278bae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:23:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facb6a00640 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278abde0 0x559a278bae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facb6a00640 1 -- 172.21.15.131:0/1554746724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a277cacb0 con 0x559a278ac7d0 2024-04-26T20:23:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facb6a00640 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278bb350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faca40b3040 tx=0x7faca40049b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5070fe9912f5cce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7faca0c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:23:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7faca0c00640 1 -- 172.21.15.131:0/1554746724 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4098da0 con 0x559a278ac7d0 2024-04-26T20:23:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7faca0c00640 1 -- 172.21.15.131:0/1554746724 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7faca40a5370 con 0x559a278ac7d0 2024-04-26T20:23:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca8031a50 con 0x559a278ac7d0 2024-04-26T20:23:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7faca0c00640 1 -- 172.21.15.131:0/1554746724 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faca409d290 con 0x559a278ac7d0 2024-04-26T20:23:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facc0f73740 10 client.26183 fetch_fsmap learned FSMap version 27 2024-04-26T20:23:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.016+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a27928040 con 0x559a278ac7d0 2024-04-26T20:23:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7faca0c00640 1 -- 172.21.15.131:0/1554746724 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca8031a50 con 0x559a278ac7d0 2024-04-26T20:23:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7faca0c00640 1 -- 172.21.15.131:0/1554746724 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faca40a20d0 con 0x559a278ac7d0 2024-04-26T20:23:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7facc0f73740 10 client.26183 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:23:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7facc0f73740 10 client.26183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:23:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7facc0f73740 20 client.26183 populate_metadata read hostname 'smithi131' 2024-04-26T20:23:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7facc0f73740 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a27928860 0x559a2792ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7facc0f73740 4 client.26183 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:23:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a27817e00 con 0x559a27928860 2024-04-26T20:23:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.017+0000 7facb7400640 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a27928860 0x559a2792ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.018+0000 7facb7400640 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a27928860 0x559a2792ac80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c838734fc3566158 server_cookie=b2c087fb313f8419 in_seq=0 out_seq=0 2024-04-26T20:23:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.018+0000 7faca0c00640 10 client.26183 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.021+0000 7faca0c00640 1 -- 172.21.15.131:0/1554746724 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a27817e00 con 0x559a27928860 2024-04-26T20:23:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.021+0000 7faca0c00640 10 client.26183 handle_command_reply: tid=0 2024-04-26T20:23:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.021+0000 7facc0f73740 1 client.26183 shutdown 2024-04-26T20:23:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 msgr2=0x559a278bb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278bb350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faca40b3040 tx=0x7faca40049b0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 shutdown_connections 2024-04-26T20:23:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559a27928860 0x559a2792ac80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a278b1940 0x559a278bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a278ac7d0 0x559a278bb350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 --2- 172.21.15.131:0/1554746724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a278abde0 0x559a278bae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7faca0c00640 0 client.26183 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:23:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 >> 172.21.15.131:0/1554746724 conn(0x559a278a7450 msgr2=0x559a278aaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 shutdown_connections 2024-04-26T20:23:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 1 -- 172.21.15.131:0/1554746724 wait complete. 2024-04-26T20:23:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.022+0000 7facc0f73740 20 client.26183 trim_cache size 0 max 16384 2024-04-26T20:23:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:23:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:23:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:23:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:23:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fac9c0a3b40 msgr2=0x7fac9c0a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fac9c0a3b40 0x7fac9c0a6000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faca8031500 tx=0x7faca8005a20 comp rx=0 tx=0).stop 2024-04-26T20:23:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8105b80 msgr2=0x7facb819b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8105b80 0x7facb819b1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faca400e9c0 tx=0x7faca400ee80 comp rx=0 tx=0).stop 2024-04-26T20:23:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 shutdown_connections 2024-04-26T20:23:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fac9c0a3b40 0x7fac9c0a6000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb81084b0 0x7facb81a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facb8105b80 0x7facb819b1a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 --2- 172.21.15.131:0/1026227728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb8103250 0x7facb819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 >> 172.21.15.131:0/1026227728 conn(0x7facb80fcc60 msgr2=0x7facb8102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 shutdown_connections 2024-04-26T20:23:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:35.023+0000 7facbcc00640 1 -- 172.21.15.131:0/1026227728 wait complete. 2024-04-26T20:23:56.996 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:47 smithi157 ceph-mon[32703]: pgmap v823: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:49 smithi157 ceph-mon[32703]: pgmap v824: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:23:57.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:51 smithi157 ceph-mon[32703]: pgmap v825: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.086 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:53 smithi157 ceph-mon[32703]: pgmap v826: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:55 smithi157 ceph-mon[32703]: pgmap v827: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:57.206 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:55 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:23:57.283 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:47 smithi148 ceph-mon[32281]: pgmap v823: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:49 smithi148 ceph-mon[32281]: pgmap v824: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:23:57.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:51 smithi148 ceph-mon[32281]: pgmap v825: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.423 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:53 smithi148 ceph-mon[32281]: pgmap v826: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.502 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:55 smithi148 ceph-mon[32281]: pgmap v827: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:57.559 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:55 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:23:57.596 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:35 smithi131 ceph-mon[27919]: pgmap v757: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:57.603 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:37 smithi131 ceph-mon[27919]: pgmap v758: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:23:57.610 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:39 smithi131 ceph-mon[27919]: pgmap v759: 129 pgs: 129 active+clean; 685 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:23:57.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:41 smithi131 ceph-mon[27919]: pgmap v760: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:23:57.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:43 smithi131 ceph-mon[27919]: pgmap v761: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:45 smithi131 ceph-mon[27919]: pgmap v762: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:23:57.792 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:47 smithi131 ceph-mon[27919]: pgmap v763: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:57.801 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:23:57.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:23:57.815 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:23:57.934 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:57 smithi157 ceph-mon[32703]: pgmap v828: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:58.058 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:23:58.126 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:23:58.134 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:23:58.186 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:57 smithi148 ceph-mon[32281]: pgmap v828: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:23:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 -- 172.21.15.131:0/2913922617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 msgr2=0x7fc9c0106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 --2- 172.21.15.131:0/2913922617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 0x7fc9c0106060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009900 tx=0x7fc9a802a410 comp rx=0 tx=0).stop 2024-04-26T20:23:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 -- 172.21.15.131:0/2913922617 shutdown_connections 2024-04-26T20:23:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 --2- 172.21.15.131:0/2913922617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c01065a0 0x7fc9c010ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 --2- 172.21.15.131:0/2913922617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 0x7fc9c0106060 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 --2- 172.21.15.131:0/2913922617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 0x7fc9c0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 -- 172.21.15.131:0/2913922617 >> 172.21.15.131:0/2913922617 conn(0x7fc9c00fffa0 msgr2=0x7fc9c01023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:23:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 -- 172.21.15.131:0/2913922617 shutdown_connections 2024-04-26T20:23:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.166+0000 7fc9c6000640 1 -- 172.21.15.131:0/2913922617 wait complete. 2024-04-26T20:23:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 Processor -- start 2024-04-26T20:23:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 -- start start 2024-04-26T20:23:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 0x7fc9c019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 0x7fc9c019b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c01065a0 0x7fc9c01a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0110520 con 0x7fc9c0105be0 2024-04-26T20:23:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01103a0 con 0x7fc9c01049f0 2024-04-26T20:23:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.167+0000 7fc9c6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01106a0 con 0x7fc9c01065a0 2024-04-26T20:23:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9c4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 0x7fc9c019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 0x7fc9c019b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9c4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 0x7fc9c019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:53946/0 (socket says 172.21.15.131:53946) 2024-04-26T20:23:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9c4c00640 1 -- 172.21.15.131:0/721656461 learned_addr learned my addr 172.21.15.131:0/721656461 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:23:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9c5600640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c01065a0 0x7fc9c01a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9b7e00640 1 -- 172.21.15.131:0/721656461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c01065a0 msgr2=0x7fc9c01a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c01065a0 0x7fc9c01a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9b7e00640 1 -- 172.21.15.131:0/721656461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 msgr2=0x7fc9c019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:23:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 0x7fc9c019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:23:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9b7e00640 1 -- 172.21.15.131:0/721656461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c01a27f0 con 0x7fc9c0105be0 2024-04-26T20:23:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.168+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 0x7fc9c019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:23:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.169+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 0x7fc9c019b110 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8006010 tx=0x7fc9a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:23:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.169+0000 7fc9be000640 1 -- 172.21.15.131:0/721656461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a803f030 con 0x7fc9c0105be0 2024-04-26T20:23:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.169+0000 7fc9be000640 1 -- 172.21.15.131:0/721656461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a80318f0 con 0x7fc9c0105be0 2024-04-26T20:23:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.169+0000 7fc9be000640 1 -- 172.21.15.131:0/721656461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8035070 con 0x7fc9c0105be0 2024-04-26T20:23:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.169+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c01a2a80 con 0x7fc9c0105be0 2024-04-26T20:23:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.169+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c01a2f70 con 0x7fc9c0105be0 2024-04-26T20:23:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.173+0000 7fc9ca46c740 1 Processor -- start 2024-04-26T20:23:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.173+0000 7fc9ca46c740 1 -- start start 2024-04-26T20:23:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9be000640 1 -- 172.21.15.131:0/721656461 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc9a8042030 con 0x7fc9c0105be0 2024-04-26T20:23:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860349200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86034e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d86034e940 0x55d860350dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:23:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d8602ad380 con 0x55d86034e940 2024-04-26T20:24:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d860249c30 con 0x55d8603497d0 2024-04-26T20:24:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d8602a9340 con 0x55d860348de0 2024-04-26T20:24:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9c5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860349200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d86034e940 0x55d860350dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9c5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860349200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53230/0 (socket says 172.21.15.131:53230) 2024-04-26T20:24:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9c5600640 1 -- 172.21.15.131:0/1186498737 learned_addr learned my addr 172.21.15.131:0/1186498737 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.174+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86034e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9c5600640 1 -- 172.21.15.131:0/1186498737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 msgr2=0x55d86034e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9c5600640 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86034e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9c5600640 1 -- 172.21.15.131:0/1186498737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d86034e940 msgr2=0x55d860350dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9c5600640 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d86034e940 0x55d860350dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9c5600640 1 -- 172.21.15.131:0/1186498737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86026b940 con 0x55d860348de0 2024-04-26T20:24:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9c5600640 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860349200 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009f50 tx=0x7fc9b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d71eed80827cd5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9b5c00640 1 -- 172.21.15.131:0/1186498737 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8004d20 con 0x55d860348de0 2024-04-26T20:24:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9b5c00640 1 -- 172.21.15.131:0/1186498737 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b8004050 con 0x55d860348de0 2024-04-26T20:24:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9b5c00640 1 -- 172.21.15.131:0/1186498737 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8005220 con 0x55d860348de0 2024-04-26T20:24:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.175+0000 7fc9be000640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc9a40a3c30 0x7fc9a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9be000640 1 -- 172.21.15.131:0/721656461 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc9a80f0da0 con 0x7fc9c0105be0 2024-04-26T20:24:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc9a40a3c30 0x7fc9a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 -- 172.21.15.131:0/1186498737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 msgr2=0x55d860349200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860349200 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009f50 tx=0x7fc9b8011810 comp rx=0 tx=0).stop 2024-04-26T20:24:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 -- 172.21.15.131:0/1186498737 shutdown_connections 2024-04-26T20:24:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d86034e940 0x55d860350dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86034e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/1186498737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860349200 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 -- 172.21.15.131:0/1186498737 >> 172.21.15.131:0/1186498737 conn(0x55d860344450 msgr2=0x55d860344850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.176+0000 7fc9ca46c740 1 -- 172.21.15.131:0/1186498737 shutdown_connections 2024-04-26T20:24:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.177+0000 7fc9ca46c740 1 -- 172.21.15.131:0/1186498737 wait complete. 2024-04-26T20:24:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.177+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc9a40a3c30 0x7fc9a40a60f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000bfd0 tx=0x7fc9b000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.177+0000 7fc9ca46c740 1 -- start start 2024-04-26T20:24:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.177+0000 7fc9ca46c740 1 Processor -- start 2024-04-26T20:24:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860352ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860353530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d86034e940 0x55d86035a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d8603518f0 con 0x55d8603497d0 2024-04-26T20:24:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d860234c60 con 0x55d86034e940 2024-04-26T20:24:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d860351a70 con 0x55d860348de0 2024-04-26T20:24:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9ca46c740 10 client.0 _ll_register_callbacks cb 0x55d860260e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860353530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860353530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39796/0 (socket says 172.21.15.131:39796) 2024-04-26T20:24:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 -- 172.21.15.131:0/3552876608 learned_addr learned my addr 172.21.15.131:0/3552876608 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860352ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9c5600640 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d86034e940 0x55d86035a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 -- 172.21.15.131:0/3552876608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 msgr2=0x55d860352ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860352ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 -- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d86034e940 msgr2=0x55d86035a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d86034e940 0x55d86035a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.178+0000 7fc9b7e00640 1 -- 172.21.15.131:0/3552876608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d860243510 con 0x55d8603497d0 2024-04-26T20:24:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860353530 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80b95b0 tx=0x7fc9a803b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc33763a3c63f13e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ae000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ae000640 1 -- 172.21.15.131:0/3552876608 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80bde60 con 0x55d8603497d0 2024-04-26T20:24:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ae000640 1 -- 172.21.15.131:0/3552876608 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc9a80dd070 con 0x55d8603497d0 2024-04-26T20:24:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b8004050 con 0x55d8603497d0 2024-04-26T20:24:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ae000640 1 -- 172.21.15.131:0/3552876608 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9a8049040 con 0x55d8603497d0 2024-04-26T20:24:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ca46c740 10 client.16077 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d85ffbf210 con 0x55d8603497d0 2024-04-26T20:24:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.179+0000 7fc9ae000640 1 -- 172.21.15.131:0/3552876608 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b8004050 con 0x55d8603497d0 2024-04-26T20:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9ae000640 1 -- 172.21.15.131:0/3552876608 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a80b9250 con 0x55d8603497d0 2024-04-26T20:24:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9ca46c740 10 client.16077 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9ca46c740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9ca46c740 20 client.16077 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d86035ebd0 0x55d860361000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9ca46c740 4 client.16077 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8602baf90 con 0x55d86035ebd0 2024-04-26T20:24:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.180+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d86035ebd0 0x55d860361000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:03.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.181+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d86035ebd0 0x55d860361000 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb49aceab6dc285 server_cookie=beec81c5b6c8a644 in_seq=0 out_seq=0 2024-04-26T20:24:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.181+0000 7fc9ae000640 10 client.16077 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:03.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.181+0000 7fc9ae000640 1 -- 172.21.15.131:0/3552876608 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8602baf90 con 0x55d86035ebd0 2024-04-26T20:24:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.181+0000 7fc9ae000640 10 client.16077 handle_command_reply: tid=0 2024-04-26T20:24:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.181+0000 7fc9ca46c740 1 client.16077 shutdown 2024-04-26T20:24:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 msgr2=0x55d860353530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860353530 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80b95b0 tx=0x7fc9a803b520 comp rx=0 tx=0).stop 2024-04-26T20:24:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 shutdown_connections 2024-04-26T20:24:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d86035ebd0 0x55d860361000 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d86034e940 0x55d86035a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860353530 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/3552876608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d860348de0 0x55d860352ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 >> 172.21.15.131:0/3552876608 conn(0x55d860344450 msgr2=0x55d860343650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ae000640 0 client.16077 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 shutdown_connections 2024-04-26T20:24:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 1 -- 172.21.15.131:0/3552876608 wait complete. 2024-04-26T20:24:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.182+0000 7fc9ca46c740 20 client.16077 trim_cache size 0 max 16384 2024-04-26T20:24:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 Processor -- start 2024-04-26T20:24:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 -- start start 2024-04-26T20:24:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d860348de0 0x55d860347b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86035a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d860365900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:04.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01aff60 con 0x55d860348de0 2024-04-26T20:24:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01ad830 con 0x55d8603497d0 2024-04-26T20:24:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.198+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01ab680 con 0x55d86034e940 2024-04-26T20:24:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d860348de0 0x55d860347b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d860348de0 0x55d860347b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39810/0 (socket says 172.21.15.131:39810) 2024-04-26T20:24:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 -- 172.21.15.131:0/133561018 learned_addr learned my addr 172.21.15.131:0/133561018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86035a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c5600640 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d860365900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 -- 172.21.15.131:0/133561018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 msgr2=0x55d860365900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d860365900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 -- 172.21.15.131:0/133561018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 msgr2=0x55d86035a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86035a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 -- 172.21.15.131:0/133561018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86008b620 con 0x55d860348de0 2024-04-26T20:24:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.199+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d860348de0 0x55d860347b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55d860349bd0 tx=0x7fc9b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1fc91c2f1a1744 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9b5200640 1 -- 172.21.15.131:0/133561018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b002a530 con 0x55d860348de0 2024-04-26T20:24:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9b5200640 1 -- 172.21.15.131:0/133561018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b002a6d0 con 0x55d860348de0 2024-04-26T20:24:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9b5200640 1 -- 172.21.15.131:0/133561018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b001ed40 con 0x55d860348de0 2024-04-26T20:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 -- 172.21.15.131:0/133561018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d860348de0 msgr2=0x55d860347b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d860348de0 0x55d860347b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55d860349bd0 tx=0x7fc9b0019000 comp rx=0 tx=0).stop 2024-04-26T20:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 -- 172.21.15.131:0/133561018 shutdown_connections 2024-04-26T20:24:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d860365900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8603497d0 0x55d86035a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/133561018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d860348de0 0x55d860347b10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 -- 172.21.15.131:0/133561018 >> 172.21.15.131:0/133561018 conn(0x55d860344450 msgr2=0x55d860347400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 -- 172.21.15.131:0/133561018 shutdown_connections 2024-04-26T20:24:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.200+0000 7fc9ca46c740 1 -- 172.21.15.131:0/133561018 wait complete. 2024-04-26T20:24:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 -- start start 2024-04-26T20:24:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 Processor -- start 2024-04-26T20:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d860348de0 0x55d860358050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860358590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d8603595a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01abcf0 con 0x55d8603497d0 2024-04-26T20:24:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01ad160 con 0x55d860348de0 2024-04-26T20:24:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01ac3c0 con 0x55d86034e940 2024-04-26T20:24:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.201+0000 7fc9ca46c740 10 client.0 _ll_register_callbacks cb 0x7fc9a80f45e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860358590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860358590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39816/0 (socket says 172.21.15.131:39816) 2024-04-26T20:24:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 -- 172.21.15.131:0/272892960 learned_addr learned my addr 172.21.15.131:0/272892960 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d860348de0 0x55d860358050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9c5600640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d8603595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 -- 172.21.15.131:0/272892960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 msgr2=0x55d8603595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d8603595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 -- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d860348de0 msgr2=0x55d860358050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d860348de0 0x55d860358050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 -- 172.21.15.131:0/272892960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d860267cb0 con 0x55d8603497d0 2024-04-26T20:24:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9c5600640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d8603595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9b7e00640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860358590 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80ee3e0 tx=0x7fc9a80f4870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fc6ec351a76ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9ae000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9ae000640 1 -- 172.21.15.131:0/272892960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80a7040 con 0x55d8603497d0 2024-04-26T20:24:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9ae000640 1 -- 172.21.15.131:0/272892960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc9a80f4390 con 0x55d8603497d0 2024-04-26T20:24:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.202+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b002a6d0 con 0x55d8603497d0 2024-04-26T20:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ae000640 1 -- 172.21.15.131:0/272892960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9a80bde60 con 0x55d8603497d0 2024-04-26T20:24:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 10 client.16083 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d860267cb0 con 0x55d8603497d0 2024-04-26T20:24:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ae000640 1 -- 172.21.15.131:0/272892960 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b002a6d0 con 0x55d8603497d0 2024-04-26T20:24:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ae000640 1 -- 172.21.15.131:0/272892960 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a80c5380 con 0x55d8603497d0 2024-04-26T20:24:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 20 client.16083 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d8603ce1e0 0x55d8603d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 4 client.16083 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8602b4e00 con 0x55d8603ce1e0 2024-04-26T20:24:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.203+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d8603ce1e0 0x55d8603d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.204+0000 7fc9c4c00640 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d8603ce1e0 0x55d8603d0600 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2653e9b605bf9d server_cookie=15571584bb2f9efd in_seq=0 out_seq=0 2024-04-26T20:24:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.204+0000 7fc9ae000640 10 client.16083 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ae000640 1 -- 172.21.15.131:0/272892960 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8602b4e00 con 0x55d8603ce1e0 2024-04-26T20:24:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ae000640 10 client.16083 handle_command_reply: tid=0 2024-04-26T20:24:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 client.16083 shutdown 2024-04-26T20:24:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 msgr2=0x55d860358590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860358590 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80ee3e0 tx=0x7fc9a80f4870 comp rx=0 tx=0).stop 2024-04-26T20:24:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 shutdown_connections 2024-04-26T20:24:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55d8603ce1e0 0x55d8603d0600 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d86034e940 0x55d8603595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8603497d0 0x55d860358590 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ae000640 0 client.16083 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 --2- 172.21.15.131:0/272892960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d860348de0 0x55d860358050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 >> 172.21.15.131:0/272892960 conn(0x55d860344450 msgr2=0x55d860342dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 shutdown_connections 2024-04-26T20:24:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 1 -- 172.21.15.131:0/272892960 wait complete. 2024-04-26T20:24:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.208+0000 7fc9ca46c740 20 client.16083 trim_cache size 0 max 16384 2024-04-26T20:24:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc9a40a3c30 msgr2=0x7fc9a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc9a40a3c30 0x7fc9a40a60f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000bfd0 tx=0x7fc9b000b4a0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 msgr2=0x7fc9c019b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 0x7fc9c019b110 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8006010 tx=0x7fc9a80042f0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 shutdown_connections 2024-04-26T20:24:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc9a40a3c30 0x7fc9a40a60f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c01065a0 0x7fc9c01a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0105be0 0x7fc9c019b110 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 --2- 172.21.15.131:0/721656461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c01049f0 0x7fc9c019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 >> 172.21.15.131:0/721656461 conn(0x7fc9c00fffa0 msgr2=0x7fc9c0108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 shutdown_connections 2024-04-26T20:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:21:48.209+0000 7fc9c6000640 1 -- 172.21.15.131:0/721656461 wait complete. 2024-04-26T20:24:09.248 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:23:59 smithi148 ceph-mon[32281]: pgmap v829: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:09.301 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:01 smithi148 ceph-mon[32281]: pgmap v830: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:09.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:03 smithi148 ceph-mon[32281]: pgmap v831: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:09.415 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:05 smithi148 ceph-mon[32281]: pgmap v832: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:09.423 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:07 smithi148 ceph-mon[32281]: pgmap v833: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:09.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:23:59 smithi157 ceph-mon[32703]: pgmap v829: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:09.477 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:01 smithi157 ceph-mon[32703]: pgmap v830: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:09.484 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:03 smithi157 ceph-mon[32703]: pgmap v831: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:09.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:05 smithi157 ceph-mon[32703]: pgmap v832: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:09.544 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:07 smithi157 ceph-mon[32703]: pgmap v833: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:09.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:49 smithi131 ceph-mon[27919]: pgmap v764: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:09.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:51 smithi131 ceph-mon[27919]: pgmap v765: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:09.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:52 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:24:09.795 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:53 smithi131 ceph-mon[27919]: pgmap v766: 129 pgs: 129 active+clean; 686 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T20:24:09.888 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:24:09.955 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:24:10.052 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:24:10.060 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:55 smithi131 ceph-mon[27919]: pgmap v767: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:10.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:56 smithi131 ceph-mon[27919]: pgmap v768: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:10.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:21:59 smithi131 ceph-mon[27919]: pgmap v769: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:10.082 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:09 smithi148 ceph-mon[32281]: pgmap v834: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:10.089 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:09 smithi157 ceph-mon[32703]: pgmap v834: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 -- 172.21.15.131:0/2869864634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 msgr2=0x7f0c78101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 --2- 172.21.15.131:0/2869864634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 0x7f0c78101e80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009860 tx=0x7f0c6002a3d0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 -- 172.21.15.131:0/2869864634 shutdown_connections 2024-04-26T20:24:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 --2- 172.21.15.131:0/2869864634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781023c0 0x7f0c78102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 --2- 172.21.15.131:0/2869864634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 0x7f0c78101e80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 --2- 172.21.15.131:0/2869864634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c781010c0 0x7f0c781014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 -- 172.21.15.131:0/2869864634 >> 172.21.15.131:0/2869864634 conn(0x7f0c780fcc60 msgr2=0x7f0c780ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 -- 172.21.15.131:0/2869864634 shutdown_connections 2024-04-26T20:24:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.352+0000 7f0c77e00640 1 -- 172.21.15.131:0/2869864634 wait complete. 2024-04-26T20:24:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.353+0000 7f0c77e00640 1 Processor -- start 2024-04-26T20:24:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.353+0000 7f0c77e00640 1 -- start start 2024-04-26T20:24:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781010c0 0x7f0c7819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 0x7f0c7819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c781023c0 0x7f0c781a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c77e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c78110550 con 0x7f0c781010c0 2024-04-26T20:24:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c77e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781103d0 con 0x7f0c781023c0 2024-04-26T20:24:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c77e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781106d0 con 0x7f0c78101a20 2024-04-26T20:24:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781010c0 0x7f0c7819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781010c0 0x7f0c7819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57926/0 (socket says 172.21.15.131:57926) 2024-04-26T20:24:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c76a00640 1 -- 172.21.15.131:0/3590586600 learned_addr learned my addr 172.21.15.131:0/3590586600 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c77400640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c781023c0 0x7f0c781a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.354+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 0x7f0c7819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c76a00640 1 -- 172.21.15.131:0/3590586600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 msgr2=0x7f0c7819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c76a00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 0x7f0c7819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c76a00640 1 -- 172.21.15.131:0/3590586600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c781023c0 msgr2=0x7f0c781a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c76a00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c781023c0 0x7f0c781a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c76a00640 1 -- 172.21.15.131:0/3590586600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c781a2880 con 0x7f0c781010c0 2024-04-26T20:24:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c76a00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781010c0 0x7f0c7819ac70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800e9f0 tx=0x7f0c6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c6f400640 1 -- 172.21.15.131:0/3590586600 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800cd80 con 0x7f0c781010c0 2024-04-26T20:24:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a2b70 con 0x7f0c781010c0 2024-04-26T20:24:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.355+0000 7f0c6f400640 1 -- 172.21.15.131:0/3590586600 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c680054d0 con 0x7f0c781010c0 2024-04-26T20:24:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.356+0000 7f0c6f400640 1 -- 172.21.15.131:0/3590586600 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68010690 con 0x7f0c781010c0 2024-04-26T20:24:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.356+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a30c0 con 0x7f0c781010c0 2024-04-26T20:24:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 Processor -- start 2024-04-26T20:24:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c6f400640 1 -- 172.21.15.131:0/3590586600 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0c68004e90 con 0x7f0c781010c0 2024-04-26T20:24:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 -- start start 2024-04-26T20:24:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2d5f0 0x55a912f2da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f32ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f33150 0x55a912f355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a912e89380 con 0x55a912f33150 2024-04-26T20:24:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a912e25c30 con 0x55a912f2dfe0 2024-04-26T20:24:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.359+0000 7f0c80434740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a912e85340 con 0x55a912f2d5f0 2024-04-26T20:24:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f32ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f32ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:56564/0 (socket says 172.21.15.131:56564) 2024-04-26T20:24:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 -- 172.21.15.131:0/847543314 learned_addr learned my addr 172.21.15.131:0/847543314 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c77400640 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2d5f0 0x55a912f2da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f33150 0x55a912f355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 -- 172.21.15.131:0/847543314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2d5f0 msgr2=0x55a912f2da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2d5f0 0x55a912f2da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 -- 172.21.15.131:0/847543314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f33150 msgr2=0x55a912f355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f33150 0x55a912f355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.360+0000 7f0c76a00640 1 -- 172.21.15.131:0/847543314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a912e47940 con 0x55a912f2dfe0 2024-04-26T20:24:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.361+0000 7f0c76a00640 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f32ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c680ce270 tx=0x7f0c680d4e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1898f4866157dd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.361+0000 7f0c6f400640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c5c0a3cb0 0x7f0c5c0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.361+0000 7f0c67400640 1 -- 172.21.15.131:0/847543314 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c680e4070 con 0x55a912f2dfe0 2024-04-26T20:24:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.361+0000 7f0c6f400640 1 -- 172.21.15.131:0/3590586600 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0c680d0d70 con 0x7f0c781010c0 2024-04-26T20:24:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.361+0000 7f0c67400640 1 -- 172.21.15.131:0/847543314 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c6809d200 con 0x55a912f2dfe0 2024-04-26T20:24:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.361+0000 7f0c67400640 1 -- 172.21.15.131:0/847543314 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6809c460 con 0x55a912f2dfe0 2024-04-26T20:24:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.361+0000 7f0c77400640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c5c0a3cb0 0x7f0c5c0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 -- 172.21.15.131:0/847543314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 msgr2=0x55a912f32ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c77400640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c5c0a3cb0 0x7f0c5c0a6170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55a912f2e840 tx=0x7f0c70010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f32ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c680ce270 tx=0x7f0c680d4e20 comp rx=0 tx=0).stop 2024-04-26T20:24:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 -- 172.21.15.131:0/847543314 shutdown_connections 2024-04-26T20:24:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f33150 0x55a912f355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f32ae0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 --2- 172.21.15.131:0/847543314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2d5f0 0x55a912f2da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 -- 172.21.15.131:0/847543314 >> 172.21.15.131:0/847543314 conn(0x55a912f28c60 msgr2=0x55a912f29060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 -- 172.21.15.131:0/847543314 shutdown_connections 2024-04-26T20:24:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.362+0000 7f0c80434740 1 -- 172.21.15.131:0/847543314 wait complete. 2024-04-26T20:24:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 -- start start 2024-04-26T20:24:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 Processor -- start 2024-04-26T20:24:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2dfe0 0x55a912f37d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f33150 0x55a912f3ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a912f36100 con 0x55a912f2d5f0 2024-04-26T20:24:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a912e10c60 con 0x55a912f33150 2024-04-26T20:24:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a912f36280 con 0x55a912f2dfe0 2024-04-26T20:24:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.363+0000 7f0c80434740 10 client.0 _ll_register_callbacks cb 0x55a912e3ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f37810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57950/0 (socket says 172.21.15.131:57950) 2024-04-26T20:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c77400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f33150 0x55a912f3ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c76a00640 1 -- 172.21.15.131:0/2204063459 learned_addr learned my addr 172.21.15.131:0/2204063459 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2dfe0 0x55a912f37d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c6fe00640 1 -- 172.21.15.131:0/2204063459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f33150 msgr2=0x55a912f3ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f33150 0x55a912f3ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c6fe00640 1 -- 172.21.15.131:0/2204063459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 msgr2=0x55a912f37810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f37810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c6fe00640 1 -- 172.21.15.131:0/2204063459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a912e1f510 con 0x55a912f2dfe0 2024-04-26T20:24:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c76a00640 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:15.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2dfe0 0x55a912f37d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009830 tx=0x7f0c60030d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9aaf254a5a4731 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.364+0000 7f0c5be00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:24:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c5be00640 1 -- 172.21.15.131:0/2204063459 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60004260 con 0x55a912f2dfe0 2024-04-26T20:24:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c6809d200 con 0x55a912f2dfe0 2024-04-26T20:24:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c5be00640 1 -- 172.21.15.131:0/2204063459 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0c60004400 con 0x55a912f2dfe0 2024-04-26T20:24:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c5be00640 1 -- 172.21.15.131:0/2204063459 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0c6004e050 con 0x55a912f2dfe0 2024-04-26T20:24:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c80434740 10 client.26191 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a912b9b210 con 0x55a912f2dfe0 2024-04-26T20:24:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c5be00640 1 -- 172.21.15.131:0/2204063459 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c6809d200 con 0x55a912f2dfe0 2024-04-26T20:24:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.365+0000 7f0c5be00640 1 -- 172.21.15.131:0/2204063459 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0c60004bf0 con 0x55a912f2dfe0 2024-04-26T20:24:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c80434740 10 client.26191 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c80434740 10 client.26191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c80434740 20 client.26191 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c80434740 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912f43300 0x55a912f45790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c80434740 4 client.26191 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a912e96f90 con 0x55a912f43300 2024-04-26T20:24:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c76a00640 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912f43300 0x55a912f45790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c76a00640 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912f43300 0x55a912f45790 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c46c62a23ddc8ea server_cookie=dbd55f98bb9c5f36 in_seq=0 out_seq=0 2024-04-26T20:24:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.366+0000 7f0c5be00640 10 client.26191 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.367+0000 7f0c5be00640 1 -- 172.21.15.131:0/2204063459 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a912e96f90 con 0x55a912f43300 2024-04-26T20:24:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.367+0000 7f0c5be00640 10 client.26191 handle_command_reply: tid=0 2024-04-26T20:24:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.367+0000 7f0c80434740 1 client.26191 shutdown 2024-04-26T20:24:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2dfe0 msgr2=0x55a912f37d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2dfe0 0x55a912f37d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009830 tx=0x7f0c60030d20 comp rx=0 tx=0).stop 2024-04-26T20:24:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 shutdown_connections 2024-04-26T20:24:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912f43300 0x55a912f45790 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f33150 0x55a912f3ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f2dfe0 0x55a912f37d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 --2- 172.21.15.131:0/2204063459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c5be00640 0 client.26191 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 >> 172.21.15.131:0/2204063459 conn(0x55a912f28c60 msgr2=0x55a912f1f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 shutdown_connections 2024-04-26T20:24:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 1 -- 172.21.15.131:0/2204063459 wait complete. 2024-04-26T20:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.368+0000 7f0c80434740 20 client.26191 trim_cache size 0 max 16384 2024-04-26T20:24:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 Processor -- start 2024-04-26T20:24:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 -- start start 2024-04-26T20:24:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f2b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f2bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781abab0 con 0x55a912f2d5f0 2024-04-26T20:24:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781a9380 con 0x55a912f2dfe0 2024-04-26T20:24:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c80434740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781a71d0 con 0x55a912f33150 2024-04-26T20:24:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f2b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f2b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57956/0 (socket says 172.21.15.131:57956) 2024-04-26T20:24:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.384+0000 7f0c76a00640 1 -- 172.21.15.131:0/1212863909 learned_addr learned my addr 172.21.15.131:0/1212863909 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c77400640 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f2bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c76a00640 1 -- 172.21.15.131:0/1212863909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 msgr2=0x55a912f2bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c76a00640 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f2bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c76a00640 1 -- 172.21.15.131:0/1212863909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 msgr2=0x55a912f3ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c76a00640 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c76a00640 1 -- 172.21.15.131:0/1212863909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a912c67620 con 0x55a912f2d5f0 2024-04-26T20:24:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c76a00640 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f2b9e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c68065d10 tx=0x7f0c680d48c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504392aac7973f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c66a00640 1 -- 172.21.15.131:0/1212863909 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68065180 con 0x55a912f2d5f0 2024-04-26T20:24:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c66a00640 1 -- 172.21.15.131:0/1212863909 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c68005680 con 0x55a912f2d5f0 2024-04-26T20:24:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.385+0000 7f0c66a00640 1 -- 172.21.15.131:0/1212863909 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c680653e0 con 0x55a912f2d5f0 2024-04-26T20:24:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 -- 172.21.15.131:0/1212863909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 msgr2=0x55a912f2b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f2b9e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c68065d10 tx=0x7f0c680d48c0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 -- 172.21.15.131:0/1212863909 shutdown_connections 2024-04-26T20:24:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 --2- 172.21.15.131:0/1212863909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f2b9e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 -- 172.21.15.131:0/1212863909 >> 172.21.15.131:0/1212863909 conn(0x55a912f28c60 msgr2=0x55a912f37810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 -- 172.21.15.131:0/1212863909 shutdown_connections 2024-04-26T20:24:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.386+0000 7f0c80434740 1 -- 172.21.15.131:0/1212863909 wait complete. 2024-04-26T20:24:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 -- start start 2024-04-26T20:24:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 Processor -- start 2024-04-26T20:24:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f3c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f3dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781a7840 con 0x55a912f2d5f0 2024-04-26T20:24:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781a8cb0 con 0x55a912f2dfe0 2024-04-26T20:24:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781a7f10 con 0x55a912f33150 2024-04-26T20:24:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c80434740 10 client.0 _ll_register_callbacks cb 0x7f0c600096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f3c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f3c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57964/0 (socket says 172.21.15.131:57964) 2024-04-26T20:24:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:56584/0 (socket says 172.21.15.131:56584) 2024-04-26T20:24:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.387+0000 7f0c76a00640 1 -- 172.21.15.131:0/2491506138 learned_addr learned my addr 172.21.15.131:0/2491506138 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c77400640 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f3dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c76a00640 1 -- 172.21.15.131:0/2491506138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 msgr2=0x55a912f3dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c76a00640 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f3dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c76a00640 1 -- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 msgr2=0x55a912f3cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c76a00640 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c76a00640 1 -- 172.21.15.131:0/2491506138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a912e43cb0 con 0x55a912f2d5f0 2024-04-26T20:24:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c76a00640 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f3c7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c680bd010 tx=0x7f0c6800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d52fd9811c2c7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c5be00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c5be00640 1 -- 172.21.15.131:0/2491506138 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68065600 con 0x55a912f2d5f0 2024-04-26T20:24:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c5be00640 1 -- 172.21.15.131:0/2491506138 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0c6806e5c0 con 0x55a912f2d5f0 2024-04-26T20:24:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.388+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c68005680 con 0x55a912f2d5f0 2024-04-26T20:24:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c5be00640 1 -- 172.21.15.131:0/2491506138 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0c680657f0 con 0x55a912f2d5f0 2024-04-26T20:24:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 10 client.16101 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a912fa9a00 con 0x55a912f2d5f0 2024-04-26T20:24:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c5be00640 1 -- 172.21.15.131:0/2491506138 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c68005680 con 0x55a912f2d5f0 2024-04-26T20:24:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c5be00640 1 -- 172.21.15.131:0/2491506138 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0c680990e0 con 0x55a912f2d5f0 2024-04-26T20:24:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 20 client.16101 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912fa9c80 0x55a912fac070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 4 client.16101 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.389+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a912f32ec0 con 0x55a912fa9c80 2024-04-26T20:24:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.390+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912fa9c80 0x55a912fac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.390+0000 7f0c6fe00640 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912fa9c80 0x55a912fac070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db72d487b1f678e7 server_cookie=a4113ddcbfa66cb8 in_seq=0 out_seq=0 2024-04-26T20:24:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.390+0000 7f0c5be00640 10 client.16101 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c5be00640 1 -- 172.21.15.131:0/2491506138 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a912f32ec0 con 0x55a912fa9c80 2024-04-26T20:24:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c5be00640 10 client.16101 handle_command_reply: tid=0 2024-04-26T20:24:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 client.16101 shutdown 2024-04-26T20:24:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 msgr2=0x55a912f3c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f3c7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c680bd010 tx=0x7f0c6800c7d0 comp rx=0 tx=0).stop 2024-04-26T20:24:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 shutdown_connections 2024-04-26T20:24:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a912fa9c80 0x55a912fac070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a912f33150 0x55a912f3dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912f2dfe0 0x55a912f3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 --2- 172.21.15.131:0/2491506138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a912f2d5f0 0x55a912f3c7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c5be00640 0 client.16101 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 >> 172.21.15.131:0/2491506138 conn(0x55a912f28c60 msgr2=0x55a912f2b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 shutdown_connections 2024-04-26T20:24:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 1 -- 172.21.15.131:0/2491506138 wait complete. 2024-04-26T20:24:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.394+0000 7f0c80434740 20 client.16101 trim_cache size 0 max 16384 2024-04-26T20:24:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c5c0a3cb0 msgr2=0x7f0c5c0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c5c0a3cb0 0x7f0c5c0a6170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55a912f2e840 tx=0x7f0c70010040 comp rx=0 tx=0).stop 2024-04-26T20:24:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781010c0 msgr2=0x7f0c7819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781010c0 0x7f0c7819ac70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800e9f0 tx=0x7f0c6800edf0 comp rx=0 tx=0).stop 2024-04-26T20:24:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 shutdown_connections 2024-04-26T20:24:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c781023c0 0x7f0c781a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0c5c0a3cb0 0x7f0c5c0a6170 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c78101a20 0x7f0c7819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 --2- 172.21.15.131:0/3590586600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c781010c0 0x7f0c7819ac70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 >> 172.21.15.131:0/3590586600 conn(0x7f0c780fcc60 msgr2=0x7f0c780fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 shutdown_connections 2024-04-26T20:24:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:01.395+0000 7f0c77e00640 1 -- 172.21.15.131:0/3590586600 wait complete. 2024-04-26T20:24:24.441 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:01 smithi131 ceph-mon[27919]: pgmap v770: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:24.448 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:03 smithi131 ceph-mon[27919]: pgmap v771: 129 pgs: 129 active+clean; 687 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:05 smithi131 ceph-mon[27919]: pgmap v772: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:24:24.554 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:07 smithi131 ceph-mon[27919]: pgmap v773: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:24.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:09 smithi131 ceph-mon[27919]: pgmap v774: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:24.706 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:11 smithi131 ceph-mon[27919]: pgmap v775: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:24.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:13 smithi131 ceph-mon[27919]: pgmap v776: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:24.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:11 smithi157 ceph-mon[32703]: pgmap v835: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:24.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:13 smithi157 ceph-mon[32703]: pgmap v836: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:24.886 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:15 smithi157 ceph-mon[32703]: pgmap v837: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:24.893 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:17 smithi157 ceph-mon[32703]: pgmap v838: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:24.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:19 smithi157 ceph-mon[32703]: pgmap v839: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:25.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:21 smithi157 ceph-mon[32703]: pgmap v840: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:25.064 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:23 smithi157 ceph-mon[32703]: pgmap v841: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:25.171 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:11 smithi148 ceph-mon[32281]: pgmap v835: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:25.214 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:13 smithi148 ceph-mon[32281]: pgmap v836: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:25.222 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:15 smithi148 ceph-mon[32281]: pgmap v837: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:25.229 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:17 smithi148 ceph-mon[32281]: pgmap v838: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:25.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:19 smithi148 ceph-mon[32281]: pgmap v839: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:25.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:21 smithi148 ceph-mon[32281]: pgmap v840: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:25.305 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:23 smithi148 ceph-mon[32281]: pgmap v841: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.536+0000 7f7801000640 1 -- 172.21.15.131:0/1056636411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc105db0 msgr2=0x7f77fc106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.536+0000 7f7801000640 1 --2- 172.21.15.131:0/1056636411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc105db0 0x7f77fc106230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009900 tx=0x7f77e402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:24:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.537+0000 7f7801000640 1 -- 172.21.15.131:0/1056636411 shutdown_connections 2024-04-26T20:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.537+0000 7f7801000640 1 --2- 172.21.15.131:0/1056636411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77fc106770 0x7f77fc10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.537+0000 7f7801000640 1 --2- 172.21.15.131:0/1056636411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc105db0 0x7f77fc106230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.537+0000 7f7801000640 1 --2- 172.21.15.131:0/1056636411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77fc104bc0 0x7f77fc104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.537+0000 7f7801000640 1 -- 172.21.15.131:0/1056636411 >> 172.21.15.131:0/1056636411 conn(0x7f77fc100150 msgr2=0x7f77fc102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.537+0000 7f7801000640 1 -- 172.21.15.131:0/1056636411 shutdown_connections 2024-04-26T20:24:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.537+0000 7f7801000640 1 -- 172.21.15.131:0/1056636411 wait complete. 2024-04-26T20:24:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 Processor -- start 2024-04-26T20:24:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 -- start start 2024-04-26T20:24:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc104bc0 0x7f77fc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77fc105db0 0x7f77fc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77fc106770 0x7f77fc1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1106f0 con 0x7f77fc105db0 2024-04-26T20:24:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc110570 con 0x7f77fc104bc0 2024-04-26T20:24:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f7801000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc110870 con 0x7f77fc106770 2024-04-26T20:24:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f77f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77fc105db0 0x7f77fc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f77f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77fc105db0 0x7f77fc19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43510/0 (socket says 172.21.15.131:43510) 2024-04-26T20:24:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f77f9400640 1 -- 172.21.15.131:0/2531535530 learned_addr learned my addr 172.21.15.131:0/2531535530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.538+0000 7f77f9e00640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc104bc0 0x7f77fc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77fa800640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77fc106770 0x7f77fc1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f9e00640 1 -- 172.21.15.131:0/2531535530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77fc106770 msgr2=0x7f77fc1a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f9e00640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77fc106770 0x7f77fc1a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f9e00640 1 -- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77fc105db0 msgr2=0x7f77fc19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f9e00640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77fc105db0 0x7f77fc19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f9e00640 1 -- 172.21.15.131:0/2531535530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc1a29c0 con 0x7f77fc104bc0 2024-04-26T20:24:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77fa800640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77fc106770 0x7f77fc1a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f9e00640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc104bc0 0x7f77fc19adb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00e9c0 tx=0x7f77ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f2a00640 1 -- 172.21.15.131:0/2531535530 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec00ccc0 con 0x7f77fc104bc0 2024-04-26T20:24:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc1a2cb0 con 0x7f77fc104bc0 2024-04-26T20:24:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.539+0000 7f77f2a00640 1 -- 172.21.15.131:0/2531535530 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77ec0035f0 con 0x7f77fc104bc0 2024-04-26T20:24:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.540+0000 7f77f2a00640 1 -- 172.21.15.131:0/2531535530 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec00ce60 con 0x7f77fc104bc0 2024-04-26T20:24:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.540+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1a31b0 con 0x7f77fc104bc0 2024-04-26T20:24:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 Processor -- start 2024-04-26T20:24:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 -- start start 2024-04-26T20:24:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f77f2a00640 1 -- 172.21.15.131:0/2531535530 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f77ec004000 con 0x7f77fc104bc0 2024-04-26T20:24:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a069728380 con 0x55a0697c3bd0 2024-04-26T20:24:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a0696c4c30 con 0x55a0697c9730 2024-04-26T20:24:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.543+0000 7f7803ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a069724340 con 0x55a0697c45c0 2024-04-26T20:24:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77fa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:56018/0 (socket says 172.21.15.131:56018) 2024-04-26T20:24:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9400640 1 -- 172.21.15.131:0/2995448897 learned_addr learned my addr 172.21.15.131:0/2995448897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9e00640 1 -- 172.21.15.131:0/2995448897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 msgr2=0x55a0697cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9e00640 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9e00640 1 -- 172.21.15.131:0/2995448897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 msgr2=0x55a0697c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9e00640 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9e00640 1 -- 172.21.15.131:0/2995448897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0696e6940 con 0x55a0697c45c0 2024-04-26T20:24:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77f9400640 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.544+0000 7f77fa800640 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.545+0000 7f77f9e00640 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697c90c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0ce310 tx=0x7f77ec0d4e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b601890965ab13e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.545+0000 7f77eb400640 1 -- 172.21.15.131:0/2995448897 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0d7ac0 con 0x55a0697c45c0 2024-04-26T20:24:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.545+0000 7f77eb400640 1 -- 172.21.15.131:0/2995448897 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77ec0d7c60 con 0x55a0697c45c0 2024-04-26T20:24:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.545+0000 7f77eb400640 1 -- 172.21.15.131:0/2995448897 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0d5d60 con 0x55a0697c45c0 2024-04-26T20:24:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.545+0000 7f77f2a00640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77e00a3d70 0x7f77e00a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.545+0000 7f77f2a00640 1 -- 172.21.15.131:0/2531535530 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f77ec0d0de0 con 0x7f77fc104bc0 2024-04-26T20:24:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.545+0000 7f77fa800640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77e00a3d70 0x7f77e00a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 -- 172.21.15.131:0/2995448897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 msgr2=0x55a0697c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697c90c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0ce310 tx=0x7f77ec0d4e00 comp rx=0 tx=0).stop 2024-04-26T20:24:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 -- 172.21.15.131:0/2995448897 shutdown_connections 2024-04-26T20:24:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697c90c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 --2- 172.21.15.131:0/2995448897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 -- 172.21.15.131:0/2995448897 >> 172.21.15.131:0/2995448897 conn(0x55a0697bf240 msgr2=0x55a0697bf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 -- 172.21.15.131:0/2995448897 shutdown_connections 2024-04-26T20:24:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 -- 172.21.15.131:0/2995448897 wait complete. 2024-04-26T20:24:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f77fa800640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77e00a3d70 0x7f77e00a6230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009fd0 tx=0x7f77f4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.546+0000 7f7803ddc740 1 -- start start 2024-04-26T20:24:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 1 Processor -- start 2024-04-26T20:24:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a0697cc6e0 con 0x55a0697c3bd0 2024-04-26T20:24:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a0696afc60 con 0x55a0697c9730 2024-04-26T20:24:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a0697cc860 con 0x55a0697c45c0 2024-04-26T20:24:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f7803ddc740 10 client.0 _ll_register_callbacks cb 0x55a0696dbe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f77f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f77f9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f77f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43526/0 (socket says 172.21.15.131:43526) 2024-04-26T20:24:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.547+0000 7f77f9e00640 1 -- 172.21.15.131:0/1389861399 learned_addr learned my addr 172.21.15.131:0/1389861399 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77fa800640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77f9e00640 1 -- 172.21.15.131:0/1389861399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 msgr2=0x55a0697ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77f9e00640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77f9e00640 1 -- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 msgr2=0x55a0697d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77f9e00640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77f9e00640 1 -- 172.21.15.131:0/1389861399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0696be510 con 0x55a0697c3bd0 2024-04-26T20:24:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77fa800640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697d53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77f9400640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77f9e00640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697cde20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f77ec098ee0 tx=0x7f77ec0d72e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33744ae661f25235 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77dfe00640 1 -- 172.21.15.131:0/1389861399 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec098d20 con 0x55a0697c3bd0 2024-04-26T20:24:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f77dfe00640 1 -- 172.21.15.131:0/1389861399 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f77ec09ae10 con 0x55a0697c3bd0 2024-04-26T20:24:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.548+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77ec0d7c60 con 0x55a0697c3bd0 2024-04-26T20:24:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f77dfe00640 1 -- 172.21.15.131:0/1389861399 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77ec0d7ac0 con 0x55a0697c3bd0 2024-04-26T20:24:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 10 client.16113 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a06943a210 con 0x55a0697c3bd0 2024-04-26T20:24:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f77dfe00640 1 -- 172.21.15.131:0/1389861399 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77ec0d7c60 con 0x55a0697c3bd0 2024-04-26T20:24:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f77dfe00640 1 -- 172.21.15.131:0/1389861399 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f77ec0ce590 con 0x55a0697c3bd0 2024-04-26T20:24:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 10 client.16113 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 20 client.16113 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a0697d9500 0x55a0697db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 4 client.16113 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.549+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a069735f90 con 0x55a0697d9500 2024-04-26T20:24:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.550+0000 7f77f9400640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a0697d9500 0x55a0697db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.550+0000 7f77f9400640 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a0697d9500 0x55a0697db8f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d24b72e365b2515 server_cookie=b55e4a2c2d38fd98 in_seq=0 out_seq=0 2024-04-26T20:24:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.550+0000 7f77dfe00640 10 client.16113 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f77dfe00640 1 -- 172.21.15.131:0/1389861399 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a069735f90 con 0x55a0697d9500 2024-04-26T20:24:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f77dfe00640 10 client.16113 handle_command_reply: tid=0 2024-04-26T20:24:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 client.16113 shutdown 2024-04-26T20:24:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 msgr2=0x55a0697cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697cde20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f77ec098ee0 tx=0x7f77ec0d72e0 comp rx=0 tx=0).stop 2024-04-26T20:24:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 shutdown_connections 2024-04-26T20:24:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a0697d9500 0x55a0697db8f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c9730 0x55a0697d53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c45c0 0x55a0697ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f77dfe00640 0 client.16113 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1389861399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c3bd0 0x55a0697cde20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 >> 172.21.15.131:0/1389861399 conn(0x55a0697bf240 msgr2=0x55a0697be440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 shutdown_connections 2024-04-26T20:24:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.551+0000 7f7803ddc740 1 -- 172.21.15.131:0/1389861399 wait complete. 2024-04-26T20:24:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.552+0000 7f7803ddc740 20 client.16113 trim_cache size 0 max 16384 2024-04-26T20:24:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 Processor -- start 2024-04-26T20:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 -- start start 2024-04-26T20:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c3bd0 0x55a0697c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c45c0 0x55a0697d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c9730 0x55a0697c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1b00e0 con 0x55a0697c9730 2024-04-26T20:24:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1ad9b0 con 0x55a0697c45c0 2024-04-26T20:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.568+0000 7f7803ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1ab800 con 0x55a0697c3bd0 2024-04-26T20:24:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c45c0 0x55a0697d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c45c0 0x55a0697d5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:56032/0 (socket says 172.21.15.131:56032) 2024-04-26T20:24:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 -- 172.21.15.131:0/1378215085 learned_addr learned my addr 172.21.15.131:0/1378215085 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77fa800640 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c9730 0x55a0697c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9e00640 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c3bd0 0x55a0697c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 -- 172.21.15.131:0/1378215085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c3bd0 msgr2=0x55a0697c21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c3bd0 0x55a0697c21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 -- 172.21.15.131:0/1378215085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c9730 msgr2=0x55a0697c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c9730 0x55a0697c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 -- 172.21.15.131:0/1378215085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a069506620 con 0x55a0697c45c0 2024-04-26T20:24:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77fa800640 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c9730 0x55a0697c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.569+0000 7f77f9400640 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c45c0 0x55a0697d5360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e4031870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7c4ca22128c227 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f77eaa00640 1 -- 172.21.15.131:0/1378215085 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4035040 con 0x55a0697c45c0 2024-04-26T20:24:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f77eaa00640 1 -- 172.21.15.131:0/1378215085 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e40468b0 con 0x55a0697c45c0 2024-04-26T20:24:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f77eaa00640 1 -- 172.21.15.131:0/1378215085 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40038e0 con 0x55a0697c45c0 2024-04-26T20:24:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 -- 172.21.15.131:0/1378215085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c45c0 msgr2=0x55a0697d5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c45c0 0x55a0697d5360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e4031870 comp rx=0 tx=0).stop 2024-04-26T20:24:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 -- 172.21.15.131:0/1378215085 shutdown_connections 2024-04-26T20:24:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c9730 0x55a0697c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c45c0 0x55a0697d5360 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 --2- 172.21.15.131:0/1378215085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c3bd0 0x55a0697c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 -- 172.21.15.131:0/1378215085 >> 172.21.15.131:0/1378215085 conn(0x55a0697bf240 msgr2=0x55a0697bf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 -- 172.21.15.131:0/1378215085 shutdown_connections 2024-04-26T20:24:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.570+0000 7f7803ddc740 1 -- 172.21.15.131:0/1378215085 wait complete. 2024-04-26T20:24:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 -- start start 2024-04-26T20:24:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 Processor -- start 2024-04-26T20:24:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c3bd0 0x55a0697d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c45c0 0x55a0697d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c9730 0x55a0697d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1abe70 con 0x55a0697c45c0 2024-04-26T20:24:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1ad2e0 con 0x55a0697c3bd0 2024-04-26T20:24:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1ac540 con 0x55a0697c9730 2024-04-26T20:24:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.571+0000 7f7803ddc740 10 client.0 _ll_register_callbacks cb 0x7f77f4009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c45c0 0x55a0697d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c9730 0x55a0697d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c9730 0x55a0697d3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37816/0 (socket says 172.21.15.131:37816) 2024-04-26T20:24:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 -- 172.21.15.131:0/3960538520 learned_addr learned my addr 172.21.15.131:0/3960538520 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c45c0 0x55a0697d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43554/0 (socket says 172.21.15.131:43554) 2024-04-26T20:24:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77f9e00640 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c3bd0 0x55a0697d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 -- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c3bd0 msgr2=0x55a0697d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c3bd0 0x55a0697d27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 -- 172.21.15.131:0/3960538520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c45c0 msgr2=0x55a0697d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c45c0 0x55a0697d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 -- 172.21.15.131:0/3960538520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0696e2cb0 con 0x55a0697c9730 2024-04-26T20:24:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77f9400640 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c45c0 0x55a0697d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77fa800640 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c9730 0x55a0697d3d20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f77f4010be0 tx=0x7f77f4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a47b8e42c609ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.572+0000 7f77dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:24:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f77dfe00640 1 -- 172.21.15.131:0/3960538520 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f401ecb0 con 0x55a0697c9730 2024-04-26T20:24:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f77dfe00640 1 -- 172.21.15.131:0/3960538520 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f77f4003970 con 0x55a0697c9730 2024-04-26T20:24:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e40468b0 con 0x55a0697c9730 2024-04-26T20:24:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f77dfe00640 1 -- 172.21.15.131:0/3960538520 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77f4034050 con 0x55a0697c9730 2024-04-26T20:24:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f7803ddc740 10 client.26215 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0697d54c0 con 0x55a0697c9730 2024-04-26T20:24:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f77dfe00640 1 -- 172.21.15.131:0/3960538520 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e40468b0 con 0x55a0697c9730 2024-04-26T20:24:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f77dfe00640 1 -- 172.21.15.131:0/3960538520 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f77f4028070 con 0x55a0697c9730 2024-04-26T20:24:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f7803ddc740 10 client.26215 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f7803ddc740 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f7803ddc740 20 client.26215 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.573+0000 7f7803ddc740 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a06983ffe0 0x55a069842400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.574+0000 7f7803ddc740 4 client.26215 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.574+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a06972fe00 con 0x55a06983ffe0 2024-04-26T20:24:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.574+0000 7f77f9e00640 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a06983ffe0 0x55a069842400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.574+0000 7f77f9e00640 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a06983ffe0 0x55a069842400 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e27f2d5c8e6745 server_cookie=f6d361a18413ac80 in_seq=0 out_seq=0 2024-04-26T20:24:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.574+0000 7f77dfe00640 10 client.26215 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f77dfe00640 1 -- 172.21.15.131:0/3960538520 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a06972fe00 con 0x55a06983ffe0 2024-04-26T20:24:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f77dfe00640 10 client.26215 handle_command_reply: tid=0 2024-04-26T20:24:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 client.26215 shutdown 2024-04-26T20:24:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c9730 msgr2=0x55a0697d3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c9730 0x55a0697d3d20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f77f4010be0 tx=0x7f77f4004ab0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 shutdown_connections 2024-04-26T20:24:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a06983ffe0 0x55a069842400 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0697c9730 0x55a0697d3d20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f77dfe00640 0 client.26215 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0697c45c0 0x55a0697d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 --2- 172.21.15.131:0/3960538520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0697c3bd0 0x55a0697d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 >> 172.21.15.131:0/3960538520 conn(0x55a0697bf240 msgr2=0x55a0697c21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 shutdown_connections 2024-04-26T20:24:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.578+0000 7f7803ddc740 1 -- 172.21.15.131:0/3960538520 wait complete. 2024-04-26T20:24:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7803ddc740 20 client.26215 trim_cache size 0 max 16384 2024-04-26T20:24:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77e00a3d70 msgr2=0x7f77e00a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77e00a3d70 0x7f77e00a6230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009fd0 tx=0x7f77f4009500 comp rx=0 tx=0).stop 2024-04-26T20:24:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc104bc0 msgr2=0x7f77fc19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc104bc0 0x7f77fc19adb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00e9c0 tx=0x7f77ec00ee80 comp rx=0 tx=0).stop 2024-04-26T20:24:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 shutdown_connections 2024-04-26T20:24:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77e00a3d70 0x7f77e00a6230 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77fc106770 0x7f77fc1a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77fc105db0 0x7f77fc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 --2- 172.21.15.131:0/2531535530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77fc104bc0 0x7f77fc19adb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 >> 172.21.15.131:0/2531535530 conn(0x7f77fc100150 msgr2=0x7f77fc109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.579+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 shutdown_connections 2024-04-26T20:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:14.580+0000 7f7801000640 1 -- 172.21.15.131:0/2531535530 wait complete. 2024-04-26T20:24:35.434 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:25 smithi148 ceph-mon[32281]: pgmap v842: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:35.478 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:27 smithi148 ceph-mon[32281]: pgmap v843: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:35.483 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:29 smithi148 ceph-mon[32281]: pgmap v844: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:24:35.491 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:31 smithi148 ceph-mon[32281]: pgmap v845: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:35.500 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:33 smithi148 ceph-mon[32281]: pgmap v846: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:35.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:25 smithi157 ceph-mon[32703]: pgmap v842: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:35.514 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:27 smithi157 ceph-mon[32703]: pgmap v843: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:35.522 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:29 smithi157 ceph-mon[32703]: pgmap v844: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:24:35.529 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:31 smithi157 ceph-mon[32703]: pgmap v845: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:35.535 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:33 smithi157 ceph-mon[32703]: pgmap v846: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:35.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:15 smithi131 ceph-mon[27919]: pgmap v777: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:35.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:17 smithi131 ceph-mon[27919]: pgmap v778: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:35.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:19 smithi131 ceph-mon[27919]: pgmap v779: 129 pgs: 129 active+clean; 688 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:24:35.575 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:21 smithi131 ceph-mon[27919]: pgmap v780: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:35.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:23 smithi131 ceph-mon[27919]: pgmap v781: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:35.589 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:25 smithi131 ceph-mon[27919]: pgmap v782: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 -- 172.21.15.131:0/2742014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8102990 msgr2=0x7fbfa8102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 --2- 172.21.15.131:0/2742014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8102990 0x7fbfa8102db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009900 tx=0x7fbf9802a410 comp rx=0 tx=0).stop 2024-04-26T20:24:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 -- 172.21.15.131:0/2742014349 shutdown_connections 2024-04-26T20:24:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 --2- 172.21.15.131:0/2742014349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8104540 0x7fbfa810add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 --2- 172.21.15.131:0/2742014349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8103b80 0x7fbfa8104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 --2- 172.21.15.131:0/2742014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8102990 0x7fbfa8102db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 -- 172.21.15.131:0/2742014349 >> 172.21.15.131:0/2742014349 conn(0x7fbfa80fdf20 msgr2=0x7fbfa8100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 -- 172.21.15.131:0/2742014349 shutdown_connections 2024-04-26T20:24:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.720+0000 7fbfad400640 1 -- 172.21.15.131:0/2742014349 wait complete. 2024-04-26T20:24:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.721+0000 7fbfad400640 1 Processor -- start 2024-04-26T20:24:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfad400640 1 -- start start 2024-04-26T20:24:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfad400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8102990 0x7fbfa80766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfad400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8103b80 0x7fbfa8074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfad400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8104540 0x7fbfa8075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfad400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e4e0 con 0x7fbfa8102990 2024-04-26T20:24:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfad400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e360 con 0x7fbfa8103b80 2024-04-26T20:24:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfad400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e660 con 0x7fbfa8104540 2024-04-26T20:24:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfa6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8102990 0x7fbfa80766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfa6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8102990 0x7fbfa80766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51406/0 (socket says 172.21.15.131:51406) 2024-04-26T20:24:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.722+0000 7fbfa6800640 1 -- 172.21.15.131:0/2278640729 learned_addr learned my addr 172.21.15.131:0/2278640729 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8103b80 0x7fbfa8074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbfa7200640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8104540 0x7fbfa8075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbfa6800640 1 -- 172.21.15.131:0/2278640729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8104540 msgr2=0x7fbfa8075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbfa6800640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8104540 0x7fbfa8075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbfa6800640 1 -- 172.21.15.131:0/2278640729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8103b80 msgr2=0x7fbfa8074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbfa6800640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8103b80 0x7fbfa8074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbfa6800640 1 -- 172.21.15.131:0/2278640729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa8075af0 con 0x7fbfa8102990 2024-04-26T20:24:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8103b80 0x7fbfa8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.723+0000 7fbfa6800640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8102990 0x7fbfa80766e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbf98031500 tx=0x7fbf98003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.724+0000 7fbf9f400640 1 -- 172.21.15.131:0/2278640729 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9803f070 con 0x7fbfa8102990 2024-04-26T20:24:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.724+0000 7fbf9f400640 1 -- 172.21.15.131:0/2278640729 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf98031c50 con 0x7fbfa8102990 2024-04-26T20:24:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.724+0000 7fbf9f400640 1 -- 172.21.15.131:0/2278640729 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98035030 con 0x7fbfa8102990 2024-04-26T20:24:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.724+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa81a2be0 con 0x7fbfa8102990 2024-04-26T20:24:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.724+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa81a30e0 con 0x7fbfa8102990 2024-04-26T20:24:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 Processor -- start 2024-04-26T20:24:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbf9f400640 1 -- 172.21.15.131:0/2278640729 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbf98042070 con 0x7fbfa8102990 2024-04-26T20:24:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 -- start start 2024-04-26T20:24:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae3700ebd0 0x55ae3700eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae37014730 0x55ae37016bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ae36f73380 con 0x55ae37014730 2024-04-26T20:24:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae36f0fc30 con 0x55ae3700f5c0 2024-04-26T20:24:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfb027e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ae36f6f340 con 0x55ae3700ebd0 2024-04-26T20:24:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbf9fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae37014730 0x55ae37016bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbf9fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae37014730 0x55ae37016bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51412/0 (socket says 172.21.15.131:51412) 2024-04-26T20:24:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.728+0000 7fbfa6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae3700ebd0 0x55ae3700eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbf9fe00640 1 -- 172.21.15.131:0/864285981 learned_addr learned my addr 172.21.15.131:0/864285981 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbfa7200640 1 -- 172.21.15.131:0/864285981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 msgr2=0x55ae370140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbfa7200640 1 --2- 172.21.15.131:0/864285981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbfa7200640 1 -- 172.21.15.131:0/864285981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae37014730 msgr2=0x55ae37016bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbfa7200640 1 --2- 172.21.15.131:0/864285981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae37014730 0x55ae37016bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbfa7200640 1 -- 172.21.15.131:0/864285981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae36f31940 con 0x55ae3700ebd0 2024-04-26T20:24:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbfa7200640 1 --2- 172.21.15.131:0/864285981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae3700ebd0 0x55ae3700eff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009f10 tx=0x7fbfa0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb472d9e2f2965d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.729+0000 7fbf9f400640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf8c0a3cf0 0x7fbf8c0a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbf97400640 1 -- 172.21.15.131:0/864285981 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0004d20 con 0x55ae3700ebd0 2024-04-26T20:24:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbf9f400640 1 -- 172.21.15.131:0/2278640729 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbf980f10c0 con 0x7fbfa8102990 2024-04-26T20:24:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbf97400640 1 -- 172.21.15.131:0/864285981 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa0004030 con 0x55ae3700ebd0 2024-04-26T20:24:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbf97400640 1 -- 172.21.15.131:0/864285981 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0005280 con 0x55ae3700ebd0 2024-04-26T20:24:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbfa6800640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf8c0a3cf0 0x7fbf8c0a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbfb027e740 1 -- 172.21.15.131:0/864285981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae3700ebd0 msgr2=0x55ae3700eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbfa6800640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf8c0a3cf0 0x7fbf8c0a61b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55ae3700f9c0 tx=0x7fbf980f4b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.730+0000 7fbfb027e740 1 --2- 172.21.15.131:0/864285981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae3700ebd0 0x55ae3700eff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009f10 tx=0x7fbfa0011810 comp rx=0 tx=0).stop 2024-04-26T20:24:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 -- 172.21.15.131:0/864285981 shutdown_connections 2024-04-26T20:24:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 --2- 172.21.15.131:0/864285981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae37014730 0x55ae37016bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 --2- 172.21.15.131:0/864285981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 --2- 172.21.15.131:0/864285981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae3700ebd0 0x55ae3700eff0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 -- 172.21.15.131:0/864285981 >> 172.21.15.131:0/864285981 conn(0x55ae3700a240 msgr2=0x55ae3700a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 -- 172.21.15.131:0/864285981 shutdown_connections 2024-04-26T20:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 -- 172.21.15.131:0/864285981 wait complete. 2024-04-26T20:24:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 -- start start 2024-04-26T20:24:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.731+0000 7fbfb027e740 1 Processor -- start 2024-04-26T20:24:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37018e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae37019350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae37020320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfb027e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ae370176e0 con 0x55ae3700ebd0 2024-04-26T20:24:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfb027e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae36efac60 con 0x55ae3700f5c0 2024-04-26T20:24:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfb027e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ae37017860 con 0x55ae37014730 2024-04-26T20:24:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfb027e740 10 client.0 _ll_register_callbacks cb 0x55ae36f26e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37018e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37018e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51414/0 (socket says 172.21.15.131:51414) 2024-04-26T20:24:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 -- 172.21.15.131:0/1648409284 learned_addr learned my addr 172.21.15.131:0/1648409284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae37019350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae37020320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 -- 172.21.15.131:0/1648409284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 msgr2=0x55ae37020320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae37020320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 -- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 msgr2=0x55ae37019350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae37019350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbfa7200640 1 -- 172.21.15.131:0/1648409284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae36f09510 con 0x55ae3700ebd0 2024-04-26T20:24:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.732+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae37019350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbfa7200640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37018e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005f40 tx=0x7fbfa0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15e131968fb5b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbf8be00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbf8be00640 1 -- 172.21.15.131:0/1648409284 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa002d060 con 0x55ae3700ebd0 2024-04-26T20:24:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbf8be00640 1 -- 172.21.15.131:0/1648409284 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbfa0003c30 con 0x55ae3700ebd0 2024-04-26T20:24:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa0004030 con 0x55ae3700ebd0 2024-04-26T20:24:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbf8be00640 1 -- 172.21.15.131:0/1648409284 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfa001e040 con 0x55ae3700ebd0 2024-04-26T20:24:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbfb027e740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.733+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae36c85210 con 0x55ae3700ebd0 2024-04-26T20:24:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbf8be00640 1 -- 172.21.15.131:0/1648409284 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa0004030 con 0x55ae3700ebd0 2024-04-26T20:24:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbf8be00640 1 -- 172.21.15.131:0/1648409284 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbfa0018070 con 0x55ae3700ebd0 2024-04-26T20:24:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbfb027e740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbfb027e740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbfb027e740 20 client.16131 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbfb027e740 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae370245a0 0x55ae37026990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbfb027e740 4 client.16131 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae36f80f90 con 0x55ae370245a0 2024-04-26T20:24:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.734+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae370245a0 0x55ae37026990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.735+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae370245a0 0x55ae37026990 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f862b83d50d48b32 server_cookie=e7abb34ad464b2ce in_seq=0 out_seq=0 2024-04-26T20:24:36.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.735+0000 7fbf8be00640 10 client.16131 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbf8be00640 1 -- 172.21.15.131:0/1648409284 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae36f80f90 con 0x55ae370245a0 2024-04-26T20:24:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbf8be00640 10 client.16131 handle_command_reply: tid=0 2024-04-26T20:24:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 client.16131 shutdown 2024-04-26T20:24:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 msgr2=0x55ae37018e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37018e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005f40 tx=0x7fbfa0009840 comp rx=0 tx=0).stop 2024-04-26T20:24:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 shutdown_connections 2024-04-26T20:24:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae370245a0 0x55ae37026990 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae37020320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae37019350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 --2- 172.21.15.131:0/1648409284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37018e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbf8be00640 0 client.16131 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 >> 172.21.15.131:0/1648409284 conn(0x55ae3700a240 msgr2=0x55ae37009440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 shutdown_connections 2024-04-26T20:24:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 1 -- 172.21.15.131:0/1648409284 wait complete. 2024-04-26T20:24:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.736+0000 7fbfb027e740 20 client.16131 trim_cache size 0 max 16384 2024-04-26T20:24:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.753+0000 7fbfb027e740 1 Processor -- start 2024-04-26T20:24:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.753+0000 7fbfb027e740 1 -- start start 2024-04-26T20:24:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37019130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3702b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfb027e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81b0080 con 0x55ae3700ebd0 2024-04-26T20:24:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfb027e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81ad950 con 0x55ae3700f5c0 2024-04-26T20:24:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfb027e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81ab7a0 con 0x55ae37014730 2024-04-26T20:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbf9fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbf9fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370202e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:37288/0 (socket says 172.21.15.131:37288) 2024-04-26T20:24:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbf9fe00640 1 -- 172.21.15.131:0/178523012 learned_addr learned my addr 172.21.15.131:0/178523012 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfa7200640 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37019130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.754+0000 7fbfa7200640 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3702b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf9fe00640 1 -- 172.21.15.131:0/178523012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 msgr2=0x55ae3702b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3702b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf9fe00640 1 -- 172.21.15.131:0/178523012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 msgr2=0x55ae37019130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37019130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf9fe00640 1 -- 172.21.15.131:0/178523012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae37026d00 con 0x55ae3700f5c0 2024-04-26T20:24:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbfa7200640 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3702b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:24:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370202e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ae37027220 tx=0x7fbf90005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fde7d231067922 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf96a00640 1 -- 172.21.15.131:0/178523012 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000fc80 con 0x55ae3700f5c0 2024-04-26T20:24:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf96a00640 1 -- 172.21.15.131:0/178523012 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9000fe20 con 0x55ae3700f5c0 2024-04-26T20:24:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.755+0000 7fbf96a00640 1 -- 172.21.15.131:0/178523012 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90004d00 con 0x55ae3700f5c0 2024-04-26T20:24:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 -- 172.21.15.131:0/178523012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 msgr2=0x55ae370202e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370202e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ae37027220 tx=0x7fbf90005f90 comp rx=0 tx=0).stop 2024-04-26T20:24:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 -- 172.21.15.131:0/178523012 shutdown_connections 2024-04-26T20:24:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3702b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700f5c0 0x55ae370202e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 --2- 172.21.15.131:0/178523012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700ebd0 0x55ae37019130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 -- 172.21.15.131:0/178523012 >> 172.21.15.131:0/178523012 conn(0x55ae3700a240 msgr2=0x55ae37018a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 -- 172.21.15.131:0/178523012 shutdown_connections 2024-04-26T20:24:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 -- 172.21.15.131:0/178523012 wait complete. 2024-04-26T20:24:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.756+0000 7fbfb027e740 1 -- start start 2024-04-26T20:24:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 1 Processor -- start 2024-04-26T20:24:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700ebd0 0x55ae3701d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700f5c0 0x55ae3701dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3701ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81abe10 con 0x55ae3700f5c0 2024-04-26T20:24:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81ad280 con 0x55ae3700ebd0 2024-04-26T20:24:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81ac4e0 con 0x55ae37014730 2024-04-26T20:24:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfb027e740 10 client.0 _ll_register_callbacks cb 0x7fbf90009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700ebd0 0x55ae3701d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700ebd0 0x55ae3701d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:37304/0 (socket says 172.21.15.131:37304) 2024-04-26T20:24:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfa7200640 1 -- 172.21.15.131:0/3055897747 learned_addr learned my addr 172.21.15.131:0/3055897747 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700f5c0 0x55ae3701dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.757+0000 7fbfa7200640 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3701ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf9fe00640 1 -- 172.21.15.131:0/3055897747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 msgr2=0x55ae3701ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3701ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf9fe00640 1 -- 172.21.15.131:0/3055897747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700ebd0 msgr2=0x55ae3701d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700ebd0 0x55ae3701d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf9fe00640 1 -- 172.21.15.131:0/3055897747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae36d51620 con 0x55ae3700f5c0 2024-04-26T20:24:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf9fe00640 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700f5c0 0x55ae3701dd30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55ae37023380 tx=0x7fbf90013fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36eddf2f9a23613 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf8be00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf8be00640 1 -- 172.21.15.131:0/3055897747 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000fc80 con 0x55ae3700f5c0 2024-04-26T20:24:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf9000fe20 con 0x55ae3700f5c0 2024-04-26T20:24:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.758+0000 7fbf8be00640 1 -- 172.21.15.131:0/3055897747 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbf90004030 con 0x55ae3700f5c0 2024-04-26T20:24:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbf8be00640 1 -- 172.21.15.131:0/3055897747 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf9003c050 con 0x55ae3700f5c0 2024-04-26T20:24:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 10 client.16137 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae36f2dcb0 con 0x55ae3700f5c0 2024-04-26T20:24:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbf8be00640 1 -- 172.21.15.131:0/3055897747 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9000fe20 con 0x55ae3700f5c0 2024-04-26T20:24:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbf8be00640 1 -- 172.21.15.131:0/3055897747 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbf90031020 con 0x55ae3700f5c0 2024-04-26T20:24:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 10 client.16137 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 20 client.16137 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae3708ad90 0x55ae3708d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 4 client.16137 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.759+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae36f7ae00 con 0x55ae3708ad90 2024-04-26T20:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.760+0000 7fbfa7200640 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae3708ad90 0x55ae3708d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.760+0000 7fbfa7200640 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae3708ad90 0x55ae3708d1b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e02d8276174543 server_cookie=1a41f59e57c177e7 in_seq=0 out_seq=0 2024-04-26T20:24:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.760+0000 7fbf8be00640 10 client.16137 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbf8be00640 1 -- 172.21.15.131:0/3055897747 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae36f7ae00 con 0x55ae3708ad90 2024-04-26T20:24:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbf8be00640 10 client.16137 handle_command_reply: tid=0 2024-04-26T20:24:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 client.16137 shutdown 2024-04-26T20:24:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700f5c0 msgr2=0x55ae3701dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700f5c0 0x55ae3701dd30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55ae37023380 tx=0x7fbf90013fd0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 shutdown_connections 2024-04-26T20:24:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ae3708ad90 0x55ae3708d1b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ae37014730 0x55ae3701ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae3700f5c0 0x55ae3701dd30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 --2- 172.21.15.131:0/3055897747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae3700ebd0 0x55ae3701d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 >> 172.21.15.131:0/3055897747 conn(0x55ae3700a240 msgr2=0x55ae37018a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbf8be00640 0 client.16137 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 shutdown_connections 2024-04-26T20:24:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 1 -- 172.21.15.131:0/3055897747 wait complete. 2024-04-26T20:24:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.764+0000 7fbfb027e740 20 client.16137 trim_cache size 0 max 16384 2024-04-26T20:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf8c0a3cf0 msgr2=0x7fbf8c0a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf8c0a3cf0 0x7fbf8c0a61b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55ae3700f9c0 tx=0x7fbf980f4b70 comp rx=0 tx=0).stop 2024-04-26T20:24:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8102990 msgr2=0x7fbfa80766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8102990 0x7fbfa80766e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbf98031500 tx=0x7fbf98003c10 comp rx=0 tx=0).stop 2024-04-26T20:24:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 shutdown_connections 2024-04-26T20:24:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf8c0a3cf0 0x7fbf8c0a61b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8104540 0x7fbfa8075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfa8103b80 0x7fbfa8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 --2- 172.21.15.131:0/2278640729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfa8102990 0x7fbfa80766e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.765+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 >> 172.21.15.131:0/2278640729 conn(0x7fbfa80fdf20 msgr2=0x7fbfa8106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.766+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 shutdown_connections 2024-04-26T20:24:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:27.766+0000 7fbfad400640 1 -- 172.21.15.131:0/2278640729 wait complete. 2024-04-26T20:24:39.027 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:27 smithi131 ceph-mon[27919]: pgmap v783: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:39.061 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:29 smithi131 ceph-mon[27919]: pgmap v784: 129 pgs: 129 active+clean; 689 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:39.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:31 smithi131 ceph-mon[27919]: pgmap v785: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:24:39.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:33 smithi131 ceph-mon[27919]: pgmap v786: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:39.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:35 smithi131 ceph-mon[27919]: pgmap v787: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:39.146 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:37 smithi131 ceph-mon[27919]: pgmap v788: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:39.156 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:39 smithi131 ceph-mon[27919]: pgmap v789: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:39.166 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:35 smithi148 ceph-mon[32281]: pgmap v847: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:39.175 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:37 smithi148 ceph-mon[32281]: pgmap v848: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:39.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:35 smithi157 ceph-mon[32703]: pgmap v847: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:39.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:37 smithi157 ceph-mon[32703]: pgmap v848: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 -- 172.21.15.131:0/4220722531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c075390 msgr2=0x7f244c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 --2- 172.21.15.131:0/4220722531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c075390 0x7f244c075810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2438009900 tx=0x7f243802a410 comp rx=0 tx=0).stop 2024-04-26T20:24:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 -- 172.21.15.131:0/4220722531 shutdown_connections 2024-04-26T20:24:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 --2- 172.21.15.131:0/4220722531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f244c10a680 0x7f244c10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 --2- 172.21.15.131:0/4220722531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c075390 0x7f244c075810 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 --2- 172.21.15.131:0/4220722531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c076970 0x7f244c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 -- 172.21.15.131:0/4220722531 >> 172.21.15.131:0/4220722531 conn(0x7f244c0fdd30 msgr2=0x7f244c100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 -- 172.21.15.131:0/4220722531 shutdown_connections 2024-04-26T20:24:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.908+0000 7f2450c00640 1 -- 172.21.15.131:0/4220722531 wait complete. 2024-04-26T20:24:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.909+0000 7f2450c00640 1 Processor -- start 2024-04-26T20:24:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.909+0000 7f2450c00640 1 -- start start 2024-04-26T20:24:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f2450c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c075390 0x7f244c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f2450c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f244c076970 0x7f244c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f2450c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c10a680 0x7f244c1a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f2450c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f244c110500 con 0x7f244c10a680 2024-04-26T20:24:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f2450c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f244c110380 con 0x7f244c076970 2024-04-26T20:24:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f2450c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f244c110680 con 0x7f244c075390 2024-04-26T20:24:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c075390 0x7f244c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c10a680 0x7f244c1a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c075390 0x7f244c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60146/0 (socket says 172.21.15.131:60146) 2024-04-26T20:24:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244b400640 1 -- 172.21.15.131:0/3576151168 learned_addr learned my addr 172.21.15.131:0/3576151168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244aa00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f244c076970 0x7f244c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244b400640 1 -- 172.21.15.131:0/3576151168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f244c076970 msgr2=0x7f244c19b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244b400640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f244c076970 0x7f244c19b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244b400640 1 -- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c10a680 msgr2=0x7f244c1a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.910+0000 7f244b400640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c10a680 0x7f244c1a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f244b400640 1 -- 172.21.15.131:0/3576151168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c1a2860 con 0x7f244c075390 2024-04-26T20:24:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f244be00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c10a680 0x7f244c1a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f244b400640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c075390 0x7f244c19ac50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f243c00ea60 tx=0x7f243c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f2443e00640 1 -- 172.21.15.131:0/3576151168 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c00cd20 con 0x7f244c075390 2024-04-26T20:24:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c1a2b50 con 0x7f244c075390 2024-04-26T20:24:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f2443e00640 1 -- 172.21.15.131:0/3576151168 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f243c0035f0 con 0x7f244c075390 2024-04-26T20:24:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f2443e00640 1 -- 172.21.15.131:0/3576151168 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c010690 con 0x7f244c075390 2024-04-26T20:24:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.911+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c1a30a0 con 0x7f244c075390 2024-04-26T20:24:39.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.914+0000 7f2443e00640 1 -- 172.21.15.131:0/3576151168 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f243c0037a0 con 0x7f244c075390 2024-04-26T20:24:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.914+0000 7f2455174740 1 Processor -- start 2024-04-26T20:24:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f2455174740 1 -- start start 2024-04-26T20:24:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f2455174740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f440de0 0x55bf0f441200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f2455174740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f4462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f2455174740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f446940 0x55bf0f448dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f2455174740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f3a5380 con 0x55bf0f4417d0 2024-04-26T20:24:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f2455174740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f341c30 con 0x55bf0f440de0 2024-04-26T20:24:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f2455174740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f3a1340 con 0x55bf0f446940 2024-04-26T20:24:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f244b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f4462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f244b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f4462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43170/0 (socket says 172.21.15.131:43170) 2024-04-26T20:24:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f244b400640 1 -- 172.21.15.131:0/1279851408 learned_addr learned my addr 172.21.15.131:0/1279851408 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.915+0000 7f244be00640 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f440de0 0x55bf0f441200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244aa00640 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f446940 0x55bf0f448dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244be00640 1 -- 172.21.15.131:0/1279851408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f446940 msgr2=0x55bf0f448dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244be00640 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f446940 0x55bf0f448dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244be00640 1 -- 172.21.15.131:0/1279851408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 msgr2=0x55bf0f4462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244be00640 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f4462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244be00640 1 -- 172.21.15.131:0/1279851408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0f363940 con 0x55bf0f440de0 2024-04-26T20:24:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f2443e00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24300a3c30 0x7f24300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244be00640 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f440de0 0x55bf0f441200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2444011760 tx=0x7f2444011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ed2f14ddaacb19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f2443e00640 1 -- 172.21.15.131:0/3576151168 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f243c0d0da0 con 0x7f244c075390 2024-04-26T20:24:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f244b400640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24300a3c30 0x7f24300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.916+0000 7f2437e00640 1 -- 172.21.15.131:0/1279851408 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444004d20 con 0x55bf0f440de0 2024-04-26T20:24:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.917+0000 7f2437e00640 1 -- 172.21.15.131:0/1279851408 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2444003970 con 0x55bf0f440de0 2024-04-26T20:24:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.917+0000 7f2437e00640 1 -- 172.21.15.131:0/1279851408 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444004410 con 0x55bf0f440de0 2024-04-26T20:24:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.917+0000 7f2455174740 1 -- 172.21.15.131:0/1279851408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f440de0 msgr2=0x55bf0f441200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.917+0000 7f2455174740 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f440de0 0x55bf0f441200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2444011760 tx=0x7f2444011c20 comp rx=0 tx=0).stop 2024-04-26T20:24:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.917+0000 7f244b400640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24300a3c30 0x7f24300a60f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f243c0d1040 tx=0x7f243c0d4850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 -- 172.21.15.131:0/1279851408 shutdown_connections 2024-04-26T20:24:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f446940 0x55bf0f448dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f4462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 --2- 172.21.15.131:0/1279851408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f440de0 0x55bf0f441200 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 -- 172.21.15.131:0/1279851408 >> 172.21.15.131:0/1279851408 conn(0x55bf0f43c450 msgr2=0x55bf0f43c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 -- 172.21.15.131:0/1279851408 shutdown_connections 2024-04-26T20:24:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 -- 172.21.15.131:0/1279851408 wait complete. 2024-04-26T20:24:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 -- start start 2024-04-26T20:24:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.918+0000 7f2455174740 1 Processor -- start 2024-04-26T20:24:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f2455174740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f2455174740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f2455174740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f4525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f2455174740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f4498f0 con 0x55bf0f4417d0 2024-04-26T20:24:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f2455174740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f32cc60 con 0x55bf0f446940 2024-04-26T20:24:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f2455174740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f449a70 con 0x55bf0f440de0 2024-04-26T20:24:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f2455174740 10 client.0 _ll_register_callbacks cb 0x55bf0f358e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f244be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f244be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60160/0 (socket says 172.21.15.131:60160) 2024-04-26T20:24:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f244aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f244be00640 1 -- 172.21.15.131:0/2086532452 learned_addr learned my addr 172.21.15.131:0/2086532452 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f244be00640 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f4525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.919+0000 7f244be00640 1 -- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 msgr2=0x55bf0f4525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f244be00640 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f4525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f244be00640 1 -- 172.21.15.131:0/2086532452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 msgr2=0x55bf0f44b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f244be00640 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f244be00640 1 -- 172.21.15.131:0/2086532452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0f33b510 con 0x55bf0f440de0 2024-04-26T20:24:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f244be00640 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44b020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2444004db0 tx=0x7f2444005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a707e323518aedf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f2434c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:24:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f2434c00640 1 -- 172.21.15.131:0/2086532452 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244401d030 con 0x55bf0f440de0 2024-04-26T20:24:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f2434c00640 1 -- 172.21.15.131:0/2086532452 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2444003d30 con 0x55bf0f440de0 2024-04-26T20:24:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2444003970 con 0x55bf0f440de0 2024-04-26T20:24:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f2434c00640 1 -- 172.21.15.131:0/2086532452 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2444035050 con 0x55bf0f440de0 2024-04-26T20:24:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f2455174740 10 client.26242 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.920+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf0f0b7210 con 0x55bf0f440de0 2024-04-26T20:24:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2434c00640 1 -- 172.21.15.131:0/2086532452 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2444003970 con 0x55bf0f440de0 2024-04-26T20:24:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2434c00640 1 -- 172.21.15.131:0/2086532452 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2444005450 con 0x55bf0f440de0 2024-04-26T20:24:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2455174740 10 client.26242 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2455174740 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2455174740 20 client.26242 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2455174740 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4568d0 0x55bf0f458cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2455174740 4 client.26242 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf0f3b2f90 con 0x55bf0f4568d0 2024-04-26T20:24:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.921+0000 7f244aa00640 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4568d0 0x55bf0f458cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.922+0000 7f244aa00640 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4568d0 0x55bf0f458cc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e95b8ea74e8dae7 server_cookie=db702db9d5d6d758 in_seq=0 out_seq=0 2024-04-26T20:24:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.922+0000 7f2434c00640 10 client.26242 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2434c00640 1 -- 172.21.15.131:0/2086532452 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf0f3b2f90 con 0x55bf0f4568d0 2024-04-26T20:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2434c00640 10 client.26242 handle_command_reply: tid=0 2024-04-26T20:24:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 client.26242 shutdown 2024-04-26T20:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 msgr2=0x55bf0f44b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44b020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2444004db0 tx=0x7f2444005bd0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 shutdown_connections 2024-04-26T20:24:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4568d0 0x55bf0f458cc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f4525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 --2- 172.21.15.131:0/2086532452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44b020 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2434c00640 0 client.26242 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 >> 172.21.15.131:0/2086532452 conn(0x55bf0f43c450 msgr2=0x55bf0f43b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 shutdown_connections 2024-04-26T20:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 1 -- 172.21.15.131:0/2086532452 wait complete. 2024-04-26T20:24:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.923+0000 7f2455174740 20 client.26242 trim_cache size 0 max 16384 2024-04-26T20:24:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.939+0000 7f2455174740 1 Processor -- start 2024-04-26T20:24:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.939+0000 7f2455174740 1 -- start start 2024-04-26T20:24:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f2455174740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f43f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f2455174740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f452560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f2455174740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f43fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f2455174740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1b0070 con 0x55bf0f4417d0 2024-04-26T20:24:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f2455174740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ad940 con 0x55bf0f446940 2024-04-26T20:24:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f2455174740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ab790 con 0x55bf0f440de0 2024-04-26T20:24:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f452560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f452560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43188/0 (socket says 172.21.15.131:43188) 2024-04-26T20:24:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 -- 172.21.15.131:0/1059802335 learned_addr learned my addr 172.21.15.131:0/1059802335 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244be00640 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f43f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244be00640 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f43fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 -- 172.21.15.131:0/1059802335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 msgr2=0x55bf0f43f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f43f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 -- 172.21.15.131:0/1059802335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 msgr2=0x55bf0f43fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f43fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244aa00640 1 -- 172.21.15.131:0/1059802335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0f183620 con 0x55bf0f4417d0 2024-04-26T20:24:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.940+0000 7f244be00640 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f43fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.941+0000 7f244aa00640 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f452560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2438009230 tx=0x7f24380314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad222a40878af0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.941+0000 7f2437400640 1 -- 172.21.15.131:0/1059802335 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438035040 con 0x55bf0f4417d0 2024-04-26T20:24:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.941+0000 7f2437400640 1 -- 172.21.15.131:0/1059802335 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f243804a070 con 0x55bf0f4417d0 2024-04-26T20:24:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.941+0000 7f2437400640 1 -- 172.21.15.131:0/1059802335 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380458c0 con 0x55bf0f4417d0 2024-04-26T20:24:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.941+0000 7f2455174740 1 -- 172.21.15.131:0/1059802335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 msgr2=0x55bf0f452560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.941+0000 7f2455174740 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f452560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2438009230 tx=0x7f24380314c0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 -- 172.21.15.131:0/1059802335 shutdown_connections 2024-04-26T20:24:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f43fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f452560 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 --2- 172.21.15.131:0/1059802335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f43f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 -- 172.21.15.131:0/1059802335 >> 172.21.15.131:0/1059802335 conn(0x55bf0f43c450 msgr2=0x55bf0f43c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 -- 172.21.15.131:0/1059802335 shutdown_connections 2024-04-26T20:24:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 -- 172.21.15.131:0/1059802335 wait complete. 2024-04-26T20:24:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 -- start start 2024-04-26T20:24:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.942+0000 7f2455174740 1 Processor -- start 2024-04-26T20:24:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f2455174740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f2455174740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f2455174740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f450bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f2455174740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1abe00 con 0x55bf0f4417d0 2024-04-26T20:24:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f2455174740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ad270 con 0x55bf0f446940 2024-04-26T20:24:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f2455174740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ac4d0 con 0x55bf0f440de0 2024-04-26T20:24:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f2455174740 10 client.0 _ll_register_callbacks cb 0x7f243c0d4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43190/0 (socket says 172.21.15.131:43190) 2024-04-26T20:24:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 -- 172.21.15.131:0/3203069864 learned_addr learned my addr 172.21.15.131:0/3203069864 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244be00640 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f450bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244be00640 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 -- 172.21.15.131:0/3203069864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 msgr2=0x55bf0f44f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 -- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 msgr2=0x55bf0f450bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f450bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.943+0000 7f244aa00640 1 -- 172.21.15.131:0/3203069864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0f35fcb0 con 0x55bf0f4417d0 2024-04-26T20:24:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f244aa00640 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44fba0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f243802a450 tx=0x7f243802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6ab0d943598df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2434c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2434c00640 1 -- 172.21.15.131:0/3203069864 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438052030 con 0x55bf0f4417d0 2024-04-26T20:24:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2434c00640 1 -- 172.21.15.131:0/3203069864 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2438005250 con 0x55bf0f4417d0 2024-04-26T20:24:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243804a070 con 0x55bf0f4417d0 2024-04-26T20:24:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2434c00640 1 -- 172.21.15.131:0/3203069864 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2438035050 con 0x55bf0f4417d0 2024-04-26T20:24:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2455174740 10 client.16155 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf0f452310 con 0x55bf0f4417d0 2024-04-26T20:24:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.944+0000 7f2434c00640 1 -- 172.21.15.131:0/3203069864 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f243804a070 con 0x55bf0f4417d0 2024-04-26T20:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f2434c00640 1 -- 172.21.15.131:0/3203069864 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f24380458e0 con 0x55bf0f4417d0 2024-04-26T20:24:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f2455174740 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f2455174740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f2455174740 20 client.16155 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f2455174740 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4bd020 0x55bf0f4bf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f2455174740 4 client.16155 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf0f3ace00 con 0x55bf0f4bd020 2024-04-26T20:24:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.945+0000 7f244be00640 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4bd020 0x55bf0f4bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.946+0000 7f244be00640 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4bd020 0x55bf0f4bf440 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10fb347574759d89 server_cookie=1a2b1feaa1ccd76d in_seq=0 out_seq=0 2024-04-26T20:24:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.946+0000 7f2434c00640 10 client.16155 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.949+0000 7f2434c00640 1 -- 172.21.15.131:0/3203069864 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf0f3ace00 con 0x55bf0f4bd020 2024-04-26T20:24:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.949+0000 7f2434c00640 10 client.16155 handle_command_reply: tid=0 2024-04-26T20:24:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.949+0000 7f2455174740 1 client.16155 shutdown 2024-04-26T20:24:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 msgr2=0x55bf0f44fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44fba0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f243802a450 tx=0x7f243802ab10 comp rx=0 tx=0).stop 2024-04-26T20:24:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 shutdown_connections 2024-04-26T20:24:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55bf0f4bd020 0x55bf0f4bf440 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf0f446940 0x55bf0f450bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf0f4417d0 0x55bf0f44fba0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 --2- 172.21.15.131:0/3203069864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf0f440de0 0x55bf0f44f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2434c00640 0 client.16155 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 >> 172.21.15.131:0/3203069864 conn(0x55bf0f43c450 msgr2=0x55bf0f43f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 shutdown_connections 2024-04-26T20:24:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 1 -- 172.21.15.131:0/3203069864 wait complete. 2024-04-26T20:24:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.950+0000 7f2455174740 20 client.16155 trim_cache size 0 max 16384 2024-04-26T20:24:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24300a3c30 msgr2=0x7f24300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24300a3c30 0x7f24300a60f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f243c0d1040 tx=0x7f243c0d4850 comp rx=0 tx=0).stop 2024-04-26T20:24:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c075390 msgr2=0x7f244c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c075390 0x7f244c19ac50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f243c00ea60 tx=0x7f243c00ee60 comp rx=0 tx=0).stop 2024-04-26T20:24:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 shutdown_connections 2024-04-26T20:24:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c10a680 0x7f244c1a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f244c076970 0x7f244c19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24300a3c30 0x7f24300a60f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 --2- 172.21.15.131:0/3576151168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c075390 0x7f244c19ac50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 >> 172.21.15.131:0/3576151168 conn(0x7f244c0fdd30 msgr2=0x7f244c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 shutdown_connections 2024-04-26T20:24:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:40.951+0000 7f2450c00640 1 -- 172.21.15.131:0/3576151168 wait complete. 2024-04-26T20:24:43.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:41 smithi131 ceph-mon[27919]: pgmap v790: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:43.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:43 smithi131 ceph-mon[27919]: pgmap v791: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:43.570 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:45 smithi131 ceph-mon[27919]: pgmap v792: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:43.632 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:47 smithi131 ceph-mon[27919]: pgmap v793: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:43.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:49 smithi131 ceph-mon[27919]: pgmap v794: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:43.653 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:51 smithi131 ceph-mon[27919]: pgmap v795: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:43.663 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:39 smithi148 ceph-mon[32281]: pgmap v849: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:43.716 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:41 smithi148 ceph-mon[32281]: pgmap v850: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:43.723 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:39 smithi157 ceph-mon[32703]: pgmap v849: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:43.733 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:41 smithi157 ceph-mon[32703]: pgmap v850: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 -- 172.21.15.131:0/1778805967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 msgr2=0x7f36ec101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 --2- 172.21.15.131:0/1778805967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 0x7f36ec101440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009900 tx=0x7f36e002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 -- 172.21.15.131:0/1778805967 shutdown_connections 2024-04-26T20:24:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 --2- 172.21.15.131:0/1778805967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec102320 0x7f36ec1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 --2- 172.21.15.131:0/1778805967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ec101980 0x7f36ec101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 --2- 172.21.15.131:0/1778805967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 0x7f36ec101440 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 -- 172.21.15.131:0/1778805967 >> 172.21.15.131:0/1778805967 conn(0x7f36ec0fcc20 msgr2=0x7f36ec0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 -- 172.21.15.131:0/1778805967 shutdown_connections 2024-04-26T20:24:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.099+0000 7f36f2600640 1 -- 172.21.15.131:0/1778805967 wait complete. 2024-04-26T20:24:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.100+0000 7f36f2600640 1 Processor -- start 2024-04-26T20:24:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.100+0000 7f36f2600640 1 -- start start 2024-04-26T20:24:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f2600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 0x7f36ec19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f2600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec101980 0x7f36ec19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f2600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ec102320 0x7f36ec1a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f2600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec110510 con 0x7f36ec101980 2024-04-26T20:24:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f2600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec110390 con 0x7f36ec102320 2024-04-26T20:24:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f2600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec110690 con 0x7f36ec101020 2024-04-26T20:24:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec101980 0x7f36ec19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec101980 0x7f36ec19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59738/0 (socket says 172.21.15.131:59738) 2024-04-26T20:24:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 -- 172.21.15.131:0/1038099630 learned_addr learned my addr 172.21.15.131:0/1038099630 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f1200640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 0x7f36ec19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f1c00640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ec102320 0x7f36ec1a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 -- 172.21.15.131:0/1038099630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 msgr2=0x7f36ec19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 0x7f36ec19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 -- 172.21.15.131:0/1038099630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ec102320 msgr2=0x7f36ec1a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ec102320 0x7f36ec1a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36ebe00640 1 -- 172.21.15.131:0/1038099630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec1a2810 con 0x7f36ec101980 2024-04-26T20:24:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.101+0000 7f36f1c00640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ec102320 0x7f36ec1a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.102+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec101980 0x7f36ec19b130 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9f0 tx=0x7f36d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.102+0000 7f36e9600640 1 -- 172.21.15.131:0/1038099630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d800cd80 con 0x7f36ec101980 2024-04-26T20:24:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.102+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec1a2b00 con 0x7f36ec101980 2024-04-26T20:24:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.102+0000 7f36e9600640 1 -- 172.21.15.131:0/1038099630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80054d0 con 0x7f36ec101980 2024-04-26T20:24:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.102+0000 7f36e9600640 1 -- 172.21.15.131:0/1038099630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8010690 con 0x7f36ec101980 2024-04-26T20:24:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.102+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a3050 con 0x7f36ec101980 2024-04-26T20:24:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.105+0000 7f36e9600640 1 -- 172.21.15.131:0/1038099630 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f36d8004e90 con 0x7f36ec101980 2024-04-26T20:24:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 Processor -- start 2024-04-26T20:24:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36e9600640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36d40a3cb0 0x7f36d40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 -- start start 2024-04-26T20:24:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36e9600640 1 -- 172.21.15.131:0/1038099630 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f36d80d0d40 con 0x7f36ec101980 2024-04-26T20:24:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f1c00640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36d40a3cb0 0x7f36d40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157ade0 0x562c1157b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c115802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c11580940 0x562c11582dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562c114df380 con 0x562c1157b7d0 2024-04-26T20:24:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c1147bc30 con 0x562c1157ade0 2024-04-26T20:24:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.107+0000 7f36f697e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562c114db340 con 0x562c11580940 2024-04-26T20:24:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c115802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36f1200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157ade0 0x562c1157b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c115802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59746/0 (socket says 172.21.15.131:59746) 2024-04-26T20:24:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 -- 172.21.15.131:0/1187892462 learned_addr learned my addr 172.21.15.131:0/1187892462 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36f1c00640 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c11580940 0x562c11582dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 -- 172.21.15.131:0/1187892462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c11580940 msgr2=0x562c11582dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c11580940 0x562c11582dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 -- 172.21.15.131:0/1187892462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157ade0 msgr2=0x562c1157b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157ade0 0x562c1157b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 -- 172.21.15.131:0/1187892462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1149d940 con 0x562c1157b7d0 2024-04-26T20:24:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36f1200640 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157ade0 0x562c1157b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36f1c00640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36d40a3cb0 0x7f36d40a6170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36e4005fd0 tx=0x7f36e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.108+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c115802d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36d809ca00 tx=0x7f36d809c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b03ab2e2fa8a44e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.109+0000 7f36de000640 1 -- 172.21.15.131:0/1187892462 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d809ca70 con 0x562c1157b7d0 2024-04-26T20:24:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.109+0000 7f36de000640 1 -- 172.21.15.131:0/1187892462 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80a0e20 con 0x562c1157b7d0 2024-04-26T20:24:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.109+0000 7f36de000640 1 -- 172.21.15.131:0/1187892462 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80a5250 con 0x562c1157b7d0 2024-04-26T20:24:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.109+0000 7f36f697e740 1 -- 172.21.15.131:0/1187892462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 msgr2=0x562c115802d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.109+0000 7f36f697e740 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c115802d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36d809ca00 tx=0x7f36d809c490 comp rx=0 tx=0).stop 2024-04-26T20:24:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 -- 172.21.15.131:0/1187892462 shutdown_connections 2024-04-26T20:24:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c11580940 0x562c11582dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c115802d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 --2- 172.21.15.131:0/1187892462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157ade0 0x562c1157b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 -- 172.21.15.131:0/1187892462 >> 172.21.15.131:0/1187892462 conn(0x562c11576450 msgr2=0x562c11576850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 -- 172.21.15.131:0/1187892462 shutdown_connections 2024-04-26T20:24:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 -- 172.21.15.131:0/1187892462 wait complete. 2024-04-26T20:24:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.110+0000 7f36f697e740 1 -- start start 2024-04-26T20:24:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 1 Processor -- start 2024-04-26T20:24:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11584fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157b7d0 0x562c11585500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c11580940 0x562c1158c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562c115838f0 con 0x562c11580940 2024-04-26T20:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c11466c60 con 0x562c1157b7d0 2024-04-26T20:24:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562c11583a70 con 0x562c1157ade0 2024-04-26T20:24:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.111+0000 7f36f697e740 10 client.0 _ll_register_callbacks cb 0x562c11492e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36f1c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c11580940 0x562c1158c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157b7d0 0x562c11585500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36f1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11584fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157b7d0 0x562c11585500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57980/0 (socket says 172.21.15.131:57980) 2024-04-26T20:24:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 -- 172.21.15.131:0/1561126004 learned_addr learned my addr 172.21.15.131:0/1561126004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36f1c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c11580940 0x562c1158c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59752/0 (socket says 172.21.15.131:59752) 2024-04-26T20:24:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 -- 172.21.15.131:0/1561126004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 msgr2=0x562c11584fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11584fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 -- 172.21.15.131:0/1561126004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c11580940 msgr2=0x562c1158c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c11580940 0x562c1158c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 -- 172.21.15.131:0/1561126004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c11475510 con 0x562c1157b7d0 2024-04-26T20:24:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36f1c00640 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c11580940 0x562c1158c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36ebe00640 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157b7d0 0x562c11585500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f36d80a51b0 tx=0x7f36d8004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a8027d6fd9a9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:24:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.112+0000 7f36d2a00640 1 -- 172.21.15.131:0/1561126004 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80aa060 con 0x562c1157b7d0 2024-04-26T20:24:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d80a0e20 con 0x562c1157b7d0 2024-04-26T20:24:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36d2a00640 1 -- 172.21.15.131:0/1561126004 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f36d809c0d0 con 0x562c1157b7d0 2024-04-26T20:24:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36d2a00640 1 -- 172.21.15.131:0/1561126004 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36d80ce5d0 con 0x562c1157b7d0 2024-04-26T20:24:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 10 client.26285 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c111f1210 con 0x562c1157b7d0 2024-04-26T20:24:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36d2a00640 1 -- 172.21.15.131:0/1561126004 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80a0e20 con 0x562c1157b7d0 2024-04-26T20:24:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36d2a00640 1 -- 172.21.15.131:0/1561126004 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f36d80b9070 con 0x562c1157b7d0 2024-04-26T20:24:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 10 client.26285 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 10 client.26285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 20 client.26285 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115906a0 0x562c11592a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 4 client.26285 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.113+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c114ecf90 con 0x562c115906a0 2024-04-26T20:24:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.114+0000 7f36f1200640 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115906a0 0x562c11592a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.114+0000 7f36d2a00640 10 client.26285 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.114+0000 7f36f1200640 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115906a0 0x562c11592a90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8042552b04c3ee89 server_cookie=5d1ead8b1dadf2f1 in_seq=0 out_seq=0 2024-04-26T20:24:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.115+0000 7f36d2a00640 1 -- 172.21.15.131:0/1561126004 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c114ecf90 con 0x562c115906a0 2024-04-26T20:24:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.115+0000 7f36d2a00640 10 client.26285 handle_command_reply: tid=0 2024-04-26T20:24:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.115+0000 7f36f697e740 1 client.26285 shutdown 2024-04-26T20:24:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157b7d0 msgr2=0x562c11585500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157b7d0 0x562c11585500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f36d80a51b0 tx=0x7f36d8004a90 comp rx=0 tx=0).stop 2024-04-26T20:24:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 shutdown_connections 2024-04-26T20:24:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115906a0 0x562c11592a90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c11580940 0x562c1158c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36d2a00640 0 client.26285 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c1157b7d0 0x562c11585500 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 --2- 172.21.15.131:0/1561126004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11584fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 >> 172.21.15.131:0/1561126004 conn(0x562c11576450 msgr2=0x562c11575650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 shutdown_connections 2024-04-26T20:24:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 1 -- 172.21.15.131:0/1561126004 wait complete. 2024-04-26T20:24:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.116+0000 7f36f697e740 20 client.26285 trim_cache size 0 max 16384 2024-04-26T20:24:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.147+0000 7f36f697e740 1 Processor -- start 2024-04-26T20:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36f697e740 1 -- start start 2024-04-26T20:24:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36f697e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157ade0 0x562c115798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36f697e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157b7d0 0x562c1158c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36f697e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c115982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36f697e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1aff60 con 0x562c1157ade0 2024-04-26T20:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36f697e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ad830 con 0x562c11580940 2024-04-26T20:24:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36f697e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ab680 con 0x562c1157b7d0 2024-04-26T20:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157b7d0 0x562c1158c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157b7d0 0x562c1158c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37416/0 (socket says 172.21.15.131:37416) 2024-04-26T20:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.148+0000 7f36ebe00640 1 -- 172.21.15.131:0/3663248044 learned_addr learned my addr 172.21.15.131:0/3663248044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36f1200640 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157ade0 0x562c115798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36f1c00640 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c115982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36ebe00640 1 -- 172.21.15.131:0/3663248044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 msgr2=0x562c115982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36ebe00640 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c115982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36ebe00640 1 -- 172.21.15.131:0/3663248044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157ade0 msgr2=0x562c115798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36ebe00640 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157ade0 0x562c115798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36ebe00640 1 -- 172.21.15.131:0/3663248044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c112bd620 con 0x562c1157b7d0 2024-04-26T20:24:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36f1c00640 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c115982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36f1200640 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157ade0 0x562c115798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36ebe00640 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157b7d0 0x562c1158c500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36d8098ed0 tx=0x7f36d80bd190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31716b5654b70085 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36dd600640 1 -- 172.21.15.131:0/3663248044 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80aa030 con 0x562c1157b7d0 2024-04-26T20:24:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36dd600640 1 -- 172.21.15.131:0/3663248044 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80a0e20 con 0x562c1157b7d0 2024-04-26T20:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.149+0000 7f36dd600640 1 -- 172.21.15.131:0/3663248044 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80a56e0 con 0x562c1157b7d0 2024-04-26T20:24:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 -- 172.21.15.131:0/3663248044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157b7d0 msgr2=0x562c1158c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157b7d0 0x562c1158c500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36d8098ed0 tx=0x7f36d80bd190 comp rx=0 tx=0).stop 2024-04-26T20:24:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 -- 172.21.15.131:0/3663248044 shutdown_connections 2024-04-26T20:24:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c115982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157b7d0 0x562c1158c500 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 --2- 172.21.15.131:0/3663248044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157ade0 0x562c115798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 -- 172.21.15.131:0/3663248044 >> 172.21.15.131:0/3663248044 conn(0x562c11576450 msgr2=0x562c115791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 -- 172.21.15.131:0/3663248044 shutdown_connections 2024-04-26T20:24:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.150+0000 7f36f697e740 1 -- 172.21.15.131:0/3663248044 wait complete. 2024-04-26T20:24:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 -- start start 2024-04-26T20:24:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 Processor -- start 2024-04-26T20:24:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11589d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c1158a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c1158b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1abcf0 con 0x562c1157b7d0 2024-04-26T20:24:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ad160 con 0x562c11580940 2024-04-26T20:24:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ac3c0 con 0x562c1157ade0 2024-04-26T20:24:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.151+0000 7f36f697e740 10 client.0 _ll_register_callbacks cb 0x7f36d8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11589d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11589d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37432/0 (socket says 172.21.15.131:37432) 2024-04-26T20:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 -- 172.21.15.131:0/2743857044 learned_addr learned my addr 172.21.15.131:0/2743857044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36ebe00640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c1158a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1c00640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c1158b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 -- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 msgr2=0x562c1158b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c1158b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 -- 172.21.15.131:0/2743857044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 msgr2=0x562c1158a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c1158a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 -- 172.21.15.131:0/2743857044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c11499cb0 con 0x562c1157ade0 2024-04-26T20:24:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36ebe00640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c1158a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1c00640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c1158b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36f1200640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11589d60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f36e002a9b0 tx=0x7f36e0009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f79b88ca2a0442b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.152+0000 7f36d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:24:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36d2a00640 1 -- 172.21.15.131:0/2743857044 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0035040 con 0x562c1157ade0 2024-04-26T20:24:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36d2a00640 1 -- 172.21.15.131:0/2743857044 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f36e00467e0 con 0x562c1157ade0 2024-04-26T20:24:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d80a0e20 con 0x562c1157ade0 2024-04-26T20:24:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36d2a00640 1 -- 172.21.15.131:0/2743857044 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36e0057050 con 0x562c1157ade0 2024-04-26T20:24:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36f697e740 10 client.26266 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c115f6b10 con 0x562c1157ade0 2024-04-26T20:24:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36d2a00640 1 -- 172.21.15.131:0/2743857044 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80a0e20 con 0x562c1157ade0 2024-04-26T20:24:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.153+0000 7f36d2a00640 1 -- 172.21.15.131:0/2743857044 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f36e00038c0 con 0x562c1157ade0 2024-04-26T20:24:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36f697e740 10 client.26266 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36f697e740 10 client.26266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36f697e740 20 client.26266 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36f697e740 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115f7430 0x562c115f9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36f697e740 4 client.26266 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c11584db0 con 0x562c115f7430 2024-04-26T20:24:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36ebe00640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115f7430 0x562c115f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36ebe00640 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115f7430 0x562c115f9850 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4946117509313c94 server_cookie=721f50b4035b2d56 in_seq=0 out_seq=0 2024-04-26T20:24:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.154+0000 7f36d2a00640 10 client.26266 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.157+0000 7f36d2a00640 1 -- 172.21.15.131:0/2743857044 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c11584db0 con 0x562c115f7430 2024-04-26T20:24:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.157+0000 7f36d2a00640 10 client.26266 handle_command_reply: tid=0 2024-04-26T20:24:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.157+0000 7f36f697e740 1 client.26266 shutdown 2024-04-26T20:24:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 msgr2=0x562c11589d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11589d60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f36e002a9b0 tx=0x7f36e0009880 comp rx=0 tx=0).stop 2024-04-26T20:24:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 shutdown_connections 2024-04-26T20:24:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c115f7430 0x562c115f9850 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c11580940 0x562c1158b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c1157b7d0 0x562c1158a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 --2- 172.21.15.131:0/2743857044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c1157ade0 0x562c11589d60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36d2a00640 0 client.26266 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 >> 172.21.15.131:0/2743857044 conn(0x562c11576450 msgr2=0x562c11574dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 shutdown_connections 2024-04-26T20:24:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 1 -- 172.21.15.131:0/2743857044 wait complete. 2024-04-26T20:24:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.158+0000 7f36f697e740 20 client.26266 trim_cache size 0 max 16384 2024-04-26T20:24:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36d40a3cb0 msgr2=0x7f36d40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36d40a3cb0 0x7f36d40a6170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36e4005fd0 tx=0x7f36e4009500 comp rx=0 tx=0).stop 2024-04-26T20:24:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec101980 msgr2=0x7f36ec19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec101980 0x7f36ec19b130 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9f0 tx=0x7f36d800edf0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 shutdown_connections 2024-04-26T20:24:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ec102320 0x7f36ec1a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36ec101980 0x7f36ec19b130 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36d40a3cb0 0x7f36d40a6170 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 --2- 172.21.15.131:0/1038099630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec101020 0x7f36ec19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 >> 172.21.15.131:0/1038099630 conn(0x7f36ec0fcc20 msgr2=0x7f36ec0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 shutdown_connections 2024-04-26T20:24:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:22:54.159+0000 7f36f2600640 1 -- 172.21.15.131:0/1038099630 wait complete. 2024-04-26T20:24:48.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:43 smithi148 ceph-mon[32281]: pgmap v851: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:48.086 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:45 smithi148 ceph-mon[32281]: pgmap v852: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:48.093 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:43 smithi157 ceph-mon[32703]: pgmap v851: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:48.098 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:45 smithi157 ceph-mon[32703]: pgmap v852: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:48.105 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:47 smithi157 ceph-mon[32703]: pgmap v853: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:48.175 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:53 smithi131 ceph-mon[27919]: pgmap v796: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:48.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:53 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:24:48.188 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:24:48.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:24:48.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:54 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:24:48.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:55 smithi131 ceph-mon[27919]: pgmap v797: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:48.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:56 smithi131 ceph-mon[27919]: pgmap v798: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:48.294 INFO:tasks.workunit.client.0.smithi131.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T20:24:48.322 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T20:24:48.334 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T20:24:48.344 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T20:24:48.356 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T20:24:48.367 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T20:24:48.427 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T20:24:48.439 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T20:24:48.449 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T20:24:48.459 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:24:48.502 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T20:24:48.511 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:47 smithi148 ceph-mon[32281]: pgmap v853: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:48.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:22:59 smithi131 ceph-mon[27919]: pgmap v799: 129 pgs: 129 active+clean; 690 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T20:24:48.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:01 smithi131 ceph-mon[27919]: pgmap v800: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:24:48.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:03 smithi131 ceph-mon[27919]: pgmap v801: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:48.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:05 smithi131 ceph-mon[27919]: pgmap v802: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 -- 172.21.15.131:0/3546968828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 msgr2=0x7ff9f010a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3546968828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f010a630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009860 tx=0x7ff9dc02a340 comp rx=0 tx=0).stop 2024-04-26T20:24:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 -- 172.21.15.131:0/3546968828 shutdown_connections 2024-04-26T20:24:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3546968828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 0x7ff9f0111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3546968828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f010a630 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3546968828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f0108fc0 0x7ff9f01093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 -- 172.21.15.131:0/3546968828 >> 172.21.15.131:0/3546968828 conn(0x7ff9f0077f80 msgr2=0x7ff9f007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 -- 172.21.15.131:0/3546968828 shutdown_connections 2024-04-26T20:24:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.297+0000 7ff9f8600640 1 -- 172.21.15.131:0/3546968828 wait complete. 2024-04-26T20:24:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.298+0000 7ff9f8600640 1 Processor -- start 2024-04-26T20:24:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.298+0000 7ff9f8600640 1 -- start start 2024-04-26T20:24:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.298+0000 7ff9f8600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f0108fc0 0x7ff9f019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.298+0000 7ff9f8600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.298+0000 7ff9f8600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 0x7ff9f01a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.298+0000 7ff9f8600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0114b00 con 0x7ff9f010ab70 2024-04-26T20:24:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.298+0000 7ff9f8600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0114980 con 0x7ff9f010a1b0 2024-04-26T20:24:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f8600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0114c80 con 0x7ff9f0108fc0 2024-04-26T20:24:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 0x7ff9f01a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 0x7ff9f01a6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34962/0 (socket says 172.21.15.131:34962) 2024-04-26T20:24:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f6600640 1 -- 172.21.15.131:0/3997860199 learned_addr learned my addr 172.21.15.131:0/3997860199 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:47162/0 (socket says 172.21.15.131:47162) 2024-04-26T20:24:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f0108fc0 0x7ff9f019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5200640 1 -- 172.21.15.131:0/3997860199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f0108fc0 msgr2=0x7ff9f019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5200640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f0108fc0 0x7ff9f019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5200640 1 -- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 msgr2=0x7ff9f01a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5200640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 0x7ff9f01a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f5200640 1 -- 172.21.15.131:0/3997860199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f01a6d80 con 0x7ff9f010a1b0 2024-04-26T20:24:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.299+0000 7ff9f6600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 0x7ff9f01a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.300+0000 7ff9f5200640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f019f6a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc02a860 tx=0x7ff9dc032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.300+0000 7ff9ee000640 1 -- 172.21.15.131:0/3997860199 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc040020 con 0x7ff9f010a1b0 2024-04-26T20:24:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.300+0000 7ff9ee000640 1 -- 172.21.15.131:0/3997860199 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9dc030d50 con 0x7ff9f010a1b0 2024-04-26T20:24:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.300+0000 7ff9ee000640 1 -- 172.21.15.131:0/3997860199 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc037040 con 0x7ff9f010a1b0 2024-04-26T20:24:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.300+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f01a7010 con 0x7ff9f010a1b0 2024-04-26T20:24:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.300+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f01a74b0 con 0x7ff9f010a1b0 2024-04-26T20:24:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.303+0000 7ff9ee000640 1 -- 172.21.15.131:0/3997860199 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff9dc043020 con 0x7ff9f010a1b0 2024-04-26T20:24:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.303+0000 7ff9fb3f2740 1 Processor -- start 2024-04-26T20:24:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9fb3f2740 1 -- start start 2024-04-26T20:24:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298efff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556229854380 con 0x5562298efbd0 2024-04-26T20:24:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562297f0c30 con 0x5562298f5730 2024-04-26T20:24:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556229850340 con 0x5562298f05c0 2024-04-26T20:24:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9f6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9f5c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9f6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298efff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34964/0 (socket says 172.21.15.131:34964) 2024-04-26T20:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9f6600640 1 -- 172.21.15.131:0/2039199061 learned_addr learned my addr 172.21.15.131:0/2039199061 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.304+0000 7ff9f5200640 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9f5c00640 1 -- 172.21.15.131:0/2039199061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 msgr2=0x5562298f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9f5c00640 1 -- 172.21.15.131:0/2039199061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 msgr2=0x5562298efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9f5c00640 1 -- 172.21.15.131:0/2039199061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556229812940 con 0x5562298f05c0 2024-04-26T20:24:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298f50c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009bd0 tx=0x7ff9e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c9547c3e6e7770 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9ee000640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff9d80a3d70 0x7ff9d80a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9e2a00640 1 -- 172.21.15.131:0/2039199061 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004230 con 0x5562298f05c0 2024-04-26T20:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9e2a00640 1 -- 172.21.15.131:0/2039199061 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e4009e40 con 0x5562298f05c0 2024-04-26T20:24:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9e2a00640 1 -- 172.21.15.131:0/2039199061 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4019660 con 0x5562298f05c0 2024-04-26T20:24:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.305+0000 7ff9ee000640 1 -- 172.21.15.131:0/3997860199 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff9dc0f04b0 con 0x7ff9f010a1b0 2024-04-26T20:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9f6600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff9d80a3d70 0x7ff9d80a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2039199061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 msgr2=0x5562298f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298f50c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009bd0 tx=0x7ff9e4011810 comp rx=0 tx=0).stop 2024-04-26T20:24:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2039199061 shutdown_connections 2024-04-26T20:24:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298f50c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2039199061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2039199061 >> 172.21.15.131:0/2039199061 conn(0x5562298eb240 msgr2=0x5562298eb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2039199061 shutdown_connections 2024-04-26T20:24:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9f6600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff9d80a3d70 0x7ff9d80a6230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5562298f0e20 tx=0x7ff9e80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.306+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2039199061 wait complete. 2024-04-26T20:24:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 -- start start 2024-04-26T20:24:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 Processor -- start 2024-04-26T20:24:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f05c0 0x5562298fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f5730 0x556229901330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5562298f86e0 con 0x5562298efbd0 2024-04-26T20:24:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562297dbc60 con 0x5562298f05c0 2024-04-26T20:24:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.307+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5562298f8860 con 0x5562298f5730 2024-04-26T20:24:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9fb3f2740 10 client.0 _ll_register_callbacks cb 0x556229807e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f05c0 0x5562298fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34970/0 (socket says 172.21.15.131:34970) 2024-04-26T20:24:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5c00640 1 -- 172.21.15.131:0/2662661220 learned_addr learned my addr 172.21.15.131:0/2662661220 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f6600640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f5730 0x556229901330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5200640 1 -- 172.21.15.131:0/2662661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f5730 msgr2=0x556229901330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5200640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f5730 0x556229901330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5200640 1 -- 172.21.15.131:0/2662661220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 msgr2=0x5562298f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5200640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5200640 1 -- 172.21.15.131:0/2662661220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562297ea510 con 0x5562298f05c0 2024-04-26T20:24:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f6600640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f5730 0x556229901330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9f5200640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f05c0 0x5562298fa2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0edb60 tx=0x7ff9dc0b85d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e87b8412686e388 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.308+0000 7ff9d7400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:24:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9d7400640 1 -- 172.21.15.131:0/2662661220 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0bbc90 con 0x5562298f05c0 2024-04-26T20:24:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556229861f90 con 0x5562298f05c0 2024-04-26T20:24:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9d7400640 1 -- 172.21.15.131:0/2662661220 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff9dc0f3c90 con 0x5562298f05c0 2024-04-26T20:24:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9d7400640 1 -- 172.21.15.131:0/2662661220 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9dc0bca30 con 0x5562298f05c0 2024-04-26T20:24:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9fb3f2740 10 client.26303 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556229566210 con 0x5562298f05c0 2024-04-26T20:24:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9d7400640 1 -- 172.21.15.131:0/2662661220 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556229861f90 con 0x5562298f05c0 2024-04-26T20:24:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.309+0000 7ff9d7400640 1 -- 172.21.15.131:0/2662661220 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff9dc04d060 con 0x5562298f05c0 2024-04-26T20:24:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.310+0000 7ff9fb3f2740 10 client.26303 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.310+0000 7ff9fb3f2740 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.310+0000 7ff9fb3f2740 20 client.26303 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.310+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556229905870 0x556229907ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.310+0000 7ff9fb3f2740 4 client.26303 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.310+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55622985be00 con 0x556229905870 2024-04-26T20:24:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.310+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556229905870 0x556229907ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.311+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556229905870 0x556229907ca0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927f8d609e39a135 server_cookie=4054e8a58b9938ed in_seq=0 out_seq=0 2024-04-26T20:24:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.311+0000 7ff9d7400640 10 client.26303 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.311+0000 7ff9d7400640 1 -- 172.21.15.131:0/2662661220 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55622985be00 con 0x556229905870 2024-04-26T20:24:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.311+0000 7ff9d7400640 10 client.26303 handle_command_reply: tid=0 2024-04-26T20:24:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 client.26303 shutdown 2024-04-26T20:24:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f05c0 msgr2=0x5562298fa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f05c0 0x5562298fa2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0edb60 tx=0x7ff9dc0b85d0 comp rx=0 tx=0).stop 2024-04-26T20:24:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 shutdown_connections 2024-04-26T20:24:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556229905870 0x556229907ca0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f5730 0x556229901330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f05c0 0x5562298fa2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2662661220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 >> 172.21.15.131:0/2662661220 conn(0x5562298eb240 msgr2=0x5562298ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9d7400640 0 client.26303 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 shutdown_connections 2024-04-26T20:24:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2662661220 wait complete. 2024-04-26T20:24:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.312+0000 7ff9fb3f2740 20 client.26303 trim_cache size 0 max 16384 2024-04-26T20:24:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.328+0000 7ff9fb3f2740 1 Processor -- start 2024-04-26T20:24:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.328+0000 7ff9fb3f2740 1 -- start start 2024-04-26T20:24:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298f05c0 0x5562299012f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b4410 con 0x5562298f05c0 2024-04-26T20:24:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b1ce0 con 0x5562298f5730 2024-04-26T20:24:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01afb30 con 0x5562298efbd0 2024-04-26T20:24:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9f5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298f05c0 0x5562299012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9f6600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9f5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298f05c0 0x5562299012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34980/0 (socket says 172.21.15.131:34980) 2024-04-26T20:24:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9f5200640 1 -- 172.21.15.131:0/3222703635 learned_addr learned my addr 172.21.15.131:0/3222703635 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.329+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9f5200640 1 -- 172.21.15.131:0/3222703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298efbd0 msgr2=0x5562298f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9f5200640 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9f5200640 1 -- 172.21.15.131:0/3222703635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 msgr2=0x5562298fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9f5200640 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298fa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9f5200640 1 -- 172.21.15.131:0/3222703635 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556229632620 con 0x5562298f05c0 2024-04-26T20:24:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9f6600640 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9f5200640 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298f05c0 0x5562299012f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc005820 tx=0x7ff9dc0ba840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc29cc69c7e986f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9e2000640 1 -- 172.21.15.131:0/3222703635 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0bc100 con 0x5562298f05c0 2024-04-26T20:24:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9e2000640 1 -- 172.21.15.131:0/3222703635 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9dc0bc330 con 0x5562298f05c0 2024-04-26T20:24:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.330+0000 7ff9e2000640 1 -- 172.21.15.131:0/3222703635 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc030550 con 0x5562298f05c0 2024-04-26T20:24:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/3222703635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298f05c0 msgr2=0x5562299012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298f05c0 0x5562299012f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc005820 tx=0x7ff9dc0ba840 comp rx=0 tx=0).stop 2024-04-26T20:24:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/3222703635 shutdown_connections 2024-04-26T20:24:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x5562298fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298f05c0 0x5562299012f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/3222703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298efbd0 0x5562298f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/3222703635 >> 172.21.15.131:0/3222703635 conn(0x5562298eb240 msgr2=0x5562298f1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/3222703635 shutdown_connections 2024-04-26T20:24:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.331+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/3222703635 wait complete. 2024-04-26T20:24:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 -- start start 2024-04-26T20:24:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 Processor -- start 2024-04-26T20:24:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298fe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x556229969de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b01a0 con 0x5562298efbd0 2024-04-26T20:24:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b1610 con 0x5562298f5730 2024-04-26T20:24:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b0870 con 0x5562298f05c0 2024-04-26T20:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.332+0000 7ff9fb3f2740 10 client.0 _ll_register_callbacks cb 0x7ff9dc0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f6600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x556229969de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f6600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x556229969de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:47186/0 (socket says 172.21.15.131:47186) 2024-04-26T20:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f6600640 1 -- 172.21.15.131:0/2196832301 learned_addr learned my addr 172.21.15.131:0/2196832301 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298fe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34996/0 (socket says 172.21.15.131:34996) 2024-04-26T20:24:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5200640 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 -- 172.21.15.131:0/2196832301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 msgr2=0x5562298fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298fe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 -- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 msgr2=0x556229969de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x556229969de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 -- 172.21.15.131:0/2196832301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55622980ecb0 con 0x5562298efbd0 2024-04-26T20:24:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f6600640 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x556229969de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9f5c00640 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298fe190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4005890 tx=0x7ff9e40055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c764688005381e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.333+0000 7ff9d7400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9d7400640 1 -- 172.21.15.131:0/2196832301 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4009780 con 0x5562298efbd0 2024-04-26T20:24:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9d7400640 1 -- 172.21.15.131:0/2196832301 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff9e4003740 con 0x5562298efbd0 2024-04-26T20:24:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9dc0bc330 con 0x5562298efbd0 2024-04-26T20:24:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9d7400640 1 -- 172.21.15.131:0/2196832301 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9e403a050 con 0x5562298efbd0 2024-04-26T20:24:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 10 client.16191 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:52.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55622996b3e0 con 0x5562298efbd0 2024-04-26T20:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9d7400640 1 -- 172.21.15.131:0/2196832301 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9dc0bc330 con 0x5562298efbd0 2024-04-26T20:24:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9d7400640 1 -- 172.21.15.131:0/2196832301 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff9dc0edca0 con 0x5562298efbd0 2024-04-26T20:24:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:52.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 20 client.16191 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55622996bba0 0x55622996df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 4 client.16191 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.334+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562297d74b0 con 0x55622996bba0 2024-04-26T20:24:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.335+0000 7ff9f5200640 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55622996bba0 0x55622996df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.335+0000 7ff9f5200640 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55622996bba0 0x55622996df90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96768e6629260303 server_cookie=24a23ebc1d00ff38 in_seq=0 out_seq=0 2024-04-26T20:24:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.335+0000 7ff9d7400640 10 client.16191 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9d7400640 1 -- 172.21.15.131:0/2196832301 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562297d74b0 con 0x55622996bba0 2024-04-26T20:24:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9d7400640 10 client.16191 handle_command_reply: tid=0 2024-04-26T20:24:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 client.16191 shutdown 2024-04-26T20:24:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 msgr2=0x5562298fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298fe190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4005890 tx=0x7ff9e40055c0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 shutdown_connections 2024-04-26T20:24:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55622996bba0 0x55622996df90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562298f5730 0x556229969de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562298f05c0 0x5562298fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 --2- 172.21.15.131:0/2196832301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562298efbd0 0x5562298fe190 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9d7400640 0 client.16191 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 >> 172.21.15.131:0/2196832301 conn(0x5562298eb240 msgr2=0x5562298ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 shutdown_connections 2024-04-26T20:24:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 1 -- 172.21.15.131:0/2196832301 wait complete. 2024-04-26T20:24:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.339+0000 7ff9fb3f2740 20 client.16191 trim_cache size 0 max 16384 2024-04-26T20:24:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff9d80a3d70 msgr2=0x7ff9d80a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff9d80a3d70 0x7ff9d80a6230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5562298f0e20 tx=0x7ff9e80095c0 comp rx=0 tx=0).stop 2024-04-26T20:24:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 msgr2=0x7ff9f019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f019f6a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc02a860 tx=0x7ff9dc032d50 comp rx=0 tx=0).stop 2024-04-26T20:24:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 shutdown_connections 2024-04-26T20:24:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9f010ab70 0x7ff9f01a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f010a1b0 0x7ff9f019f6a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff9d80a3d70 0x7ff9d80a6230 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 --2- 172.21.15.131:0/3997860199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f0108fc0 0x7ff9f019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 >> 172.21.15.131:0/3997860199 conn(0x7ff9f0077f80 msgr2=0x7ff9f010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 shutdown_connections 2024-04-26T20:24:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:07.340+0000 7ff9f8600640 1 -- 172.21.15.131:0/3997860199 wait complete. 2024-04-26T20:24:53.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:49 smithi157 ceph-mon[32703]: pgmap v854: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:53.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:51 smithi157 ceph-mon[32703]: pgmap v855: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:53.243 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:49 smithi148 ceph-mon[32281]: pgmap v854: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:53.284 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:51 smithi148 ceph-mon[32281]: pgmap v855: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:53.293 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:07 smithi131 ceph-mon[27919]: pgmap v803: 129 pgs: 129 active+clean; 691 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:53.301 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:09 smithi131 ceph-mon[27919]: pgmap v804: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:24:53.311 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:11 smithi131 ceph-mon[27919]: pgmap v805: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:24:53.318 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:13 smithi131 ceph-mon[27919]: pgmap v806: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:53.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:15 smithi131 ceph-mon[27919]: pgmap v807: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:53.377 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:17 smithi131 ceph-mon[27919]: pgmap v808: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:53.384 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:19 smithi131 ceph-mon[27919]: pgmap v809: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.481+0000 7fc1ada00640 1 -- 172.21.15.131:0/3860728525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810a1b0 msgr2=0x7fc1a810a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.481+0000 7fc1ada00640 1 --2- 172.21.15.131:0/3860728525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810a1b0 0x7fc1a810a630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc190009900 tx=0x7fc19002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.481+0000 7fc1ada00640 1 -- 172.21.15.131:0/3860728525 shutdown_connections 2024-04-26T20:24:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.481+0000 7fc1ada00640 1 --2- 172.21.15.131:0/3860728525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810ab70 0x7fc1a8111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.481+0000 7fc1ada00640 1 --2- 172.21.15.131:0/3860728525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810a1b0 0x7fc1a810a630 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.481+0000 7fc1ada00640 1 --2- 172.21.15.131:0/3860728525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 0x7fc1a81093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.481+0000 7fc1ada00640 1 -- 172.21.15.131:0/3860728525 >> 172.21.15.131:0/3860728525 conn(0x7fc1a8077f80 msgr2=0x7fc1a807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.482+0000 7fc1ada00640 1 -- 172.21.15.131:0/3860728525 shutdown_connections 2024-04-26T20:24:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.482+0000 7fc1ada00640 1 -- 172.21.15.131:0/3860728525 wait complete. 2024-04-26T20:24:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 Processor -- start 2024-04-26T20:24:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 -- start start 2024-04-26T20:24:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 0x7fc1a819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810a1b0 0x7fc1a819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810ab70 0x7fc1a81a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8114b50 con 0x7fc1a810a1b0 2024-04-26T20:24:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81149d0 con 0x7fc1a8108fc0 2024-04-26T20:24:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.483+0000 7fc1ada00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8114cd0 con 0x7fc1a810ab70 2024-04-26T20:24:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 0x7fc1a819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810a1b0 0x7fc1a819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810a1b0 0x7fc1a819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43514/0 (socket says 172.21.15.131:43514) 2024-04-26T20:24:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 0x7fc1a819f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:41188/0 (socket says 172.21.15.131:41188) 2024-04-26T20:24:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc19fe00640 1 -- 172.21.15.131:0/578536488 learned_addr learned my addr 172.21.15.131:0/578536488 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a7400640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810ab70 0x7fc1a81a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a6a00640 1 -- 172.21.15.131:0/578536488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810ab70 msgr2=0x7fc1a81a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a6a00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810ab70 0x7fc1a81a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a6a00640 1 -- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810a1b0 msgr2=0x7fc1a819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a6a00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810a1b0 0x7fc1a819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc1a6a00640 1 -- 172.21.15.131:0/578536488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a81a6d90 con 0x7fc1a8108fc0 2024-04-26T20:24:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.484+0000 7fc19fe00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810a1b0 0x7fc1a819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.485+0000 7fc1a6a00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 0x7fc1a819f170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9c0 tx=0x7fc19800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.485+0000 7fc19f400640 1 -- 172.21.15.131:0/578536488 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800ccc0 con 0x7fc1a8108fc0 2024-04-26T20:24:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.485+0000 7fc19f400640 1 -- 172.21.15.131:0/578536488 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1980035f0 con 0x7fc1a8108fc0 2024-04-26T20:24:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.485+0000 7fc19f400640 1 -- 172.21.15.131:0/578536488 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800ce60 con 0x7fc1a8108fc0 2024-04-26T20:24:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.485+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81a7080 con 0x7fc1a8108fc0 2024-04-26T20:24:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.485+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81a7580 con 0x7fc1a8108fc0 2024-04-26T20:24:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.488+0000 7fc19f400640 1 -- 172.21.15.131:0/578536488 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc1980037a0 con 0x7fc1a8108fc0 2024-04-26T20:24:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.488+0000 7fc1b076d740 1 Processor -- start 2024-04-26T20:24:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1b076d740 1 -- start start 2024-04-26T20:24:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a45bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1b076d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5576129a2380 con 0x557612a3e5c0 2024-04-26T20:24:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1b076d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55761293ec30 con 0x557612a43730 2024-04-26T20:24:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1b076d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55761299e340 con 0x557612a3dbd0 2024-04-26T20:24:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44234/0 (socket says 172.21.15.131:44234) 2024-04-26T20:24:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc1a7400640 1 -- 172.21.15.131:0/407455557 learned_addr learned my addr 172.21.15.131:0/407455557 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.489+0000 7fc19fe00640 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a45bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.490+0000 7fc1a7400640 1 -- 172.21.15.131:0/407455557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 msgr2=0x557612a45bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.490+0000 7fc1a7400640 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a45bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.490+0000 7fc1a7400640 1 -- 172.21.15.131:0/407455557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 msgr2=0x557612a430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.490+0000 7fc1a7400640 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.490+0000 7fc1a7400640 1 -- 172.21.15.131:0/407455557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557612960940 con 0x557612a3dbd0 2024-04-26T20:24:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.490+0000 7fc1a7400640 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3dff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0011440 tx=0x7fc1a0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4725b6b2e05d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.490+0000 7fc19f400640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc18c0a3d70 0x7fc18c0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc19f400640 1 -- 172.21.15.131:0/578536488 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc1980d0c80 con 0x7fc1a8108fc0 2024-04-26T20:24:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1a6a00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc18c0a3d70 0x7fc18c0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc197e00640 1 -- 172.21.15.131:0/407455557 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0003c30 con 0x557612a3dbd0 2024-04-26T20:24:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc197e00640 1 -- 172.21.15.131:0/407455557 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0003dd0 con 0x557612a3dbd0 2024-04-26T20:24:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc197e00640 1 -- 172.21.15.131:0/407455557 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000fae0 con 0x557612a3dbd0 2024-04-26T20:24:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1b076d740 1 -- 172.21.15.131:0/407455557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 msgr2=0x557612a3dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1b076d740 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3dff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0011440 tx=0x7fc1a0011900 comp rx=0 tx=0).stop 2024-04-26T20:24:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1a6a00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc18c0a3d70 0x7fc18c0a6230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x557612a3e9c0 tx=0x7fc1980d4730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1b076d740 1 -- 172.21.15.131:0/407455557 shutdown_connections 2024-04-26T20:24:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1b076d740 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a45bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1b076d740 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1b076d740 1 --2- 172.21.15.131:0/407455557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3dff0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.491+0000 7fc1b076d740 1 -- 172.21.15.131:0/407455557 >> 172.21.15.131:0/407455557 conn(0x557612a39240 msgr2=0x557612a39640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.492+0000 7fc1b076d740 1 -- 172.21.15.131:0/407455557 shutdown_connections 2024-04-26T20:24:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.492+0000 7fc1b076d740 1 -- 172.21.15.131:0/407455557 wait complete. 2024-04-26T20:24:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.492+0000 7fc1b076d740 1 -- start start 2024-04-26T20:24:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.492+0000 7fc1b076d740 1 Processor -- start 2024-04-26T20:24:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3dbd0 0x557612a47de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a48320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a43730 0x557612a4f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1b076d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557612a466e0 con 0x557612a3e5c0 2024-04-26T20:24:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1b076d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557612929c60 con 0x557612a3dbd0 2024-04-26T20:24:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1b076d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557612a46860 con 0x557612a43730 2024-04-26T20:24:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1b076d740 10 client.0 _ll_register_callbacks cb 0x557612955e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a48320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a48320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43532/0 (socket says 172.21.15.131:43532) 2024-04-26T20:24:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 -- 172.21.15.131:0/1297665404 learned_addr learned my addr 172.21.15.131:0/1297665404 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1a7400640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3dbd0 0x557612a47de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1a7400640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a43730 0x557612a4f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 -- 172.21.15.131:0/1297665404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a43730 msgr2=0x557612a4f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a43730 0x557612a4f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 -- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3dbd0 msgr2=0x557612a47de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3dbd0 0x557612a47de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc19fe00640 1 -- 172.21.15.131:0/1297665404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557612938510 con 0x557612a3e5c0 2024-04-26T20:24:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.493+0000 7fc1a7400640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3dbd0 0x557612a47de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:24:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc19fe00640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a48320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc19002a9e0 tx=0x7fc190003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee437e5b692f1f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc194c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc194c00640 1 -- 172.21.15.131:0/1297665404 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190035070 con 0x557612a3e5c0 2024-04-26T20:24:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc194c00640 1 -- 172.21.15.131:0/1297665404 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc190003ba0 con 0x557612a3e5c0 2024-04-26T20:24:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a0003dd0 con 0x557612a3e5c0 2024-04-26T20:24:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc194c00640 1 -- 172.21.15.131:0/1297665404 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc19004f050 con 0x557612a3e5c0 2024-04-26T20:24:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc1b076d740 10 client.16203 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.494+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576126b4210 con 0x557612a3e5c0 2024-04-26T20:24:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc194c00640 1 -- 172.21.15.131:0/1297665404 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0003dd0 con 0x557612a3e5c0 2024-04-26T20:24:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc194c00640 1 -- 172.21.15.131:0/1297665404 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc1900464d0 con 0x557612a3e5c0 2024-04-26T20:24:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc1b076d740 10 client.16203 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc1b076d740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc1b076d740 20 client.16203 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc1b076d740 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612a538b0 0x557612a55d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc1b076d740 4 client.16203 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:55.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.495+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576129aff90 con 0x557612a538b0 2024-04-26T20:24:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.496+0000 7fc1a7400640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612a538b0 0x557612a55d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.496+0000 7fc1a7400640 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612a538b0 0x557612a55d00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19dd75d34d4fa5d2 server_cookie=24776aafa61d4def in_seq=0 out_seq=0 2024-04-26T20:24:55.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.496+0000 7fc194c00640 10 client.16203 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc194c00640 1 -- 172.21.15.131:0/1297665404 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576129aff90 con 0x557612a538b0 2024-04-26T20:24:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc194c00640 10 client.16203 handle_command_reply: tid=0 2024-04-26T20:24:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 client.16203 shutdown 2024-04-26T20:24:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 msgr2=0x557612a48320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a48320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc19002a9e0 tx=0x7fc190003820 comp rx=0 tx=0).stop 2024-04-26T20:24:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 shutdown_connections 2024-04-26T20:24:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612a538b0 0x557612a55d00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a43730 0x557612a4f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a48320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 --2- 172.21.15.131:0/1297665404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3dbd0 0x557612a47de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc194c00640 0 client.16203 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 >> 172.21.15.131:0/1297665404 conn(0x557612a39240 msgr2=0x557612a38440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 shutdown_connections 2024-04-26T20:24:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 1 -- 172.21.15.131:0/1297665404 wait complete. 2024-04-26T20:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.497+0000 7fc1b076d740 20 client.16203 trim_cache size 0 max 16384 2024-04-26T20:24:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.513+0000 7fc1b076d740 1 Processor -- start 2024-04-26T20:24:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1b076d740 1 -- start start 2024-04-26T20:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a4f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a5a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1b076d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b4470 con 0x557612a3e5c0 2024-04-26T20:24:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1b076d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b1d40 con 0x557612a43730 2024-04-26T20:24:55.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1b076d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81afb90 con 0x557612a3dbd0 2024-04-26T20:24:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a4f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44256/0 (socket says 172.21.15.131:44256) 2024-04-26T20:24:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a4f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43538/0 (socket says 172.21.15.131:43538) 2024-04-26T20:24:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1a7400640 1 -- 172.21.15.131:0/814131272 learned_addr learned my addr 172.21.15.131:0/814131272 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.514+0000 7fc1a7400640 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a5a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc19fe00640 1 -- 172.21.15.131:0/814131272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 msgr2=0x557612a3c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc19fe00640 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc19fe00640 1 -- 172.21.15.131:0/814131272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 msgr2=0x557612a5a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc19fe00640 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a5a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc19fe00640 1 -- 172.21.15.131:0/814131272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557612780620 con 0x557612a3e5c0 2024-04-26T20:24:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc19fe00640 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a4f320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc190046a90 tx=0x7fc19002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cadbcf7122754d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc197400640 1 -- 172.21.15.131:0/814131272 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190058070 con 0x557612a3e5c0 2024-04-26T20:24:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc197400640 1 -- 172.21.15.131:0/814131272 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc190046b10 con 0x557612a3e5c0 2024-04-26T20:24:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.515+0000 7fc197400640 1 -- 172.21.15.131:0/814131272 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190035040 con 0x557612a3e5c0 2024-04-26T20:24:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 -- 172.21.15.131:0/814131272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 msgr2=0x557612a4f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a4f320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc190046a90 tx=0x7fc19002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 -- 172.21.15.131:0/814131272 shutdown_connections 2024-04-26T20:24:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a43730 0x557612a5a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a3e5c0 0x557612a4f320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 --2- 172.21.15.131:0/814131272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a3c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 -- 172.21.15.131:0/814131272 >> 172.21.15.131:0/814131272 conn(0x557612a39240 msgr2=0x557612a3c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 -- 172.21.15.131:0/814131272 shutdown_connections 2024-04-26T20:24:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 -- 172.21.15.131:0/814131272 wait complete. 2024-04-26T20:24:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.516+0000 7fc1b076d740 1 -- start start 2024-04-26T20:24:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 1 Processor -- start 2024-04-26T20:24:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a4cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3e5c0 0x557612a4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a43730 0x557612a4e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b0200 con 0x557612a43730 2024-04-26T20:24:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b1670 con 0x557612a3e5c0 2024-04-26T20:24:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b08d0 con 0x557612a3dbd0 2024-04-26T20:24:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1b076d740 10 client.0 _ll_register_callbacks cb 0x7fc190009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a4cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3e5c0 0x557612a4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a4cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44264/0 (socket says 172.21.15.131:44264) 2024-04-26T20:24:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1a7400640 1 -- 172.21.15.131:0/3387444128 learned_addr learned my addr 172.21.15.131:0/3387444128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.517+0000 7fc1a7400640 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a43730 0x557612a4e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc19fe00640 1 -- 172.21.15.131:0/3387444128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 msgr2=0x557612a4cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc19fe00640 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a4cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc19fe00640 1 -- 172.21.15.131:0/3387444128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a43730 msgr2=0x557612a4e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc19fe00640 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a43730 0x557612a4e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc19fe00640 1 -- 172.21.15.131:0/3387444128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761295ccb0 con 0x557612a3e5c0 2024-04-26T20:24:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc1a7400640 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a4cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc19fe00640 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3e5c0 0x557612a4d120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x557612a43700 tx=0x7fc190005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c5c10d3ef52251 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc194c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:24:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.518+0000 7fc194c00640 1 -- 172.21.15.131:0/3387444128 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190058030 con 0x557612a3e5c0 2024-04-26T20:24:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc190046b10 con 0x557612a3e5c0 2024-04-26T20:24:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc194c00640 1 -- 172.21.15.131:0/3387444128 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc190042040 con 0x557612a3e5c0 2024-04-26T20:24:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc194c00640 1 -- 172.21.15.131:0/3387444128 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc190065050 con 0x557612a3e5c0 2024-04-26T20:24:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc1b076d740 10 client.26327 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557612ab9e80 con 0x557612a3e5c0 2024-04-26T20:24:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc194c00640 1 -- 172.21.15.131:0/3387444128 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc190046b10 con 0x557612a3e5c0 2024-04-26T20:24:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc194c00640 1 -- 172.21.15.131:0/3387444128 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc19003c440 con 0x557612a3e5c0 2024-04-26T20:24:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc1b076d740 10 client.26327 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc1b076d740 10 client.26327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.519+0000 7fc1b076d740 20 client.26327 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.520+0000 7fc1b076d740 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612aba740 0x557612abcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.520+0000 7fc1b076d740 4 client.26327 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:24:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.520+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576129a9e00 con 0x557612aba740 2024-04-26T20:24:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.520+0000 7fc1a7400640 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612aba740 0x557612abcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.521+0000 7fc1a7400640 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612aba740 0x557612abcb60 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46451ade2cbaacd9 server_cookie=b4a169944fb248e6 in_seq=0 out_seq=0 2024-04-26T20:24:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.521+0000 7fc194c00640 10 client.26327 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc194c00640 1 -- 172.21.15.131:0/3387444128 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576129a9e00 con 0x557612aba740 2024-04-26T20:24:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc194c00640 10 client.26327 handle_command_reply: tid=0 2024-04-26T20:24:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 client.26327 shutdown 2024-04-26T20:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3e5c0 msgr2=0x557612a4d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3e5c0 0x557612a4d120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x557612a43700 tx=0x7fc190005a20 comp rx=0 tx=0).stop 2024-04-26T20:24:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 shutdown_connections 2024-04-26T20:24:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557612aba740 0x557612abcb60 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557612a43730 0x557612a4e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557612a3e5c0 0x557612a4d120 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 --2- 172.21.15.131:0/3387444128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557612a3dbd0 0x557612a4cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc194c00640 0 client.26327 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 >> 172.21.15.131:0/3387444128 conn(0x557612a39240 msgr2=0x557612a3c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 shutdown_connections 2024-04-26T20:24:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 1 -- 172.21.15.131:0/3387444128 wait complete. 2024-04-26T20:24:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.525+0000 7fc1b076d740 20 client.26327 trim_cache size 0 max 16384 2024-04-26T20:24:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:24:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:24:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:24:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:24:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc18c0a3d70 msgr2=0x7fc18c0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc18c0a3d70 0x7fc18c0a6230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x557612a3e9c0 tx=0x7fc1980d4730 comp rx=0 tx=0).stop 2024-04-26T20:24:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 msgr2=0x7fc1a819f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 0x7fc1a819f170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9c0 tx=0x7fc19800edc0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 shutdown_connections 2024-04-26T20:24:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc18c0a3d70 0x7fc18c0a6230 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1a810ab70 0x7fc1a81a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1a810a1b0 0x7fc1a819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 --2- 172.21.15.131:0/578536488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a8108fc0 0x7fc1a819f170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 >> 172.21.15.131:0/578536488 conn(0x7fc1a8077f80 msgr2=0x7fc1a810d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 shutdown_connections 2024-04-26T20:24:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:20.526+0000 7fc1ada00640 1 -- 172.21.15.131:0/578536488 wait complete. 2024-04-26T20:24:57.441 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:21 smithi131 ceph-mon[27919]: pgmap v810: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:57.448 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:23 smithi131 ceph-mon[27919]: pgmap v811: 129 pgs: 129 active+clean; 692 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:24:57.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:25 smithi131 ceph-mon[27919]: pgmap v812: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:57.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:27 smithi131 ceph-mon[27919]: pgmap v813: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:57.502 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:29 smithi131 ceph-mon[27919]: pgmap v814: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:24:57.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:31 smithi131 ceph-mon[27919]: pgmap v815: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:33 smithi131 ceph-mon[27919]: pgmap v816: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:57.525 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:53 smithi157 ceph-mon[32703]: pgmap v856: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:57.532 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:55 smithi157 ceph-mon[32703]: pgmap v857: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:24:57.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:56 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:24:57.591 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:53 smithi148 ceph-mon[32281]: pgmap v856: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:24:57.655 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:55 smithi148 ceph-mon[32281]: pgmap v857: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:24:57.664 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:56 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:24:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 -- 172.21.15.131:0/1146211054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354103b80 msgr2=0x7fe354104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 --2- 172.21.15.131:0/1146211054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354103b80 0x7fe354104000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009860 tx=0x7fe33c02a340 comp rx=0 tx=0).stop 2024-04-26T20:24:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 -- 172.21.15.131:0/1146211054 shutdown_connections 2024-04-26T20:24:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 --2- 172.21.15.131:0/1146211054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354104540 0x7fe35410add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 --2- 172.21.15.131:0/1146211054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354103b80 0x7fe354104000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 --2- 172.21.15.131:0/1146211054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe354102990 0x7fe354102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 -- 172.21.15.131:0/1146211054 >> 172.21.15.131:0/1146211054 conn(0x7fe3540fdf20 msgr2=0x7fe354100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 -- 172.21.15.131:0/1146211054 shutdown_connections 2024-04-26T20:24:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.670+0000 7fe35a000640 1 -- 172.21.15.131:0/1146211054 wait complete. 2024-04-26T20:24:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 Processor -- start 2024-04-26T20:24:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 -- start start 2024-04-26T20:24:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354102990 0x7fe35419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354103b80 0x7fe35419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe354104540 0x7fe3541a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe35410e4a0 con 0x7fe354103b80 2024-04-26T20:24:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe35410e320 con 0x7fe354104540 2024-04-26T20:24:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.671+0000 7fe35a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe35410e620 con 0x7fe354102990 2024-04-26T20:24:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354103b80 0x7fe35419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354103b80 0x7fe35419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35858/0 (socket says 172.21.15.131:35858) 2024-04-26T20:24:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 -- 172.21.15.131:0/917187690 learned_addr learned my addr 172.21.15.131:0/917187690 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe353e00640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe354104540 0x7fe3541a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe353400640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354102990 0x7fe35419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 -- 172.21.15.131:0/917187690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354102990 msgr2=0x7fe35419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354102990 0x7fe35419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 -- 172.21.15.131:0/917187690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe354104540 msgr2=0x7fe3541a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe354104540 0x7fe3541a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.672+0000 7fe34be00640 1 -- 172.21.15.131:0/917187690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3541a2980 con 0x7fe354103b80 2024-04-26T20:24:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.673+0000 7fe34be00640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354103b80 0x7fe35419b2a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009980 tx=0x7fe33c032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.673+0000 7fe350c00640 1 -- 172.21.15.131:0/917187690 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c040020 con 0x7fe354103b80 2024-04-26T20:24:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.673+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3541a2c10 con 0x7fe354103b80 2024-04-26T20:24:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.673+0000 7fe350c00640 1 -- 172.21.15.131:0/917187690 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c030dd0 con 0x7fe354103b80 2024-04-26T20:24:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.673+0000 7fe350c00640 1 -- 172.21.15.131:0/917187690 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c037040 con 0x7fe354103b80 2024-04-26T20:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.673+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3541a3100 con 0x7fe354103b80 2024-04-26T20:24:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.676+0000 7fe35ce3f740 1 Processor -- start 2024-04-26T20:24:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.676+0000 7fe35ce3f740 1 -- start start 2024-04-26T20:24:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.676+0000 7fe350c00640 1 -- 172.21.15.131:0/917187690 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe33c043020 con 0x7fe354103b80 2024-04-26T20:24:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040eadff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040eb5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558040e12380 con 0x558040eb3730 2024-04-26T20:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558040daec30 con 0x558040eae5c0 2024-04-26T20:24:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558040e0e340 con 0x558040eadbd0 2024-04-26T20:24:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe34be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040eb5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040eadff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36112/0 (socket says 172.21.15.131:36112) 2024-04-26T20:24:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe34be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040eb5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35862/0 (socket says 172.21.15.131:35862) 2024-04-26T20:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 -- 172.21.15.131:0/106299261 learned_addr learned my addr 172.21.15.131:0/106299261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 -- 172.21.15.131:0/106299261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 msgr2=0x558040eadff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 --2- 172.21.15.131:0/106299261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040eadff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 -- 172.21.15.131:0/106299261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 msgr2=0x558040eb5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 --2- 172.21.15.131:0/106299261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040eb5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.677+0000 7fe353400640 1 -- 172.21.15.131:0/106299261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558040dd0940 con 0x558040eae5c0 2024-04-26T20:24:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.678+0000 7fe353400640 1 --2- 172.21.15.131:0/106299261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb30c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe344009c80 tx=0x7fe344011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84edaede8d32bbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.678+0000 7fe343e00640 1 -- 172.21.15.131:0/106299261 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34401e070 con 0x558040eae5c0 2024-04-26T20:24:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.678+0000 7fe343e00640 1 -- 172.21.15.131:0/106299261 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe344009e30 con 0x558040eae5c0 2024-04-26T20:24:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.678+0000 7fe343e00640 1 -- 172.21.15.131:0/106299261 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34400ea10 con 0x558040eae5c0 2024-04-26T20:24:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.678+0000 7fe350c00640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3380a3cf0 0x7fe3380a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.678+0000 7fe350c00640 1 -- 172.21.15.131:0/917187690 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe33c0f0590 con 0x7fe354103b80 2024-04-26T20:24:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.678+0000 7fe353e00640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3380a3cf0 0x7fe3380a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 -- 172.21.15.131:0/106299261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 msgr2=0x558040eb30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/106299261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb30c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe344009c80 tx=0x7fe344011810 comp rx=0 tx=0).stop 2024-04-26T20:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 -- 172.21.15.131:0/106299261 shutdown_connections 2024-04-26T20:24:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/106299261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040eb5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/106299261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb30c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/106299261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040eadff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 -- 172.21.15.131:0/106299261 >> 172.21.15.131:0/106299261 conn(0x558040ea9240 msgr2=0x558040ea9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 -- 172.21.15.131:0/106299261 shutdown_connections 2024-04-26T20:24:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 -- 172.21.15.131:0/106299261 wait complete. 2024-04-26T20:24:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe353e00640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3380a3cf0 0x7fe3380a61b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x558040eaee20 tx=0x7fe34c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.679+0000 7fe35ce3f740 1 -- start start 2024-04-26T20:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 1 Processor -- start 2024-04-26T20:24:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eadbd0 0x558040eb7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eb3730 0x558040ebf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558040eb66e0 con 0x558040eadbd0 2024-04-26T20:24:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558040d99c60 con 0x558040eae5c0 2024-04-26T20:24:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558040eb6860 con 0x558040eb3730 2024-04-26T20:24:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe35ce3f740 10 client.0 _ll_register_callbacks cb 0x558040dc5e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:24:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe353400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eadbd0 0x558040eb7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe353400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eadbd0 0x558040eb7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35868/0 (socket says 172.21.15.131:35868) 2024-04-26T20:24:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.680+0000 7fe353400640 1 -- 172.21.15.131:0/3867501473 learned_addr learned my addr 172.21.15.131:0/3867501473 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:24:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe34be00640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe353e00640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eb3730 0x558040ebf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe353400640 1 -- 172.21.15.131:0/3867501473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eb3730 msgr2=0x558040ebf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe353400640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eb3730 0x558040ebf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe353400640 1 -- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 msgr2=0x558040eb8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:24:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe353400640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe353400640 1 -- 172.21.15.131:0/3867501473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558040da8510 con 0x558040eadbd0 2024-04-26T20:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe34be00640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:24:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe353400640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eadbd0 0x558040eb7e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe344000c00 tx=0x7fe3440056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0431233c5ee46c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:24:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe340c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:24:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe340c00640 1 -- 172.21.15.131:0/3867501473 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344028020 con 0x558040eadbd0 2024-04-26T20:24:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe340c00640 1 -- 172.21.15.131:0/3867501473 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe34400e990 con 0x558040eadbd0 2024-04-26T20:24:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.681+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe344009e30 con 0x558040eadbd0 2024-04-26T20:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe340c00640 1 -- 172.21.15.131:0/3867501473 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe344033050 con 0x558040eadbd0 2024-04-26T20:24:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 10 client.16221 fetch_fsmap learned FSMap version 27 2024-04-26T20:24:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558040b24210 con 0x558040eadbd0 2024-04-26T20:24:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe340c00640 1 -- 172.21.15.131:0/3867501473 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe344009e30 con 0x558040eadbd0 2024-04-26T20:24:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe340c00640 1 -- 172.21.15.131:0/3867501473 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe344004cc0 con 0x558040eadbd0 2024-04-26T20:24:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 10 client.16221 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:24:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:24:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 20 client.16221 populate_metadata read hostname 'smithi131' 2024-04-26T20:24:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040ec3620 0x558040ec5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:24:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 4 client.16221 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.682+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558040e1ff90 con 0x558040ec3620 2024-04-26T20:24:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.683+0000 7fe34be00640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040ec3620 0x558040ec5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:24:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.683+0000 7fe34be00640 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040ec3620 0x558040ec5a10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e087aea656cd16 server_cookie=444ba27e7204bf2 in_seq=0 out_seq=0 2024-04-26T20:24:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.683+0000 7fe340c00640 10 client.16221 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe340c00640 1 -- 172.21.15.131:0/3867501473 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558040e1ff90 con 0x558040ec3620 2024-04-26T20:24:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe340c00640 10 client.16221 handle_command_reply: tid=0 2024-04-26T20:24:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 client.16221 shutdown 2024-04-26T20:24:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eadbd0 msgr2=0x558040eb7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:24:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eadbd0 0x558040eb7e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe344000c00 tx=0x7fe3440056f0 comp rx=0 tx=0).stop 2024-04-26T20:24:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 shutdown_connections 2024-04-26T20:24:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040ec3620 0x558040ec5a10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eb3730 0x558040ebf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eae5c0 0x558040eb8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe340c00640 0 client.16221 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:24:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3867501473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eadbd0 0x558040eb7e20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:24:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.684+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 >> 172.21.15.131:0/3867501473 conn(0x558040ea9240 msgr2=0x558040ea8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:24:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.685+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 shutdown_connections 2024-04-26T20:25:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.685+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3867501473 wait complete. 2024-04-26T20:25:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.685+0000 7fe35ce3f740 20 client.16221 trim_cache size 0 max 16384 2024-04-26T20:25:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 Processor -- start 2024-04-26T20:25:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 -- start start 2024-04-26T20:25:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040ea82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eae5c0 0x558040ea8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eb3730 0x558040ebf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541b0140 con 0x558040eae5c0 2024-04-26T20:25:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541ada10 con 0x558040eb3730 2024-04-26T20:25:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.701+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541ab860 con 0x558040eadbd0 2024-04-26T20:25:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eae5c0 0x558040ea8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe353e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eb3730 0x558040ebf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe353400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040ea82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eae5c0 0x558040ea8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35882/0 (socket says 172.21.15.131:35882) 2024-04-26T20:25:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 -- 172.21.15.131:0/4152007958 learned_addr learned my addr 172.21.15.131:0/4152007958 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 -- 172.21.15.131:0/4152007958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 msgr2=0x558040ea82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040ea82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 -- 172.21.15.131:0/4152007958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eb3730 msgr2=0x558040ebf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eb3730 0x558040ebf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 -- 172.21.15.131:0/4152007958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558040bf0620 con 0x558040eae5c0 2024-04-26T20:25:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe353400640 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040ea82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe34be00640 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eae5c0 0x558040ea8800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0b8e00 tx=0x7fe33c0b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4de5f57534df04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.702+0000 7fe343400640 1 -- 172.21.15.131:0/4152007958 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0bbf70 con 0x558040eae5c0 2024-04-26T20:25:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe343400640 1 -- 172.21.15.131:0/4152007958 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c0f6380 con 0x558040eae5c0 2024-04-26T20:25:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe343400640 1 -- 172.21.15.131:0/4152007958 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0bbd90 con 0x558040eae5c0 2024-04-26T20:25:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 -- 172.21.15.131:0/4152007958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eae5c0 msgr2=0x558040ea8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eae5c0 0x558040ea8800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0b8e00 tx=0x7fe33c0b8040 comp rx=0 tx=0).stop 2024-04-26T20:25:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 -- 172.21.15.131:0/4152007958 shutdown_connections 2024-04-26T20:25:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eb3730 0x558040ebf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eae5c0 0x558040ea8800 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/4152007958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eadbd0 0x558040ea82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 -- 172.21.15.131:0/4152007958 >> 172.21.15.131:0/4152007958 conn(0x558040ea9240 msgr2=0x558040ea7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 -- 172.21.15.131:0/4152007958 shutdown_connections 2024-04-26T20:25:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.703+0000 7fe35ce3f740 1 -- 172.21.15.131:0/4152007958 wait complete. 2024-04-26T20:25:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 -- start start 2024-04-26T20:25:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 Processor -- start 2024-04-26T20:25:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eadbd0 0x558040ebf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eae5c0 0x558040ebbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040ebc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541abed0 con 0x558040eb3730 2024-04-26T20:25:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541ad340 con 0x558040eadbd0 2024-04-26T20:25:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541ac5a0 con 0x558040eae5c0 2024-04-26T20:25:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.704+0000 7fe35ce3f740 10 client.0 _ll_register_callbacks cb 0x7fe34401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe353e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040ebc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe353400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eadbd0 0x558040ebf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe34be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eae5c0 0x558040ebbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe353400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eadbd0 0x558040ebf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:36128/0 (socket says 172.21.15.131:36128) 2024-04-26T20:25:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe353e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040ebc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35896/0 (socket says 172.21.15.131:35896) 2024-04-26T20:25:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe353400640 1 -- 172.21.15.131:0/3423338088 learned_addr learned my addr 172.21.15.131:0/3423338088 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe34be00640 1 -- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eadbd0 msgr2=0x558040ebf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe34be00640 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eadbd0 0x558040ebf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe34be00640 1 -- 172.21.15.131:0/3423338088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 msgr2=0x558040ebc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe34be00640 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040ebc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe34be00640 1 -- 172.21.15.131:0/3423338088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558040dcccb0 con 0x558040eae5c0 2024-04-26T20:25:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe353400640 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eadbd0 0x558040ebf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe34be00640 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eae5c0 0x558040ebbca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0b8e00 tx=0x7fe33c030f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868b29a51a8bce3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.705+0000 7fe340c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:25:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe340c00640 1 -- 172.21.15.131:0/3423338088 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0bbf70 con 0x558040eae5c0 2024-04-26T20:25:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe340c00640 1 -- 172.21.15.131:0/3423338088 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe33c0edda0 con 0x558040eae5c0 2024-04-26T20:25:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33c0f6380 con 0x558040eae5c0 2024-04-26T20:25:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe340c00640 1 -- 172.21.15.131:0/3423338088 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe33c0bcb30 con 0x558040eae5c0 2024-04-26T20:25:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 10 client.26320 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558040f31470 con 0x558040eae5c0 2024-04-26T20:25:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe340c00640 1 -- 172.21.15.131:0/3423338088 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c0f6380 con 0x558040eae5c0 2024-04-26T20:25:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe340c00640 1 -- 172.21.15.131:0/3423338088 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe33c04d060 con 0x558040eae5c0 2024-04-26T20:25:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 10 client.26320 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 20 client.26320 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040f31d90 0x558040f341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 4 client.26320 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.706+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558040e19e00 con 0x558040f31d90 2024-04-26T20:25:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.707+0000 7fe353400640 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040f31d90 0x558040f341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.707+0000 7fe353400640 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040f31d90 0x558040f341b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd520881b413caf7 server_cookie=8fe74b9e74905678 in_seq=0 out_seq=0 2024-04-26T20:25:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.707+0000 7fe340c00640 10 client.26320 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe340c00640 1 -- 172.21.15.131:0/3423338088 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558040e19e00 con 0x558040f31d90 2024-04-26T20:25:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe340c00640 10 client.26320 handle_command_reply: tid=0 2024-04-26T20:25:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 client.26320 shutdown 2024-04-26T20:25:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eae5c0 msgr2=0x558040ebbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eae5c0 0x558040ebbca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0b8e00 tx=0x7fe33c030f80 comp rx=0 tx=0).stop 2024-04-26T20:25:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 shutdown_connections 2024-04-26T20:25:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558040f31d90 0x558040f341b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558040eb3730 0x558040ebc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558040eae5c0 0x558040ebbca0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe340c00640 0 client.26320 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 --2- 172.21.15.131:0/3423338088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558040eadbd0 0x558040ebf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 >> 172.21.15.131:0/3423338088 conn(0x558040ea9240 msgr2=0x558040ea8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 shutdown_connections 2024-04-26T20:25:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 1 -- 172.21.15.131:0/3423338088 wait complete. 2024-04-26T20:25:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.711+0000 7fe35ce3f740 20 client.26320 trim_cache size 0 max 16384 2024-04-26T20:25:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3380a3cf0 msgr2=0x7fe3380a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3380a3cf0 0x7fe3380a61b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x558040eaee20 tx=0x7fe34c010040 comp rx=0 tx=0).stop 2024-04-26T20:25:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354103b80 msgr2=0x7fe35419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354103b80 0x7fe35419b2a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009980 tx=0x7fe33c032dc0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 shutdown_connections 2024-04-26T20:25:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe354104540 0x7fe3541a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe354103b80 0x7fe35419b2a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3380a3cf0 0x7fe3380a61b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 --2- 172.21.15.131:0/917187690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe354102990 0x7fe35419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 >> 172.21.15.131:0/917187690 conn(0x7fe3540fdf20 msgr2=0x7fe354074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 shutdown_connections 2024-04-26T20:25:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:33.712+0000 7fe35a000640 1 -- 172.21.15.131:0/917187690 wait complete. 2024-04-26T20:25:02.032 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:35 smithi131 ceph-mon[27919]: pgmap v817: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:02.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:37 smithi131 ceph-mon[27919]: pgmap v818: 129 pgs: 129 active+clean; 693 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:02.088 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:39 smithi131 ceph-mon[27919]: pgmap v819: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:02.095 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:41 smithi131 ceph-mon[27919]: pgmap v820: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:02.103 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:43 smithi131 ceph-mon[27919]: pgmap v821: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:02.110 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:45 smithi131 ceph-mon[27919]: pgmap v822: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:02.119 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:57 smithi157 ceph-mon[32703]: pgmap v858: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:02.176 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:25:02.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:25:02.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:25:02.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:24:59 smithi157 ceph-mon[32703]: pgmap v859: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:02.260 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:57 smithi148 ceph-mon[32281]: pgmap v858: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:02.267 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:25:02.274 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:25:02.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:25:02.344 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:24:59 smithi148 ceph-mon[32281]: pgmap v859: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:02.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:01 smithi148 ceph-mon[32281]: pgmap v860: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:02.360 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:01 smithi157 ceph-mon[32703]: pgmap v860: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.855+0000 7f5b35400640 1 -- 172.21.15.131:0/1497787757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 msgr2=0x7f5b3010a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.855+0000 7f5b35400640 1 --2- 172.21.15.131:0/1497787757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 0x7f5b3010a450 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009860 tx=0x7f5b1802a340 comp rx=0 tx=0).stop 2024-04-26T20:25:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.856+0000 7f5b35400640 1 -- 172.21.15.131:0/1497787757 shutdown_connections 2024-04-26T20:25:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.856+0000 7f5b35400640 1 --2- 172.21.15.131:0/1497787757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3010a990 0x7f5b30111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.856+0000 7f5b35400640 1 --2- 172.21.15.131:0/1497787757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 0x7f5b3010a450 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.856+0000 7f5b35400640 1 --2- 172.21.15.131:0/1497787757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 0x7f5b30109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.856+0000 7f5b35400640 1 -- 172.21.15.131:0/1497787757 >> 172.21.15.131:0/1497787757 conn(0x7f5b30077dc0 msgr2=0x7f5b3007a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.856+0000 7f5b35400640 1 -- 172.21.15.131:0/1497787757 shutdown_connections 2024-04-26T20:25:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.856+0000 7f5b35400640 1 -- 172.21.15.131:0/1497787757 wait complete. 2024-04-26T20:25:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.857+0000 7f5b35400640 1 Processor -- start 2024-04-26T20:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.857+0000 7f5b35400640 1 -- start start 2024-04-26T20:25:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b35400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 0x7f5b3019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b35400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 0x7f5b3019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b35400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3010a990 0x7f5b301a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b35400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30114910 con 0x7f5b30108de0 2024-04-26T20:25:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b35400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30114790 con 0x7f5b30109fd0 2024-04-26T20:25:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b35400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30114a90 con 0x7f5b3010a990 2024-04-26T20:25:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 0x7f5b3019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b34a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3010a990 0x7f5b301a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 0x7f5b3019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44162/0 (socket says 172.21.15.131:44162) 2024-04-26T20:25:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b2fe00640 1 -- 172.21.15.131:0/2521423265 learned_addr learned my addr 172.21.15.131:0/2521423265 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b27e00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 0x7f5b3019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b34a00640 1 -- 172.21.15.131:0/2521423265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 msgr2=0x7f5b3019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b34a00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 0x7f5b3019f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b34a00640 1 -- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 msgr2=0x7f5b3019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b34a00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 0x7f5b3019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b34a00640 1 -- 172.21.15.131:0/2521423265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b301a6c00 con 0x7f5b3010a990 2024-04-26T20:25:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b27e00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 0x7f5b3019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.858+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 0x7f5b3019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.859+0000 7f5b34a00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3010a990 0x7f5b301a6590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009fd0 tx=0x7f5b2800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.859+0000 7f5b2d600640 1 -- 172.21.15.131:0/2521423265 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800ce20 con 0x7f5b3010a990 2024-04-26T20:25:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.859+0000 7f5b2d600640 1 -- 172.21.15.131:0/2521423265 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b28005020 con 0x7f5b3010a990 2024-04-26T20:25:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.859+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b301a6ef0 con 0x7f5b3010a990 2024-04-26T20:25:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.859+0000 7f5b2d600640 1 -- 172.21.15.131:0/2521423265 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28010690 con 0x7f5b3010a990 2024-04-26T20:25:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.859+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301a73b0 con 0x7f5b3010a990 2024-04-26T20:25:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.863+0000 7f5b3983c740 1 Processor -- start 2024-04-26T20:25:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.863+0000 7f5b2d600640 1 -- 172.21.15.131:0/2521423265 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b280054d0 con 0x7f5b3010a990 2024-04-26T20:25:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.863+0000 7f5b3983c740 1 -- start start 2024-04-26T20:25:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0f2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd0f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd0f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b3983c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562abd056380 con 0x562abd0f7940 2024-04-26T20:25:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b3983c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562abcff2c30 con 0x562abd0f27d0 2024-04-26T20:25:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b3983c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562abd052340 con 0x562abd0f1de0 2024-04-26T20:25:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd0f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd0f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:47212/0 (socket says 172.21.15.131:47212) 2024-04-26T20:25:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b2fe00640 1 -- 172.21.15.131:0/2957563372 learned_addr learned my addr 172.21.15.131:0/2957563372 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.864+0000 7f5b27e00640 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd0f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2fe00640 1 -- 172.21.15.131:0/2957563372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 msgr2=0x562abd0f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2fe00640 1 -- 172.21.15.131:0/2957563372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 msgr2=0x562abd0f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd0f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2fe00640 1 -- 172.21.15.131:0/2957563372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abd014940 con 0x562abd0f27d0 2024-04-26T20:25:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2d600640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b100a3c30 0x7f5b100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd0f72d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b200115c0 tx=0x7f5b20011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d3750185ffa6243 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b2d600640 1 -- 172.21.15.131:0/2521423265 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5b280d1120 con 0x7f5b3010a990 2024-04-26T20:25:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b27e00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b100a3c30 0x7f5b100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.865+0000 7f5b25200640 1 -- 172.21.15.131:0/2957563372 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20004020 con 0x562abd0f27d0 2024-04-26T20:25:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.866+0000 7f5b25200640 1 -- 172.21.15.131:0/2957563372 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200041c0 con 0x562abd0f27d0 2024-04-26T20:25:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.866+0000 7f5b25200640 1 -- 172.21.15.131:0/2957563372 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20003960 con 0x562abd0f27d0 2024-04-26T20:25:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.866+0000 7f5b3983c740 1 -- 172.21.15.131:0/2957563372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 msgr2=0x562abd0f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.866+0000 7f5b27e00640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b100a3c30 0x7f5b100a60f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18030bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.866+0000 7f5b3983c740 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd0f72d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b200115c0 tx=0x7f5b20011a80 comp rx=0 tx=0).stop 2024-04-26T20:25:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.866+0000 7f5b3983c740 1 -- 172.21.15.131:0/2957563372 shutdown_connections 2024-04-26T20:25:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.866+0000 7f5b3983c740 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd0f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.867+0000 7f5b3983c740 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd0f72d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.867+0000 7f5b3983c740 1 --2- 172.21.15.131:0/2957563372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.867+0000 7f5b3983c740 1 -- 172.21.15.131:0/2957563372 >> 172.21.15.131:0/2957563372 conn(0x562abd0ed450 msgr2=0x562abd0ed850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.867+0000 7f5b3983c740 1 -- 172.21.15.131:0/2957563372 shutdown_connections 2024-04-26T20:25:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.867+0000 7f5b3983c740 1 -- 172.21.15.131:0/2957563372 wait complete. 2024-04-26T20:25:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.867+0000 7f5b3983c740 1 -- start start 2024-04-26T20:25:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.867+0000 7f5b3983c740 1 Processor -- start 2024-04-26T20:25:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f27d0 0x562abd0fc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f7940 0x562abd103540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b3983c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562abd0fa8f0 con 0x562abd0f27d0 2024-04-26T20:25:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b3983c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562abcfddc60 con 0x562abd0f7940 2024-04-26T20:25:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b3983c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562abd0faa70 con 0x562abd0f1de0 2024-04-26T20:25:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b3983c740 10 client.0 _ll_register_callbacks cb 0x562abd009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f27d0 0x562abd0fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f27d0 0x562abd0fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44176/0 (socket says 172.21.15.131:44176) 2024-04-26T20:25:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b34a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f7940 0x562abd103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b27e00640 1 -- 172.21.15.131:0/597449464 learned_addr learned my addr 172.21.15.131:0/597449464 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b34a00640 1 -- 172.21.15.131:0/597449464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 msgr2=0x562abd0fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b34a00640 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b34a00640 1 -- 172.21.15.131:0/597449464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f27d0 msgr2=0x562abd0fc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b34a00640 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f27d0 0x562abd0fc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.868+0000 7f5b34a00640 1 -- 172.21.15.131:0/597449464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abcfec510 con 0x562abd0f7940 2024-04-26T20:25:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b34a00640 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f7940 0x562abd103540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b280ce680 tx=0x7f5b2809cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64528db93e8ea1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b1d600640 1 -- 172.21.15.131:0/597449464 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b280b5020 con 0x562abd0f7940 2024-04-26T20:25:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b200041c0 con 0x562abd0f7940 2024-04-26T20:25:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b1d600640 1 -- 172.21.15.131:0/597449464 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5b280051d0 con 0x562abd0f7940 2024-04-26T20:25:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b1d600640 1 -- 172.21.15.131:0/597449464 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b280b6220 con 0x562abd0f7940 2024-04-26T20:25:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b3983c740 10 client.26357 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.869+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562abcd68210 con 0x562abd0f7940 2024-04-26T20:25:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b1d600640 1 -- 172.21.15.131:0/597449464 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200041c0 con 0x562abd0f7940 2024-04-26T20:25:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b1d600640 1 -- 172.21.15.131:0/597449464 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5b28098d30 con 0x562abd0f7940 2024-04-26T20:25:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b3983c740 10 client.26357 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b3983c740 10 client.26357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b3983c740 20 client.26357 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b3983c740 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd107b00 0x562abd109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b3983c740 4 client.26357 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562abd063f90 con 0x562abd107b00 2024-04-26T20:25:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.870+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd107b00 0x562abd109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.871+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd107b00 0x562abd109ef0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2533f11a5564a8f9 server_cookie=1f69ec0c486902fd in_seq=0 out_seq=0 2024-04-26T20:25:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.871+0000 7f5b1d600640 10 client.26357 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b1d600640 1 -- 172.21.15.131:0/597449464 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562abd063f90 con 0x562abd107b00 2024-04-26T20:25:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b1d600640 10 client.26357 handle_command_reply: tid=0 2024-04-26T20:25:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 client.26357 shutdown 2024-04-26T20:25:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f7940 msgr2=0x562abd103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f7940 0x562abd103540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b280ce680 tx=0x7f5b2809cf90 comp rx=0 tx=0).stop 2024-04-26T20:25:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 shutdown_connections 2024-04-26T20:25:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd107b00 0x562abd109ef0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f7940 0x562abd103540 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f27d0 0x562abd0fc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 --2- 172.21.15.131:0/597449464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd0fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b1d600640 0 client.26357 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 >> 172.21.15.131:0/597449464 conn(0x562abd0ed450 msgr2=0x562abd0ec650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 shutdown_connections 2024-04-26T20:25:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 1 -- 172.21.15.131:0/597449464 wait complete. 2024-04-26T20:25:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.872+0000 7f5b3983c740 20 client.26357 trim_cache size 0 max 16384 2024-04-26T20:25:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.889+0000 7f5b3983c740 1 Processor -- start 2024-04-26T20:25:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.889+0000 7f5b3983c740 1 -- start start 2024-04-26T20:25:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd10f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd10fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd103500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b3983c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b4360 con 0x562abd0f7940 2024-04-26T20:25:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b3983c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b1c30 con 0x562abd0f27d0 2024-04-26T20:25:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b3983c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301afa80 con 0x562abd0f1de0 2024-04-26T20:25:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd10f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd10f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33612/0 (socket says 172.21.15.131:33612) 2024-04-26T20:25:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 -- 172.21.15.131:0/348956216 learned_addr learned my addr 172.21.15.131:0/348956216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b34a00640 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b27e00640 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd10fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 -- 172.21.15.131:0/348956216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 msgr2=0x562abd10fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd10fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 -- 172.21.15.131:0/348956216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 msgr2=0x562abd103500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd103500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.890+0000 7f5b2fe00640 1 -- 172.21.15.131:0/348956216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abd10a260 con 0x562abd0f1de0 2024-04-26T20:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.891+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd10f800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b20010010 tx=0x7f5b20009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeacb5d45efc13e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.891+0000 7f5b1fe00640 1 -- 172.21.15.131:0/348956216 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2002d020 con 0x562abd0f1de0 2024-04-26T20:25:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.891+0000 7f5b1fe00640 1 -- 172.21.15.131:0/348956216 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200055f0 con 0x562abd0f1de0 2024-04-26T20:25:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.891+0000 7f5b1fe00640 1 -- 172.21.15.131:0/348956216 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20029e60 con 0x562abd0f1de0 2024-04-26T20:25:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 -- 172.21.15.131:0/348956216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 msgr2=0x562abd10f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd10f800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b20010010 tx=0x7f5b20009d10 comp rx=0 tx=0).stop 2024-04-26T20:25:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 -- 172.21.15.131:0/348956216 shutdown_connections 2024-04-26T20:25:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f7940 0x562abd103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd10fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 --2- 172.21.15.131:0/348956216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f1de0 0x562abd10f800 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 -- 172.21.15.131:0/348956216 >> 172.21.15.131:0/348956216 conn(0x562abd0ed450 msgr2=0x562abd0f3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 -- 172.21.15.131:0/348956216 shutdown_connections 2024-04-26T20:25:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 -- 172.21.15.131:0/348956216 wait complete. 2024-04-26T20:25:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.892+0000 7f5b3983c740 1 -- start start 2024-04-26T20:25:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 1 Processor -- start 2024-04-26T20:25:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f1de0 0x562abd1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd100630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f7940 0x562abd16c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b00f0 con 0x562abd0f1de0 2024-04-26T20:25:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b1560 con 0x562abd0f27d0 2024-04-26T20:25:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b07c0 con 0x562abd0f7940 2024-04-26T20:25:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b3983c740 10 client.0 _ll_register_callbacks cb 0x7f5b28099170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f1de0 0x562abd1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f1de0 0x562abd1000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44194/0 (socket says 172.21.15.131:44194) 2024-04-26T20:25:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b2fe00640 1 -- 172.21.15.131:0/3101320939 learned_addr learned my addr 172.21.15.131:0/3101320939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.893+0000 7f5b34a00640 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f7940 0x562abd16c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b2fe00640 1 -- 172.21.15.131:0/3101320939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f7940 msgr2=0x562abd16c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f7940 0x562abd16c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b2fe00640 1 -- 172.21.15.131:0/3101320939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 msgr2=0x562abd100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b2fe00640 1 -- 172.21.15.131:0/3101320939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abce34620 con 0x562abd0f1de0 2024-04-26T20:25:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b2fe00640 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f1de0 0x562abd1000f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b200057c0 tx=0x7f5b20009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9928461ee1888a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b1d600640 1 -- 172.21.15.131:0/3101320939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20029e60 con 0x562abd0f1de0 2024-04-26T20:25:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b1d600640 1 -- 172.21.15.131:0/3101320939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f5b20005140 con 0x562abd0f1de0 2024-04-26T20:25:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.894+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b200055f0 con 0x562abd0f1de0 2024-04-26T20:25:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b1d600640 1 -- 172.21.15.131:0/3101320939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b20045050 con 0x562abd0f1de0 2024-04-26T20:25:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 10 client.16245 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562abd010cb0 con 0x562abd0f1de0 2024-04-26T20:25:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b1d600640 1 -- 172.21.15.131:0/3101320939 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200055f0 con 0x562abd0f1de0 2024-04-26T20:25:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b1d600640 1 -- 172.21.15.131:0/3101320939 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5b2000fad0 con 0x562abd0f1de0 2024-04-26T20:25:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 10 client.16245 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 20 client.16245 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd16d9b0 0x562abd16fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 4 client.16245 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.895+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562abd05de00 con 0x562abd16d9b0 2024-04-26T20:25:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.896+0000 7f5b27e00640 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd16d9b0 0x562abd16fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.896+0000 7f5b27e00640 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd16d9b0 0x562abd16fdd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3282b27f7e9174 server_cookie=1ec0a295f6e2228e in_seq=0 out_seq=0 2024-04-26T20:25:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.896+0000 7f5b1d600640 10 client.16245 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b1d600640 1 -- 172.21.15.131:0/3101320939 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562abd05de00 con 0x562abd16d9b0 2024-04-26T20:25:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b1d600640 10 client.16245 handle_command_reply: tid=0 2024-04-26T20:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 client.16245 shutdown 2024-04-26T20:25:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f1de0 msgr2=0x562abd1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f1de0 0x562abd1000f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b200057c0 tx=0x7f5b20009fd0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 shutdown_connections 2024-04-26T20:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562abd16d9b0 0x562abd16fdd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562abd0f7940 0x562abd16c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562abd0f27d0 0x562abd100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 --2- 172.21.15.131:0/3101320939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562abd0f1de0 0x562abd1000f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 >> 172.21.15.131:0/3101320939 conn(0x562abd0ed450 msgr2=0x562abd0f3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b1d600640 0 client.16245 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 shutdown_connections 2024-04-26T20:25:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 1 -- 172.21.15.131:0/3101320939 wait complete. 2024-04-26T20:25:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.900+0000 7f5b3983c740 20 client.16245 trim_cache size 0 max 16384 2024-04-26T20:25:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b100a3c30 msgr2=0x7f5b100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b100a3c30 0x7f5b100a60f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18030bd0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3010a990 msgr2=0x7f5b301a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3010a990 0x7f5b301a6590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009fd0 tx=0x7f5b2800ef10 comp rx=0 tx=0).stop 2024-04-26T20:25:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 shutdown_connections 2024-04-26T20:25:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f5b100a3c30 0x7f5b100a60f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3010a990 0x7f5b301a6590 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b30109fd0 0x7f5b3019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 --2- 172.21.15.131:0/2521423265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b30108de0 0x7f5b3019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 >> 172.21.15.131:0/2521423265 conn(0x7f5b30077dc0 msgr2=0x7f5b3010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 shutdown_connections 2024-04-26T20:25:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:23:46.901+0000 7f5b35400640 1 -- 172.21.15.131:0/2521423265 wait complete. 2024-04-26T20:25:06.939 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:47 smithi131 ceph-mon[27919]: pgmap v823: 129 pgs: 129 active+clean; 694 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:06.949 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:49 smithi131 ceph-mon[27919]: pgmap v824: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:25:06.958 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:51 smithi131 ceph-mon[27919]: pgmap v825: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:07.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:53 smithi131 ceph-mon[27919]: pgmap v826: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:07.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:55 smithi131 ceph-mon[27919]: pgmap v827: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:07.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:55 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:25:07.087 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:25:07.096 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:25:07.103 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:25:07.113 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:03 smithi157 ceph-mon[32703]: pgmap v861: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:07.166 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:05 smithi157 ceph-mon[32703]: pgmap v862: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:07.176 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:03 smithi148 ceph-mon[32281]: pgmap v861: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:07.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:05 smithi148 ceph-mon[32281]: pgmap v862: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:07.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:57 smithi131 ceph-mon[27919]: pgmap v828: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:07.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 -- 172.21.15.131:0/140149380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f41010c0 msgr2=0x7fe2f41034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 --2- 172.21.15.131:0/140149380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f41010c0 0x7fe2f41034f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009860 tx=0x7fe2e802a340 comp rx=0 tx=0).stop 2024-04-26T20:25:07.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 -- 172.21.15.131:0/140149380 shutdown_connections 2024-04-26T20:25:07.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 --2- 172.21.15.131:0/140149380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f41063e0 0x7fe2f4106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 --2- 172.21.15.131:0/140149380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f4103a30 0x7fe2f4105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 --2- 172.21.15.131:0/140149380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f41010c0 0x7fe2f41034f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:07.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 -- 172.21.15.131:0/140149380 >> 172.21.15.131:0/140149380 conn(0x7fe2f40fcce0 msgr2=0x7fe2f40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:07.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 -- 172.21.15.131:0/140149380 shutdown_connections 2024-04-26T20:25:07.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.813+0000 7fe2fa400640 1 -- 172.21.15.131:0/140149380 wait complete. 2024-04-26T20:25:07.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.814+0000 7fe2fa400640 1 Processor -- start 2024-04-26T20:25:07.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2fa400640 1 -- start start 2024-04-26T20:25:07.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2fa400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f41010c0 0x7fe2f419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:07.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2fa400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f4103a30 0x7fe2f419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:07.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2fa400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f41063e0 0x7fe2f41a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:07.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2fa400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4114970 con 0x7fe2f41063e0 2024-04-26T20:25:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2fa400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41147f0 con 0x7fe2f4103a30 2024-04-26T20:25:07.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2fa400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4114af0 con 0x7fe2f41010c0 2024-04-26T20:25:07.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2f9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f41063e0 0x7fe2f41a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2f9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f41063e0 0x7fe2f41a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50952/0 (socket says 172.21.15.131:50952) 2024-04-26T20:25:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2f9a00640 1 -- 172.21.15.131:0/3452004716 learned_addr learned my addr 172.21.15.131:0/3452004716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f4103a30 0x7fe2f419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:07.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.815+0000 7fe2f9000640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f41010c0 0x7fe2f419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:07.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f9a00640 1 -- 172.21.15.131:0/3452004716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f41010c0 msgr2=0x7fe2f419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:07.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f9a00640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f41010c0 0x7fe2f419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:07.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f9a00640 1 -- 172.21.15.131:0/3452004716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f4103a30 msgr2=0x7fe2f419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:07.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f9a00640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f4103a30 0x7fe2f419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:07.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f9a00640 1 -- 172.21.15.131:0/3452004716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f41a6c60 con 0x7fe2f41063e0 2024-04-26T20:25:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f9000640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f41010c0 0x7fe2f419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f4103a30 0x7fe2f419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:07.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f9a00640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f41063e0 0x7fe2f41a65f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00eb40 tx=0x7fe2ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.816+0000 7fe2f1600640 1 -- 172.21.15.131:0/3452004716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec010430 con 0x7fe2f41063e0 2024-04-26T20:25:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.817+0000 7fe2f1600640 1 -- 172.21.15.131:0/3452004716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2ec010a80 con 0x7fe2f41063e0 2024-04-26T20:25:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.817+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f41a6ef0 con 0x7fe2f41063e0 2024-04-26T20:25:07.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.817+0000 7fe2f1600640 1 -- 172.21.15.131:0/3452004716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec018b60 con 0x7fe2f41063e0 2024-04-26T20:25:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.817+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f41a7400 con 0x7fe2f41063e0 2024-04-26T20:25:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.821+0000 7fe2fe9d6740 1 Processor -- start 2024-04-26T20:25:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.821+0000 7fe2f1600640 1 -- 172.21.15.131:0/3452004716 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe2ec018d00 con 0x7fe2f41063e0 2024-04-26T20:25:07.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.821+0000 7fe2fe9d6740 1 -- start start 2024-04-26T20:25:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c46651470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46656540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:07.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46659040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:07.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564c465b4c70 con 0x564c46651050 2024-04-26T20:25:07.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564c4656d980 con 0x564c46651a40 2024-04-26T20:25:07.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564c465b0c30 con 0x564c46656bb0 2024-04-26T20:25:08.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c46651470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:08.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46656540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:08.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46656540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42452/0 (socket says 172.21.15.131:42452) 2024-04-26T20:25:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f9000640 1 -- 172.21.15.131:0/475124270 learned_addr learned my addr 172.21.15.131:0/475124270 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:08.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c46651470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50966/0 (socket says 172.21.15.131:50966) 2024-04-26T20:25:08.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46659040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f9000640 1 -- 172.21.15.131:0/475124270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 msgr2=0x564c46659040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f9000640 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46659040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f9000640 1 -- 172.21.15.131:0/475124270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 msgr2=0x564c46651470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:08.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.822+0000 7fe2f9000640 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c46651470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2f9000640 1 -- 172.21.15.131:0/475124270 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c465457b0 con 0x564c46651a40 2024-04-26T20:25:08.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46659040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2f1600640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2dc0a3cc0 0x7fe2dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:08.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2f9000640 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46656540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009860 tx=0x7fe2e8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5968df26828296f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:08.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2f1600640 1 -- 172.21.15.131:0/3452004716 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe2ec0d0ee0 con 0x7fe2f41063e0 2024-04-26T20:25:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2e2000640 1 -- 172.21.15.131:0/475124270 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8037040 con 0x564c46651a40 2024-04-26T20:25:08.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2e2000640 1 -- 172.21.15.131:0/475124270 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e8030dd0 con 0x564c46651a40 2024-04-26T20:25:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2e2000640 1 -- 172.21.15.131:0/475124270 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8032c10 con 0x564c46651a40 2024-04-26T20:25:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.823+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2dc0a3cc0 0x7fe2dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:08.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/475124270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 msgr2=0x564c46656540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:08.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46656540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009860 tx=0x7fe2e8005dc0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2dc0a3cc0 0x7fe2dc0a6180 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x564c466522a0 tx=0x7fe2e40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:08.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/475124270 shutdown_connections 2024-04-26T20:25:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46659040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46656540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/475124270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c46651470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/475124270 >> 172.21.15.131:0/475124270 conn(0x564c4664c6c0 msgr2=0x564c4664cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:08.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/475124270 shutdown_connections 2024-04-26T20:25:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.824+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/475124270 wait complete. 2024-04-26T20:25:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 -- start start 2024-04-26T20:25:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 Processor -- start 2024-04-26T20:25:08.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:08.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c4665b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:08.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46662450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:08.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564c465b3f80 con 0x564c46651050 2024-04-26T20:25:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564c46561260 con 0x564c46651a40 2024-04-26T20:25:08.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564c465b92a0 con 0x564c46656bb0 2024-04-26T20:25:08.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.825+0000 7fe2fe9d6740 10 client.0 _ll_register_callbacks cb 0x564c465ab960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:08.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:08.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50972/0 (socket says 172.21.15.131:50972) 2024-04-26T20:25:08.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 -- 172.21.15.131:0/2777041620 learned_addr learned my addr 172.21.15.131:0/2777041620 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:08.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c4665b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:08.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9a00640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46662450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:08.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 -- 172.21.15.131:0/2777041620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 msgr2=0x564c46662450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:08.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46662450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 -- 172.21.15.131:0/2777041620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 msgr2=0x564c4665b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:08.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c4665b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:08.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 -- 172.21.15.131:0/2777041620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c465431c0 con 0x564c46651050 2024-04-26T20:25:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9a00640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46662450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:09.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2f9000640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665aed0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe2e802a930 tx=0x7fe2e80309d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247d8f30cc91de3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:09.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.826+0000 7fe2d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:09.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2777041620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8037030 con 0x564c46651050 2024-04-26T20:25:09.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2777041620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe2e8032ba0 con 0x564c46651050 2024-04-26T20:25:09.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e8030dd0 con 0x564c46651050 2024-04-26T20:25:09.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2777041620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2e8058050 con 0x564c46651050 2024-04-26T20:25:09.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2fe9d6740 10 client.16257 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:09.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c462c63f0 con 0x564c46651050 2024-04-26T20:25:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2777041620 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e8030dd0 con 0x564c46651050 2024-04-26T20:25:09.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.827+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2777041620 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe2e8046460 con 0x564c46651050 2024-04-26T20:25:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2fe9d6740 10 client.16257 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2fe9d6740 10 client.16257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-26T20:25:09.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2fe9d6740 20 client.16257 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:09.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466665b0 0x564c466689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:09.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2fe9d6740 4 client.16257 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c465c2880 con 0x564c466665b0 2024-04-26T20:25:09.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466665b0 0x564c466689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:09.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.828+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466665b0 0x564c466689c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f23d6cdf7483533 server_cookie=ead3d38a467e86e6 in_seq=0 out_seq=0 2024-04-26T20:25:09.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.829+0000 7fe2d6a00640 10 client.16257 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:25:09.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.829+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2777041620 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c465c2880 con 0x564c466665b0 2024-04-26T20:25:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.829+0000 7fe2d6a00640 10 client.16257 handle_command_reply: tid=0 2024-04-26T20:25:09.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.829+0000 7fe2fe9d6740 1 client.16257 shutdown 2024-04-26T20:25:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.829+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 msgr2=0x564c4665aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:09.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.829+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665aed0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe2e802a930 tx=0x7fe2e80309d0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 shutdown_connections 2024-04-26T20:25:09.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466665b0 0x564c466689c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c46662450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c4665b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2d6a00640 0 client.16257 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:25:09.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2777041620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665aed0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 >> 172.21.15.131:0/2777041620 conn(0x564c4664c6c0 msgr2=0x564c4664b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:09.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 shutdown_connections 2024-04-26T20:25:09.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2777041620 wait complete. 2024-04-26T20:25:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.830+0000 7fe2fe9d6740 20 client.16257 trim_cache size 0 max 16384 2024-04-26T20:25:09.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.844+0000 7fe2fe9d6740 1 Processor -- start 2024-04-26T20:25:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2fe9d6740 1 -- start start 2024-04-26T20:25:09.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46662370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:09.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c4665aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:09.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41b42b0 con 0x564c46651050 2024-04-26T20:25:09.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41b1b80 con 0x564c46651a40 2024-04-26T20:25:09.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41af9d0 con 0x564c46656bb0 2024-04-26T20:25:09.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50980/0 (socket says 172.21.15.131:50980) 2024-04-26T20:25:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9000640 1 -- 172.21.15.131:0/3688214013 learned_addr learned my addr 172.21.15.131:0/3688214013 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:09.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46662370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:09.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9a00640 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c4665aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:09.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9000640 1 -- 172.21.15.131:0/3688214013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 msgr2=0x564c4665aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:09.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9000640 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c4665aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9000640 1 -- 172.21.15.131:0/3688214013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 msgr2=0x564c46662370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.845+0000 7fe2f9000640 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46662370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.846+0000 7fe2f9000640 1 -- 172.21.15.131:0/3688214013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c463907d0 con 0x564c46651050 2024-04-26T20:25:09.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.846+0000 7fe2f9000640 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665a4f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8001eb0 tx=0x7fe2e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2e7138df60b943 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.846+0000 7fe2e1600640 1 -- 172.21.15.131:0/3688214013 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e804a070 con 0x564c46651050 2024-04-26T20:25:09.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.846+0000 7fe2e1600640 1 -- 172.21.15.131:0/3688214013 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e8046aa0 con 0x564c46651050 2024-04-26T20:25:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.846+0000 7fe2e1600640 1 -- 172.21.15.131:0/3688214013 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8037040 con 0x564c46651050 2024-04-26T20:25:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/3688214013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 msgr2=0x564c4665a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665a4f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8001eb0 tx=0x7fe2e8000bc0 comp rx=0 tx=0).stop 2024-04-26T20:25:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/3688214013 shutdown_connections 2024-04-26T20:25:09.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46656bb0 0x564c4665aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46651a40 0x564c46662370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/3688214013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665a4f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/3688214013 >> 172.21.15.131:0/3688214013 conn(0x564c4664c6c0 msgr2=0x564c46652d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/3688214013 shutdown_connections 2024-04-26T20:25:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/3688214013 wait complete. 2024-04-26T20:25:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 -- start start 2024-04-26T20:25:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.847+0000 7fe2fe9d6740 1 Processor -- start 2024-04-26T20:25:10.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:10.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46651a40 0x564c4665fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:10.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2fe9d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46656bb0 0x564c46660c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:10.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41b0040 con 0x564c46651050 2024-04-26T20:25:10.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41b14b0 con 0x564c46656bb0 2024-04-26T20:25:10.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2fe9d6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41b0710 con 0x564c46651a40 2024-04-26T20:25:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2fe9d6740 10 client.0 _ll_register_callbacks cb 0x7fe2e4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:10.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46651a40 0x564c4665fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:10.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50996/0 (socket says 172.21.15.131:50996) 2024-04-26T20:25:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 -- 172.21.15.131:0/2374882460 learned_addr learned my addr 172.21.15.131:0/2374882460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9a00640 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46656bb0 0x564c46660c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:10.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 -- 172.21.15.131:0/2374882460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46651a40 msgr2=0x564c4665fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:10.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46651a40 0x564c4665fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 -- 172.21.15.131:0/2374882460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46656bb0 msgr2=0x564c46660c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:10.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46656bb0 0x564c46660c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f9000640 1 -- 172.21.15.131:0/2374882460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4655b2a0 con 0x564c46651050 2024-04-26T20:25:10.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.848+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46651a40 0x564c4665fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:10.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2f9000640 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665f6e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8046a70 tx=0x7fe2e80309b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2e4e17615a61f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:10.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:10.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2374882460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8061070 con 0x564c46651050 2024-04-26T20:25:10.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2374882460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe2e8030d50 con 0x564c46651050 2024-04-26T20:25:10.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e8046aa0 con 0x564c46651050 2024-04-26T20:25:10.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2374882460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2e806d050 con 0x564c46651050 2024-04-26T20:25:10.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2fe9d6740 10 client.16263 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:10.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c46662350 con 0x564c46651050 2024-04-26T20:25:10.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.849+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2374882460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e8046aa0 con 0x564c46651050 2024-04-26T20:25:10.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2374882460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe2e8003890 con 0x564c46651050 2024-04-26T20:25:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2fe9d6740 10 client.16263 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2fe9d6740 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-26T20:25:10.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2fe9d6740 20 client.16263 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:10.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466ccd30 0x564c466cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:10.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2fe9d6740 4 client.16263 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:25:10.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c465bc6f0 con 0x564c466ccd30 2024-04-26T20:25:10.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.850+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466ccd30 0x564c466cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:10.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2f3e00640 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466ccd30 0x564c466cf150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b53dcee904daf30 server_cookie=1cb71542fccdcd79 in_seq=0 out_seq=0 2024-04-26T20:25:10.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2d6a00640 10 client.16263 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:25:10.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2d6a00640 1 -- 172.21.15.131:0/2374882460 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c465bc6f0 con 0x564c466ccd30 2024-04-26T20:25:10.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2d6a00640 10 client.16263 handle_command_reply: tid=0 2024-04-26T20:25:10.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 client.16263 shutdown 2024-04-26T20:25:10.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 msgr2=0x564c4665f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:10.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665f6e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8046a70 tx=0x7fe2e80309b0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 shutdown_connections 2024-04-26T20:25:10.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x564c466ccd30 0x564c466cf150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c46656bb0 0x564c46660c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:10.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c46651a40 0x564c4665fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 --2- 172.21.15.131:0/2374882460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c46651050 0x564c4665f6e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2d6a00640 0 client.16263 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:25:11.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 >> 172.21.15.131:0/2374882460 conn(0x564c4664c6c0 msgr2=0x564c4664f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:11.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 shutdown_connections 2024-04-26T20:25:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.851+0000 7fe2fe9d6740 1 -- 172.21.15.131:0/2374882460 wait complete. 2024-04-26T20:25:11.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fe9d6740 20 client.16263 trim_cache size 0 max 16384 2024-04-26T20:25:11.176 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:25:11.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:07 smithi157 ceph-mon[32703]: pgmap v863: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:11.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:09 smithi157 ceph-mon[32703]: pgmap v864: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:11.209 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:07 smithi148 ceph-mon[32281]: pgmap v863: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:11.219 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:09 smithi148 ceph-mon[32281]: pgmap v864: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2dc0a3cc0 msgr2=0x7fe2dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2dc0a3cc0 0x7fe2dc0a6180 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x564c466522a0 tx=0x7fe2e40095c0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f41063e0 msgr2=0x7fe2f41a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f41063e0 0x7fe2f41a65f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00eb40 tx=0x7fe2ec00c2d0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 shutdown_connections 2024-04-26T20:25:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f41063e0 0x7fe2f41a65f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f4103a30 0x7fe2f419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe2dc0a3cc0 0x7fe2dc0a6180 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 --2- 172.21.15.131:0/3452004716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f41010c0 0x7fe2f419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 >> 172.21.15.131:0/3452004716 conn(0x7fe2f40fcce0 msgr2=0x7fe2f4104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 shutdown_connections 2024-04-26T20:25:11.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:23:57.852+0000 7fe2fa400640 1 -- 172.21.15.131:0/3452004716 wait complete. 2024-04-26T20:25:11.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T20:25:11.465 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:23:59 smithi131 ceph-mon[27919]: pgmap v829: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:11.476 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:10 smithi157 ceph-mon[32703]: pgmap v865: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:11.486 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:10 smithi148 ceph-mon[32281]: pgmap v865: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.041+0000 7f7829800640 1 -- 172.21.15.131:0/2033816791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78241065a0 msgr2=0x7f782410ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.041+0000 7f7829800640 1 --2- 172.21.15.131:0/2033816791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78241065a0 0x7f782410ce30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f781c009ca0 tx=0x7f781c02a8b0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.042+0000 7f7829800640 1 -- 172.21.15.131:0/2033816791 shutdown_connections 2024-04-26T20:25:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.042+0000 7f7829800640 1 --2- 172.21.15.131:0/2033816791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78241065a0 0x7f782410ce30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.042+0000 7f7829800640 1 --2- 172.21.15.131:0/2033816791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7824105be0 0x7f7824106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.042+0000 7f7829800640 1 --2- 172.21.15.131:0/2033816791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78241049f0 0x7f7824104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.042+0000 7f7829800640 1 -- 172.21.15.131:0/2033816791 >> 172.21.15.131:0/2033816791 conn(0x7f78240fffa0 msgr2=0x7f78241023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.042+0000 7f7829800640 1 -- 172.21.15.131:0/2033816791 shutdown_connections 2024-04-26T20:25:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.042+0000 7f7829800640 1 -- 172.21.15.131:0/2033816791 wait complete. 2024-04-26T20:25:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.043+0000 7f7829800640 1 Processor -- start 2024-04-26T20:25:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.043+0000 7f7829800640 1 -- start start 2024-04-26T20:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.043+0000 7f7829800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78241049f0 0x7f782419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.043+0000 7f7829800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7824105be0 0x7f782419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.043+0000 7f7829800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78241065a0 0x7f78241a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7829800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7824110570 con 0x7f78241065a0 2024-04-26T20:25:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7829800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78241103f0 con 0x7f78241049f0 2024-04-26T20:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7829800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78241106f0 con 0x7f7824105be0 2024-04-26T20:25:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78241049f0 0x7f782419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78241049f0 0x7f782419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42512/0 (socket says 172.21.15.131:42512) 2024-04-26T20:25:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78241065a0 0x7f78241a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 -- 172.21.15.131:0/773064697 learned_addr learned my addr 172.21.15.131:0/773064697 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f781b400640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7824105be0 0x7f782419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 -- 172.21.15.131:0/773064697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7824105be0 msgr2=0x7f782419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7824105be0 0x7f782419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 -- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78241065a0 msgr2=0x7f78241a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78241065a0 0x7f78241a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.044+0000 7f7823e00640 1 -- 172.21.15.131:0/773064697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78241a2800 con 0x7f78241049f0 2024-04-26T20:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.045+0000 7f7823e00640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78241049f0 0x7f782419abe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f781400e9c0 tx=0x7f781400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.045+0000 7f7821600640 1 -- 172.21.15.131:0/773064697 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781400cd80 con 0x7f78241049f0 2024-04-26T20:25:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.045+0000 7f7821600640 1 -- 172.21.15.131:0/773064697 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78140035f0 con 0x7f78241049f0 2024-04-26T20:25:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.045+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78241a2af0 con 0x7f78241049f0 2024-04-26T20:25:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.045+0000 7f7821600640 1 -- 172.21.15.131:0/773064697 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814010690 con 0x7f78241049f0 2024-04-26T20:25:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.045+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241a3040 con 0x7f78241049f0 2024-04-26T20:25:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.048+0000 7f782d7fd740 1 Processor -- start 2024-04-26T20:25:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.048+0000 7f782d7fd740 1 -- start start 2024-04-26T20:25:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.048+0000 7f7821600640 1 -- 172.21.15.131:0/773064697 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f78140037a0 con 0x7f78241049f0 2024-04-26T20:25:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b69078f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b690796dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f782d7fd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b6906f3380 con 0x55b69078ede0 2024-04-26T20:25:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f782d7fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b69068fc30 con 0x55b690794940 2024-04-26T20:25:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f782d7fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b6906ef340 con 0x55b69078f7d0 2024-04-26T20:25:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b69078f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7823e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b69078f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51026/0 (socket says 172.21.15.131:51026) 2024-04-26T20:25:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 -- 172.21.15.131:0/4180677420 learned_addr learned my addr 172.21.15.131:0/4180677420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f781b400640 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b690796dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 -- 172.21.15.131:0/4180677420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 msgr2=0x55b6907942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 -- 172.21.15.131:0/4180677420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 msgr2=0x55b690796dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b690796dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f7828e00640 1 -- 172.21.15.131:0/4180677420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6906b1940 con 0x55b69078ede0 2024-04-26T20:25:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.049+0000 7f781b400640 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b690796dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.050+0000 7f7828e00640 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b69078f200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f781c035000 tx=0x7f781c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4513417ad24e209 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.050+0000 7f7819600640 1 -- 172.21.15.131:0/4180677420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c030900 con 0x55b69078ede0 2024-04-26T20:25:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.050+0000 7f7819600640 1 -- 172.21.15.131:0/4180677420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c036030 con 0x55b69078ede0 2024-04-26T20:25:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.050+0000 7f7819600640 1 -- 172.21.15.131:0/4180677420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c004460 con 0x55b69078ede0 2024-04-26T20:25:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.050+0000 7f7821600640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78080a3c30 0x7f78080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.050+0000 7f7821600640 1 -- 172.21.15.131:0/773064697 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f78140d0d40 con 0x7f78241049f0 2024-04-26T20:25:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.050+0000 7f7823e00640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78080a3c30 0x7f78080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 -- 172.21.15.131:0/4180677420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 msgr2=0x55b69078f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b69078f200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f781c035000 tx=0x7f781c009a20 comp rx=0 tx=0).stop 2024-04-26T20:25:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 -- 172.21.15.131:0/4180677420 shutdown_connections 2024-04-26T20:25:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b690796dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 --2- 172.21.15.131:0/4180677420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b69078f200 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 -- 172.21.15.131:0/4180677420 >> 172.21.15.131:0/4180677420 conn(0x55b69078a450 msgr2=0x55b69078a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 -- 172.21.15.131:0/4180677420 shutdown_connections 2024-04-26T20:25:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 -- 172.21.15.131:0/4180677420 wait complete. 2024-04-26T20:25:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f7823e00640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78080a3c30 0x7f78080a60f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55b69078fbd0 tx=0x7f78140d47f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.051+0000 7f782d7fd740 1 -- start start 2024-04-26T20:25:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 1 Processor -- start 2024-04-26T20:25:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b690798fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b690799500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b6907a0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b690797890 con 0x55b690794940 2024-04-26T20:25:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b69067ac60 con 0x55b69078f7d0 2024-04-26T20:25:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b690797a10 con 0x55b69078ede0 2024-04-26T20:25:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f782d7fd740 10 client.0 _ll_register_callbacks cb 0x55b6906a6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b6907a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.052+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b6907a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51032/0 (socket says 172.21.15.131:51032) 2024-04-26T20:25:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 -- 172.21.15.131:0/1381611522 learned_addr learned my addr 172.21.15.131:0/1381611522 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f781b400640 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b690799500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b690798fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 -- 172.21.15.131:0/1381611522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 msgr2=0x55b690798fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b690798fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 -- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 msgr2=0x55b690799500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b690799500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 -- 172.21.15.131:0/1381611522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b690689510 con 0x55b690794940 2024-04-26T20:25:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7828e00640 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b6907a0540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f781c030e30 tx=0x7f781c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354fb1f8b11c96f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7812000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7812000640 1 -- 172.21.15.131:0/1381611522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c004b40 con 0x55b690794940 2024-04-26T20:25:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f7812000640 1 -- 172.21.15.131:0/1381611522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f781c004ce0 con 0x55b690794940 2024-04-26T20:25:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.053+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f781c036030 con 0x55b690794940 2024-04-26T20:25:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f7812000640 1 -- 172.21.15.131:0/1381611522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f781c003970 con 0x55b690794940 2024-04-26T20:25:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 10 client.16275 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b690405210 con 0x55b690794940 2024-04-26T20:25:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f7812000640 1 -- 172.21.15.131:0/1381611522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c036030 con 0x55b690794940 2024-04-26T20:25:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f7812000640 1 -- 172.21.15.131:0/1381611522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f781c043070 con 0x55b690794940 2024-04-26T20:25:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 10 client.16275 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 20 client.16275 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b6907a46a0 0x55b6907a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 4 client.16275 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.054+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b690700f90 con 0x55b6907a46a0 2024-04-26T20:25:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.055+0000 7f781b400640 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b6907a46a0 0x55b6907a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.055+0000 7f781b400640 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b6907a46a0 0x55b6907a6a90 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd70c54eb7bdf7d server_cookie=ff590a6b6015dda2 in_seq=0 out_seq=0 2024-04-26T20:25:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.055+0000 7f7812000640 10 client.16275 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f7812000640 1 -- 172.21.15.131:0/1381611522 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b690700f90 con 0x55b6907a46a0 2024-04-26T20:25:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f7812000640 10 client.16275 handle_command_reply: tid=0 2024-04-26T20:25:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 client.16275 shutdown 2024-04-26T20:25:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 msgr2=0x55b6907a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b6907a0540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f781c030e30 tx=0x7f781c0093f0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 shutdown_connections 2024-04-26T20:25:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b6907a46a0 0x55b6907a6a90 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b6907a0540 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b690799500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1381611522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b690798fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f7812000640 0 client.16275 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.056+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 >> 172.21.15.131:0/1381611522 conn(0x55b69078a450 msgr2=0x55b690789650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.057+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 shutdown_connections 2024-04-26T20:25:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.057+0000 7f782d7fd740 1 -- 172.21.15.131:0/1381611522 wait complete. 2024-04-26T20:25:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.057+0000 7f782d7fd740 20 client.16275 trim_cache size 0 max 16384 2024-04-26T20:25:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.073+0000 7f782d7fd740 1 Processor -- start 2024-04-26T20:25:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.073+0000 7f782d7fd740 1 -- start start 2024-04-26T20:25:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b690789dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b69078d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f782d7fd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78241aff90 con 0x55b69078ede0 2024-04-26T20:25:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f782d7fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78241ad860 con 0x55b690794940 2024-04-26T20:25:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f782d7fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78241ab6b0 con 0x55b69078f7d0 2024-04-26T20:25:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b690789dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b690789dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51040/0 (socket says 172.21.15.131:51040) 2024-04-26T20:25:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 -- 172.21.15.131:0/1397379921 learned_addr learned my addr 172.21.15.131:0/1397379921 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f781b400640 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 -- 172.21.15.131:0/1397379921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 msgr2=0x55b6907a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 -- 172.21.15.131:0/1397379921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 msgr2=0x55b69078d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b69078d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.074+0000 7f7828e00640 1 -- 172.21.15.131:0/1397379921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6904d1620 con 0x55b69078ede0 2024-04-26T20:25:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.075+0000 7f7828e00640 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b690789dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f781c031040 tx=0x7f781c030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935a3915434ce368 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.075+0000 7f7818c00640 1 -- 172.21.15.131:0/1397379921 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c004450 con 0x55b69078ede0 2024-04-26T20:25:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.075+0000 7f7818c00640 1 -- 172.21.15.131:0/1397379921 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c036040 con 0x55b69078ede0 2024-04-26T20:25:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.075+0000 7f7818c00640 1 -- 172.21.15.131:0/1397379921 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c004eb0 con 0x55b69078ede0 2024-04-26T20:25:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 -- 172.21.15.131:0/1397379921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 msgr2=0x55b690789dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b690789dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f781c031040 tx=0x7f781c030d40 comp rx=0 tx=0).stop 2024-04-26T20:25:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 -- 172.21.15.131:0/1397379921 shutdown_connections 2024-04-26T20:25:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b690794940 0x55b69078d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078f7d0 0x55b6907a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 --2- 172.21.15.131:0/1397379921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b69078ede0 0x55b690789dd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 -- 172.21.15.131:0/1397379921 >> 172.21.15.131:0/1397379921 conn(0x55b69078a450 msgr2=0x55b6907909b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 -- 172.21.15.131:0/1397379921 shutdown_connections 2024-04-26T20:25:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 -- 172.21.15.131:0/1397379921 wait complete. 2024-04-26T20:25:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 -- start start 2024-04-26T20:25:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.076+0000 7f782d7fd740 1 Processor -- start 2024-04-26T20:25:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b69079d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b69079de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f782d7fd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b69079ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f782d7fd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78241abd20 con 0x55b690794940 2024-04-26T20:25:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f782d7fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78241ad190 con 0x55b69078f7d0 2024-04-26T20:25:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f782d7fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78241ac3f0 con 0x55b69078ede0 2024-04-26T20:25:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f782d7fd740 10 client.0 _ll_register_callbacks cb 0x7f78140d46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f7828e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b69079ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f7828e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b69079d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f781b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b69079de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f7828e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b69079d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44638/0 (socket says 172.21.15.131:44638) 2024-04-26T20:25:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.077+0000 7f7828e00640 1 -- 172.21.15.131:0/3780835411 learned_addr learned my addr 172.21.15.131:0/3780835411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f781b400640 1 -- 172.21.15.131:0/3780835411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 msgr2=0x55b69079d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f781b400640 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b69079d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f781b400640 1 -- 172.21.15.131:0/3780835411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 msgr2=0x55b69079ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f781b400640 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b69079ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f781b400640 1 -- 172.21.15.131:0/3780835411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6906adcb0 con 0x55b69078f7d0 2024-04-26T20:25:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f781b400640 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b69079de30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55b6907a7180 tx=0x7f780c020410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89fcbaed36b7a897 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f7812000640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f7812000640 1 -- 172.21.15.131:0/3780835411 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c004230 con 0x55b69078f7d0 2024-04-26T20:25:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f781c036040 con 0x55b69078f7d0 2024-04-26T20:25:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.078+0000 7f7812000640 1 -- 172.21.15.131:0/3780835411 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f780c0043d0 con 0x55b69078f7d0 2024-04-26T20:25:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f7812000640 1 -- 172.21.15.131:0/3780835411 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f780c032050 con 0x55b69078f7d0 2024-04-26T20:25:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 10 client.26396 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b6907a0690 con 0x55b69078f7d0 2024-04-26T20:25:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f7812000640 1 -- 172.21.15.131:0/3780835411 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c036040 con 0x55b69078f7d0 2024-04-26T20:25:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f7812000640 1 -- 172.21.15.131:0/3780835411 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f780c027020 con 0x55b69078f7d0 2024-04-26T20:25:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 10 client.26396 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 10 client.26396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 20 client.26396 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b69080b080 0x55b69080d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 4 client.26396 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6906fae00 con 0x55b69080b080 2024-04-26T20:25:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.079+0000 7f7828e00640 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b69080b080 0x55b69080d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:14.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.080+0000 7f7828e00640 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b69080b080 0x55b69080d4a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=113e518edfc013ad server_cookie=1d599018d0a0d1fc in_seq=0 out_seq=0 2024-04-26T20:25:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.080+0000 7f7812000640 10 client.26396 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.083+0000 7f7812000640 1 -- 172.21.15.131:0/3780835411 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6906fae00 con 0x55b69080b080 2024-04-26T20:25:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.083+0000 7f7812000640 10 client.26396 handle_command_reply: tid=0 2024-04-26T20:25:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.083+0000 7f782d7fd740 1 client.26396 shutdown 2024-04-26T20:25:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 msgr2=0x55b69079de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b69079de30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55b6907a7180 tx=0x7f780c020410 comp rx=0 tx=0).stop 2024-04-26T20:25:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 shutdown_connections 2024-04-26T20:25:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b69080b080 0x55b69080d4a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b690794940 0x55b69079ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b69078f7d0 0x55b69079de30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 --2- 172.21.15.131:0/3780835411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b69078ede0 0x55b69079d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f7812000640 0 client.26396 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 >> 172.21.15.131:0/3780835411 conn(0x55b69078a450 msgr2=0x55b69078d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 shutdown_connections 2024-04-26T20:25:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 1 -- 172.21.15.131:0/3780835411 wait complete. 2024-04-26T20:25:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.084+0000 7f782d7fd740 20 client.26396 trim_cache size 0 max 16384 2024-04-26T20:25:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78080a3c30 msgr2=0x7f78080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78080a3c30 0x7f78080a60f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55b69078fbd0 tx=0x7f78140d47f0 comp rx=0 tx=0).stop 2024-04-26T20:25:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78241049f0 msgr2=0x7f782419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78241049f0 0x7f782419abe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f781400e9c0 tx=0x7f781400ee80 comp rx=0 tx=0).stop 2024-04-26T20:25:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 shutdown_connections 2024-04-26T20:25:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78241065a0 0x7f78241a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f78080a3c30 0x7f78080a60f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7824105be0 0x7f782419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 --2- 172.21.15.131:0/773064697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78241049f0 0x7f782419abe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 >> 172.21.15.131:0/773064697 conn(0x7f78240fffa0 msgr2=0x7f7824108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 shutdown_connections 2024-04-26T20:25:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:00.085+0000 7f7829800640 1 -- 172.21.15.131:0/773064697 wait complete. 2024-04-26T20:25:15.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:01 smithi131 ceph-mon[27919]: pgmap v830: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:15.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:03 smithi131 ceph-mon[27919]: pgmap v831: 129 pgs: 129 active+clean; 695 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:15.146 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:05 smithi131 ceph-mon[27919]: pgmap v832: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:15.153 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:07 smithi131 ceph-mon[27919]: pgmap v833: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:15.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:09 smithi131 ceph-mon[27919]: pgmap v834: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:15.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:11 smithi131 ceph-mon[27919]: pgmap v835: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:15.229 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:13 smithi157 ceph-mon[32703]: pgmap v866: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:15.237 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:13 smithi148 ceph-mon[32281]: pgmap v866: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 -- 172.21.15.131:0/19824303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 msgr2=0x7f72a8105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 --2- 172.21.15.131:0/19824303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 0x7f72a8105800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7298009860 tx=0x7f729802a340 comp rx=0 tx=0).stop 2024-04-26T20:25:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 -- 172.21.15.131:0/19824303 shutdown_connections 2024-04-26T20:25:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 --2- 172.21.15.131:0/19824303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8108670 0x7f72a810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 --2- 172.21.15.131:0/19824303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8105d40 0x7f72a8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 --2- 172.21.15.131:0/19824303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 0x7f72a8105800 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 -- 172.21.15.131:0/19824303 >> 172.21.15.131:0/19824303 conn(0x7f72a80fce20 msgr2=0x7f72a80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 -- 172.21.15.131:0/19824303 shutdown_connections 2024-04-26T20:25:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.224+0000 7f72ae000640 1 -- 172.21.15.131:0/19824303 wait complete. 2024-04-26T20:25:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.225+0000 7f72ae000640 1 Processor -- start 2024-04-26T20:25:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.225+0000 7f72ae000640 1 -- start start 2024-04-26T20:25:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72ae000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 0x7f72a819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72ae000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8105d40 0x7f72a819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72ae000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8108670 0x7f72a81a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72ae000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810e4c0 con 0x7f72a8105d40 2024-04-26T20:25:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72ae000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810e340 con 0x7f72a8103410 2024-04-26T20:25:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72ae000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810e640 con 0x7f72a8108670 2024-04-26T20:25:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72a6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8105d40 0x7f72a819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8108670 0x7f72a81a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72a6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8105d40 0x7f72a819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58126/0 (socket says 172.21.15.131:58126) 2024-04-26T20:25:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72a6a00640 1 -- 172.21.15.131:0/2265936837 learned_addr learned my addr 172.21.15.131:0/2265936837 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8108670 0x7f72a81a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34056/0 (socket says 172.21.15.131:34056) 2024-04-26T20:25:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.226+0000 7f72a7400640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 0x7f72a819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a7e00640 1 -- 172.21.15.131:0/2265936837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 msgr2=0x7f72a819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a7e00640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 0x7f72a819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a7e00640 1 -- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8105d40 msgr2=0x7f72a819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:15.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a7e00640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8105d40 0x7f72a819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a7e00640 1 -- 172.21.15.131:0/2265936837 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a81a2940 con 0x7f72a8108670 2024-04-26T20:25:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a7400640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 0x7f72a819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a6a00640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8105d40 0x7f72a819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72a7e00640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8108670 0x7f72a81a22a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72a000ef00 tx=0x7f72a000c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f729fe00640 1 -- 172.21.15.131:0/2265936837 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0004b20 con 0x7f72a8108670 2024-04-26T20:25:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f729fe00640 1 -- 172.21.15.131:0/2265936837 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a0003e20 con 0x7f72a8108670 2024-04-26T20:25:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a81a2c30 con 0x7f72a8108670 2024-04-26T20:25:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f729fe00640 1 -- 172.21.15.131:0/2265936837 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a000b570 con 0x7f72a8108670 2024-04-26T20:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.227+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a81a3180 con 0x7f72a8108670 2024-04-26T20:25:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f729fe00640 1 -- 172.21.15.131:0/2265936837 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f72a001d020 con 0x7f72a8108670 2024-04-26T20:25:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 Processor -- start 2024-04-26T20:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 -- start start 2024-04-26T20:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdbaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdc0730 0x5610cbdc2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5610cbd1f380 con 0x5610cbdbb5c0 2024-04-26T20:25:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5610cbcbbc30 con 0x5610cbdc0730 2024-04-26T20:25:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.231+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610cbd1b340 con 0x5610cbdbabd0 2024-04-26T20:25:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdbaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58138/0 (socket says 172.21.15.131:58138) 2024-04-26T20:25:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 -- 172.21.15.131:0/1926778617 learned_addr learned my addr 172.21.15.131:0/1926778617 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a6a00640 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdc0730 0x5610cbdc2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 -- 172.21.15.131:0/1926778617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 msgr2=0x5610cbdbaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdbaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 -- 172.21.15.131:0/1926778617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdc0730 msgr2=0x5610cbdc2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdc0730 0x5610cbdc2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a7400640 1 -- 172.21.15.131:0/1926778617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cbcdd940 con 0x5610cbdbb5c0 2024-04-26T20:25:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.232+0000 7f72a6a00640 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdc0730 0x5610cbdc2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.233+0000 7f729fe00640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f728c0a3d70 0x7f728c0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.233+0000 7f72a7400640 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc00c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7298009860 tx=0x7f7298005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a0b21e439a8b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.233+0000 7f729fe00640 1 -- 172.21.15.131:0/2265936837 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f72a00d0850 con 0x7f72a8108670 2024-04-26T20:25:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.233+0000 7f72a6a00640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f728c0a3d70 0x7f728c0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.233+0000 7f7293e00640 1 -- 172.21.15.131:0/1926778617 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298037040 con 0x5610cbdbb5c0 2024-04-26T20:25:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.233+0000 7f7293e00640 1 -- 172.21.15.131:0/1926778617 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7298030dd0 con 0x5610cbdbb5c0 2024-04-26T20:25:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.233+0000 7f7293e00640 1 -- 172.21.15.131:0/1926778617 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298032b70 con 0x5610cbdbb5c0 2024-04-26T20:25:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1926778617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 msgr2=0x5610cbdc00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc00c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7298009860 tx=0x7f7298005dc0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72a6a00640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f728c0a3d70 0x7f728c0a6230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5610cbdbbe20 tx=0x7f7294013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1926778617 shutdown_connections 2024-04-26T20:25:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdc0730 0x5610cbdc2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc00c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1926778617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdbaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1926778617 >> 172.21.15.131:0/1926778617 conn(0x5610cbdb6240 msgr2=0x5610cbdb6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1926778617 shutdown_connections 2024-04-26T20:25:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.234+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1926778617 wait complete. 2024-04-26T20:25:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.235+0000 7f72b0fa7740 1 -- start start 2024-04-26T20:25:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.235+0000 7f72b0fa7740 1 Processor -- start 2024-04-26T20:25:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.235+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbabd0 0x5610cbdc4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.235+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdc0730 0x5610cbdcc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5610cbdc3680 con 0x5610cbdc0730 2024-04-26T20:25:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5610cbca6c60 con 0x5610cbdbabd0 2024-04-26T20:25:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610cbdc3800 con 0x5610cbdbb5c0 2024-04-26T20:25:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72b0fa7740 10 client.0 _ll_register_callbacks cb 0x5610cbcd2e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbabd0 0x5610cbdc4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbabd0 0x5610cbdc4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:39934/0 (socket says 172.21.15.131:39934) 2024-04-26T20:25:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 -- 172.21.15.131:0/875203248 learned_addr learned my addr 172.21.15.131:0/875203248 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34062/0 (socket says 172.21.15.131:34062) 2024-04-26T20:25:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7e00640 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdc0730 0x5610cbdcc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 -- 172.21.15.131:0/875203248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbb5c0 msgr2=0x5610cbdc5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 -- 172.21.15.131:0/875203248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdc0730 msgr2=0x5610cbdcc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdc0730 0x5610cbdcc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a7400640 1 -- 172.21.15.131:0/875203248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cbcb5510 con 0x5610cbdbabd0 2024-04-26T20:25:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.236+0000 7f72a6a00640 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f72a7400640 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbabd0 0x5610cbdc4e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7298035040 tx=0x7f729802aa60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7b5a2964cc0edc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f7290c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f7290c00640 1 -- 172.21.15.131:0/875203248 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298037030 con 0x5610cbdbabd0 2024-04-26T20:25:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f7290c00640 1 -- 172.21.15.131:0/875203248 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7298032d00 con 0x5610cbdbabd0 2024-04-26T20:25:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7298030dd0 con 0x5610cbdbabd0 2024-04-26T20:25:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f7290c00640 1 -- 172.21.15.131:0/875203248 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7298057050 con 0x5610cbdbabd0 2024-04-26T20:25:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f72b0fa7740 10 client.26408 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.237+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610cba31210 con 0x5610cbdbabd0 2024-04-26T20:25:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f7290c00640 1 -- 172.21.15.131:0/875203248 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7298030dd0 con 0x5610cbdbabd0 2024-04-26T20:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f7290c00640 1 -- 172.21.15.131:0/875203248 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7298046460 con 0x5610cbdbabd0 2024-04-26T20:25:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f72b0fa7740 10 client.26408 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f72b0fa7740 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f72b0fa7740 20 client.26408 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdd04f0 0x5610cbdd28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f72b0fa7740 4 client.26408 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610cbd2cf90 con 0x5610cbdd04f0 2024-04-26T20:25:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.238+0000 7f72a6a00640 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdd04f0 0x5610cbdd28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.239+0000 7f72a6a00640 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdd04f0 0x5610cbdd28e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d04db0c3fb770d server_cookie=9c60b380afd1523d in_seq=0 out_seq=0 2024-04-26T20:25:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.239+0000 7f7290c00640 10 client.26408 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f7290c00640 1 -- 172.21.15.131:0/875203248 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610cbd2cf90 con 0x5610cbdd04f0 2024-04-26T20:25:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f7290c00640 10 client.26408 handle_command_reply: tid=0 2024-04-26T20:25:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 client.26408 shutdown 2024-04-26T20:25:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbabd0 msgr2=0x5610cbdc4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbabd0 0x5610cbdc4e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7298035040 tx=0x7f729802aa60 comp rx=0 tx=0).stop 2024-04-26T20:25:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 shutdown_connections 2024-04-26T20:25:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdd04f0 0x5610cbdd28e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdc0730 0x5610cbdcc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:17.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f7290c00640 0 client.26408 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/875203248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbabd0 0x5610cbdc4e10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 >> 172.21.15.131:0/875203248 conn(0x5610cbdb6240 msgr2=0x5610cbdb5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 shutdown_connections 2024-04-26T20:25:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 1 -- 172.21.15.131:0/875203248 wait complete. 2024-04-26T20:25:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.240+0000 7f72b0fa7740 20 client.26408 trim_cache size 0 max 16384 2024-04-26T20:25:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 Processor -- start 2024-04-26T20:25:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 -- start start 2024-04-26T20:25:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdb6240 0x5610cbdc4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdcc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72a81b0000 con 0x5610cbdb6240 2024-04-26T20:25:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72a81ad8d0 con 0x5610cbdbb5c0 2024-04-26T20:25:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.257+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72a81ab720 con 0x5610cbdbabd0 2024-04-26T20:25:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdb6240 0x5610cbdc4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdcc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdb6240 0x5610cbdc4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58146/0 (socket says 172.21.15.131:58146) 2024-04-26T20:25:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a7400640 1 -- 172.21.15.131:0/3607705798 learned_addr learned my addr 172.21.15.131:0/3607705798 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a6a00640 1 -- 172.21.15.131:0/3607705798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbb5c0 msgr2=0x5610cbdc4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a6a00640 1 --2- 172.21.15.131:0/3607705798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a6a00640 1 -- 172.21.15.131:0/3607705798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdb6240 msgr2=0x5610cbdc4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a6a00640 1 --2- 172.21.15.131:0/3607705798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdb6240 0x5610cbdc4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a6a00640 1 -- 172.21.15.131:0/3607705798 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cbafd620 con 0x5610cbdbabd0 2024-04-26T20:25:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.258+0000 7f72a6a00640 1 --2- 172.21.15.131:0/3607705798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdcc2b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5610cbdd3160 tx=0x7f7294009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea6df81e09fe883 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.259+0000 7f7293400640 1 -- 172.21.15.131:0/3607705798 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294021db0 con 0x5610cbdbabd0 2024-04-26T20:25:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.259+0000 7f7293400640 1 -- 172.21.15.131:0/3607705798 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7294011030 con 0x5610cbdbabd0 2024-04-26T20:25:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.259+0000 7f7293400640 1 -- 172.21.15.131:0/3607705798 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729402d8f0 con 0x5610cbdbabd0 2024-04-26T20:25:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 -- 172.21.15.131:0/3607705798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 msgr2=0x5610cbdcc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/3607705798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdcc2b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5610cbdd3160 tx=0x7f7294009530 comp rx=0 tx=0).stop 2024-04-26T20:25:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 -- 172.21.15.131:0/3607705798 shutdown_connections 2024-04-26T20:25:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/3607705798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdbb5c0 0x5610cbdc4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/3607705798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdcc2b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/3607705798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdb6240 0x5610cbdc4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 -- 172.21.15.131:0/3607705798 >> 172.21.15.131:0/3607705798 conn(0x5610cbdc0200 msgr2=0x5610cbdb9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 -- 172.21.15.131:0/3607705798 shutdown_connections 2024-04-26T20:25:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 -- 172.21.15.131:0/3607705798 wait complete. 2024-04-26T20:25:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 -- start start 2024-04-26T20:25:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.260+0000 7f72b0fa7740 1 Processor -- start 2024-04-26T20:25:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdb6240 0x5610cbdc99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdc9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72b0fa7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdcae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72a81abd90 con 0x5610cbdbb5c0 2024-04-26T20:25:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72a81ad200 con 0x5610cbdb6240 2024-04-26T20:25:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72b0fa7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72a81ac460 con 0x5610cbdbabd0 2024-04-26T20:25:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72b0fa7740 10 client.0 _ll_register_callbacks cb 0x7f7294009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdcae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72a6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdcae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58150/0 (socket says 172.21.15.131:58150) 2024-04-26T20:25:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72a7e00640 1 -- 172.21.15.131:0/1322556795 learned_addr learned my addr 172.21.15.131:0/1322556795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.261+0000 7f72a7400640 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdb6240 0x5610cbdc99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72a6a00640 1 -- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdb6240 msgr2=0x5610cbdc99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72a6a00640 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdb6240 0x5610cbdc99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72a6a00640 1 -- 172.21.15.131:0/1322556795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 msgr2=0x5610cbdcae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72a6a00640 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdcae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72a6a00640 1 -- 172.21.15.131:0/1322556795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cbcd9cb0 con 0x5610cbdbabd0 2024-04-26T20:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72a7e00640 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdcae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72a6a00640 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdc9f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f729400fe50 tx=0x7f7294009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10264d7cc4b9af5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f7290c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:25:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f7290c00640 1 -- 172.21.15.131:0/1322556795 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294021db0 con 0x5610cbdbabd0 2024-04-26T20:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7294011030 con 0x5610cbdbabd0 2024-04-26T20:25:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.262+0000 7f7290c00640 1 -- 172.21.15.131:0/1322556795 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7294019070 con 0x5610cbdbabd0 2024-04-26T20:25:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f7290c00640 1 -- 172.21.15.131:0/1322556795 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7294043050 con 0x5610cbdbabd0 2024-04-26T20:25:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 10 client.26392 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610cbdcc5c0 con 0x5610cbdbabd0 2024-04-26T20:25:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f7290c00640 1 -- 172.21.15.131:0/1322556795 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7294011030 con 0x5610cbdbabd0 2024-04-26T20:25:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f7290c00640 1 -- 172.21.15.131:0/1322556795 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7294036c80 con 0x5610cbdbabd0 2024-04-26T20:25:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 10 client.26392 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 10 client.26392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 20 client.26392 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdc0730 0x5610cbe391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 4 client.26392 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610cbd26e00 con 0x5610cbdc0730 2024-04-26T20:25:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.263+0000 7f72a7400640 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdc0730 0x5610cbe391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.264+0000 7f72a7400640 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdc0730 0x5610cbe391f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f1960ec3a2252b server_cookie=40759d5e5187dfe3 in_seq=0 out_seq=0 2024-04-26T20:25:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.264+0000 7f7290c00640 10 client.26392 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.267+0000 7f7290c00640 1 -- 172.21.15.131:0/1322556795 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610cbd26e00 con 0x5610cbdc0730 2024-04-26T20:25:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.267+0000 7f7290c00640 10 client.26392 handle_command_reply: tid=0 2024-04-26T20:25:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.267+0000 7f72b0fa7740 1 client.26392 shutdown 2024-04-26T20:25:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 msgr2=0x5610cbdc9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdc9f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f729400fe50 tx=0x7f7294009fd0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 shutdown_connections 2024-04-26T20:25:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610cbdc0730 0x5610cbe391f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610cbdbb5c0 0x5610cbdcae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610cbdbabd0 0x5610cbdc9f20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f7290c00640 0 client.26392 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 --2- 172.21.15.131:0/1322556795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610cbdb6240 0x5610cbdc99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 >> 172.21.15.131:0/1322556795 conn(0x5610cbdc0200 msgr2=0x5610cbdb4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 shutdown_connections 2024-04-26T20:25:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 1 -- 172.21.15.131:0/1322556795 wait complete. 2024-04-26T20:25:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.268+0000 7f72b0fa7740 20 client.26392 trim_cache size 0 max 16384 2024-04-26T20:25:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f728c0a3d70 msgr2=0x7f728c0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f728c0a3d70 0x7f728c0a6230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5610cbdbbe20 tx=0x7f7294013040 comp rx=0 tx=0).stop 2024-04-26T20:25:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8108670 msgr2=0x7f72a81a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8108670 0x7f72a81a22a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72a000ef00 tx=0x7f72a000c740 comp rx=0 tx=0).stop 2024-04-26T20:25:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 shutdown_connections 2024-04-26T20:25:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f728c0a3d70 0x7f728c0a6230 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a8108670 0x7f72a81a22a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a8105d40 0x7f72a819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 --2- 172.21.15.131:0/2265936837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8103410 0x7f72a819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 >> 172.21.15.131:0/2265936837 conn(0x7f72a80fce20 msgr2=0x7f72a8102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 shutdown_connections 2024-04-26T20:25:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:13.269+0000 7f72ae000640 1 -- 172.21.15.131:0/2265936837 wait complete. 2024-04-26T20:25:19.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:13 smithi131 ceph-mon[27919]: pgmap v836: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:15 smithi131 ceph-mon[27919]: pgmap v837: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:19.590 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:17 smithi131 ceph-mon[27919]: pgmap v838: 129 pgs: 129 active+clean; 696 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:19.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:19 smithi131 ceph-mon[27919]: pgmap v839: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:19.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:21 smithi131 ceph-mon[27919]: pgmap v840: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:19.665 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:23 smithi131 ceph-mon[27919]: pgmap v841: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:19.672 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:25 smithi131 ceph-mon[27919]: pgmap v842: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:19.681 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:15 smithi157 ceph-mon[32703]: pgmap v867: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:19.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:17 smithi157 ceph-mon[32703]: pgmap v868: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:19.695 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:15 smithi148 ceph-mon[32281]: pgmap v867: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:19.703 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:17 smithi148 ceph-mon[32281]: pgmap v868: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.418+0000 7f03d4c00640 1 -- 172.21.15.131:0/1582388648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d010ab70 msgr2=0x7f03d0111400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.418+0000 7f03d4c00640 1 --2- 172.21.15.131:0/1582388648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d010ab70 0x7f03d0111400 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009ca0 tx=0x7f03c802a8c0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.419+0000 7f03d4c00640 1 -- 172.21.15.131:0/1582388648 shutdown_connections 2024-04-26T20:25:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.419+0000 7f03d4c00640 1 --2- 172.21.15.131:0/1582388648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d010ab70 0x7f03d0111400 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.419+0000 7f03d4c00640 1 --2- 172.21.15.131:0/1582388648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 0x7f03d010a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.419+0000 7f03d4c00640 1 --2- 172.21.15.131:0/1582388648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d0108fc0 0x7f03d01093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.419+0000 7f03d4c00640 1 -- 172.21.15.131:0/1582388648 >> 172.21.15.131:0/1582388648 conn(0x7f03d0077f80 msgr2=0x7f03d007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.419+0000 7f03d4c00640 1 -- 172.21.15.131:0/1582388648 shutdown_connections 2024-04-26T20:25:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.419+0000 7f03d4c00640 1 -- 172.21.15.131:0/1582388648 wait complete. 2024-04-26T20:25:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 Processor -- start 2024-04-26T20:25:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 -- start start 2024-04-26T20:25:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d0108fc0 0x7f03d019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 0x7f03d019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d010ab70 0x7f03d01a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0114b50 con 0x7f03d010a1b0 2024-04-26T20:25:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01149d0 con 0x7f03d0108fc0 2024-04-26T20:25:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.420+0000 7f03d4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0114cd0 con 0x7f03d010ab70 2024-04-26T20:25:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d0108fc0 0x7f03d019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 0x7f03d019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 0x7f03d019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49270/0 (socket says 172.21.15.131:49270) 2024-04-26T20:25:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cd400640 1 -- 172.21.15.131:0/667296295 learned_addr learned my addr 172.21.15.131:0/667296295 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d0108fc0 0x7f03d019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50628/0 (socket says 172.21.15.131:50628) 2024-04-26T20:25:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03ce800640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d010ab70 0x7f03d01a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 -- 172.21.15.131:0/667296295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d010ab70 msgr2=0x7f03d01a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d010ab70 0x7f03d01a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 -- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 msgr2=0x7f03d019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 0x7f03d019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 -- 172.21.15.131:0/667296295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01a6ce0 con 0x7f03d0108fc0 2024-04-26T20:25:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03ce800640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d010ab70 0x7f03d01a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cd400640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 0x7f03d019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.421+0000 7f03cde00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d0108fc0 0x7f03d019f0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f03d010a010 tx=0x7f03c000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.422+0000 7f03c6a00640 1 -- 172.21.15.131:0/667296295 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0017070 con 0x7f03d0108fc0 2024-04-26T20:25:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.422+0000 7f03c6a00640 1 -- 172.21.15.131:0/667296295 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0009c30 con 0x7f03d0108fc0 2024-04-26T20:25:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.422+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01a6fd0 con 0x7f03d0108fc0 2024-04-26T20:25:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.422+0000 7f03c6a00640 1 -- 172.21.15.131:0/667296295 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000f620 con 0x7f03d0108fc0 2024-04-26T20:25:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.422+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01a7430 con 0x7f03d0108fc0 2024-04-26T20:25:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03c6a00640 1 -- 172.21.15.131:0/667296295 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f03c001a070 con 0x7f03d0108fc0 2024-04-26T20:25:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 Processor -- start 2024-04-26T20:25:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 -- start start 2024-04-26T20:25:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96cd5c0 0x559fa96d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96d2730 0x559fa96d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559fa9631380 con 0x559fa96cd5c0 2024-04-26T20:25:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fa95cdc30 con 0x559fa96d2730 2024-04-26T20:25:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.425+0000 7f03d7a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559fa962d340 con 0x559fa96ccbd0 2024-04-26T20:25:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96d2730 0x559fa96d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96d2730 0x559fa96d4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50640/0 (socket says 172.21.15.131:50640) 2024-04-26T20:25:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 -- 172.21.15.131:0/1032674284 learned_addr learned my addr 172.21.15.131:0/1032674284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cde00640 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96cd5c0 0x559fa96d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03ce800640 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 -- 172.21.15.131:0/1032674284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 msgr2=0x559fa96ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 -- 172.21.15.131:0/1032674284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96cd5c0 msgr2=0x559fa96d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96cd5c0 0x559fa96d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cd400640 1 -- 172.21.15.131:0/1032674284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa95ef940 con 0x559fa96d2730 2024-04-26T20:25:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.426+0000 7f03cde00640 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96cd5c0 0x559fa96d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.427+0000 7f03cd400640 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96d2730 0x559fa96d4bc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f03bc011510 tx=0x7f03bc0119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9f0ccbf589774c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.427+0000 7f03c6a00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03b40a3cf0 0x7f03b40a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.427+0000 7f03bb000640 1 -- 172.21.15.131:0/1032674284 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01e070 con 0x559fa96d2730 2024-04-26T20:25:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.427+0000 7f03bb000640 1 -- 172.21.15.131:0/1032674284 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc00fa70 con 0x559fa96d2730 2024-04-26T20:25:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.427+0000 7f03c6a00640 1 -- 172.21.15.131:0/667296295 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f03c00d0460 con 0x7f03d0108fc0 2024-04-26T20:25:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.427+0000 7f03bb000640 1 -- 172.21.15.131:0/1032674284 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc019ba0 con 0x559fa96d2730 2024-04-26T20:25:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.427+0000 7f03ce800640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03b40a3cf0 0x7f03b40a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 -- 172.21.15.131:0/1032674284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96d2730 msgr2=0x559fa96d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96d2730 0x559fa96d4bc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f03bc011510 tx=0x7f03bc0119d0 comp rx=0 tx=0).stop 2024-04-26T20:25:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03ce800640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03b40a3cf0 0x7f03b40a61b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005e30 tx=0x7f03c8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 -- 172.21.15.131:0/1032674284 shutdown_connections 2024-04-26T20:25:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96d2730 0x559fa96d4bc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96cd5c0 0x559fa96d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1032674284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 -- 172.21.15.131:0/1032674284 >> 172.21.15.131:0/1032674284 conn(0x559fa96c8240 msgr2=0x559fa96c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 -- 172.21.15.131:0/1032674284 shutdown_connections 2024-04-26T20:25:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 -- 172.21.15.131:0/1032674284 wait complete. 2024-04-26T20:25:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 -- start start 2024-04-26T20:25:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.428+0000 7f03d7a56740 1 Processor -- start 2024-04-26T20:25:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96ccbd0 0x559fa96d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96cd5c0 0x559fa96d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03d7a56740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559fa96d56e0 con 0x559fa96d2730 2024-04-26T20:25:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03d7a56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fa95b8c60 con 0x559fa96ccbd0 2024-04-26T20:25:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03d7a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559fa96d5860 con 0x559fa96cd5c0 2024-04-26T20:25:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03d7a56740 10 client.0 _ll_register_callbacks cb 0x559fa95e4e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03cde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96ccbd0 0x559fa96d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03cde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96ccbd0 0x559fa96d6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50646/0 (socket says 172.21.15.131:50646) 2024-04-26T20:25:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03cde00640 1 -- 172.21.15.131:0/2805297910 learned_addr learned my addr 172.21.15.131:0/2805297910 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03ce800640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.429+0000 7f03cd400640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96cd5c0 0x559fa96d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03cde00640 1 -- 172.21.15.131:0/2805297910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96cd5c0 msgr2=0x559fa96d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03cde00640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96cd5c0 0x559fa96d7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03cde00640 1 -- 172.21.15.131:0/2805297910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 msgr2=0x559fa96de320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03cde00640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96de320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03cde00640 1 -- 172.21.15.131:0/2805297910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa95c7510 con 0x559fa96ccbd0 2024-04-26T20:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03cd400640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96cd5c0 0x559fa96d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03cde00640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96ccbd0 0x559fa96d6e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f03c00cd9f0 tx=0x7f03c009bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6eef5df08e7ee7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03af400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03af400640 1 -- 172.21.15.131:0/2805297910 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0021070 con 0x559fa96ccbd0 2024-04-26T20:25:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03af400640 1 -- 172.21.15.131:0/2805297910 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f03c0098800 con 0x559fa96ccbd0 2024-04-26T20:25:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc00fa70 con 0x559fa96ccbd0 2024-04-26T20:25:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.430+0000 7f03af400640 1 -- 172.21.15.131:0/2805297910 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03c0009de0 con 0x559fa96ccbd0 2024-04-26T20:25:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 10 client.26426 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fa9343210 con 0x559fa96ccbd0 2024-04-26T20:25:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03af400640 1 -- 172.21.15.131:0/2805297910 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc00fa70 con 0x559fa96ccbd0 2024-04-26T20:25:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03af400640 1 -- 172.21.15.131:0/2805297910 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03c00cdc10 con 0x559fa96ccbd0 2024-04-26T20:25:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 10 client.26426 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 20 client.26426 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa96e2900 0x559fa96e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 4 client.26426 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa963ef90 con 0x559fa96e2900 2024-04-26T20:25:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.431+0000 7f03cd400640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa96e2900 0x559fa96e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.432+0000 7f03cd400640 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa96e2900 0x559fa96e4d50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffef53c0f30dd173 server_cookie=d9c39b940b2b24a4 in_seq=0 out_seq=0 2024-04-26T20:25:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.432+0000 7f03af400640 10 client.26426 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03af400640 1 -- 172.21.15.131:0/2805297910 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fa963ef90 con 0x559fa96e2900 2024-04-26T20:25:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03af400640 10 client.26426 handle_command_reply: tid=0 2024-04-26T20:25:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 client.26426 shutdown 2024-04-26T20:25:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96ccbd0 msgr2=0x559fa96d6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96ccbd0 0x559fa96d6e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f03c00cd9f0 tx=0x7f03c009bf60 comp rx=0 tx=0).stop 2024-04-26T20:25:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 shutdown_connections 2024-04-26T20:25:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa96e2900 0x559fa96e4d50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96de320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96cd5c0 0x559fa96d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03af400640 0 client.26426 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2805297910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96ccbd0 0x559fa96d6e10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 >> 172.21.15.131:0/2805297910 conn(0x559fa96c8240 msgr2=0x559fa96c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 shutdown_connections 2024-04-26T20:25:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 1 -- 172.21.15.131:0/2805297910 wait complete. 2024-04-26T20:25:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.433+0000 7f03d7a56740 20 client.26426 trim_cache size 0 max 16384 2024-04-26T20:25:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.449+0000 7f03d7a56740 1 Processor -- start 2024-04-26T20:25:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.449+0000 7f03d7a56740 1 -- start start 2024-04-26T20:25:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.449+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.449+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03d7a56740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01b43d0 con 0x559fa96d2730 2024-04-26T20:25:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03d7a56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01b1ca0 con 0x559fa96cd5c0 2024-04-26T20:25:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03d7a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01afaf0 con 0x559fa96ccbd0 2024-04-26T20:25:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50652/0 (socket says 172.21.15.131:50652) 2024-04-26T20:25:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 -- 172.21.15.131:0/2213047212 learned_addr learned my addr 172.21.15.131:0/2213047212 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cde00640 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 -- 172.21.15.131:0/2213047212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 msgr2=0x559fa96cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 -- 172.21.15.131:0/2213047212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 msgr2=0x559fa96c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.450+0000 7f03cd400640 1 -- 172.21.15.131:0/2213047212 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa940f620 con 0x559fa96cd5c0 2024-04-26T20:25:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03cd400640 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96cb730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03bc000c00 tx=0x7f03bc009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b65be301bc63a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03ba600640 1 -- 172.21.15.131:0/2213047212 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01e040 con 0x559fa96cd5c0 2024-04-26T20:25:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03ba600640 1 -- 172.21.15.131:0/2213047212 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc00fe40 con 0x559fa96cd5c0 2024-04-26T20:25:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03ba600640 1 -- 172.21.15.131:0/2213047212 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc00f820 con 0x559fa96cd5c0 2024-04-26T20:25:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03d7a56740 1 -- 172.21.15.131:0/2213047212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 msgr2=0x559fa96cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96cb730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03bc000c00 tx=0x7f03bc009530 comp rx=0 tx=0).stop 2024-04-26T20:25:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03d7a56740 1 -- 172.21.15.131:0/2213047212 shutdown_connections 2024-04-26T20:25:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96cb730 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03d7a56740 1 --2- 172.21.15.131:0/2213047212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.451+0000 7f03d7a56740 1 -- 172.21.15.131:0/2213047212 >> 172.21.15.131:0/2213047212 conn(0x559fa96c8240 msgr2=0x559fa96c8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.452+0000 7f03d7a56740 1 -- 172.21.15.131:0/2213047212 shutdown_connections 2024-04-26T20:25:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.452+0000 7f03d7a56740 1 -- 172.21.15.131:0/2213047212 wait complete. 2024-04-26T20:25:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.452+0000 7f03d7a56740 1 -- start start 2024-04-26T20:25:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.452+0000 7f03d7a56740 1 Processor -- start 2024-04-26T20:25:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.452+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96da870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.452+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03d7a56740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03d7a56740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01b0160 con 0x559fa96d2730 2024-04-26T20:25:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03d7a56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01b15d0 con 0x559fa96cd5c0 2024-04-26T20:25:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03d7a56740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01b0830 con 0x559fa96ccbd0 2024-04-26T20:25:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03d7a56740 10 client.0 _ll_register_callbacks cb 0x7f03c0010b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96dadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:50654/0 (socket says 172.21.15.131:50654) 2024-04-26T20:25:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 -- 172.21.15.131:0/1440043550 learned_addr learned my addr 172.21.15.131:0/1440043550 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cde00640 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03ce800640 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 -- 172.21.15.131:0/1440043550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 msgr2=0x559fa96da870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96da870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 -- 172.21.15.131:0/1440043550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 msgr2=0x559fa96dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96dbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.453+0000 7f03cd400640 1 -- 172.21.15.131:0/1440043550 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa95ebcb0 con 0x559fa96cd5c0 2024-04-26T20:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03cd400640 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96dadb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f03bc01c040 tx=0x7f03bc0093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8857f3446bc847 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03af400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03af400640 1 -- 172.21.15.131:0/1440043550 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01e040 con 0x559fa96cd5c0 2024-04-26T20:25:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc00fe40 con 0x559fa96cd5c0 2024-04-26T20:25:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03af400640 1 -- 172.21.15.131:0/1440043550 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f03bc011050 con 0x559fa96cd5c0 2024-04-26T20:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03af400640 1 -- 172.21.15.131:0/1440043550 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03bc011270 con 0x559fa96cd5c0 2024-04-26T20:25:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03d7a56740 10 client.26432 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fa96dd4f0 con 0x559fa96cd5c0 2024-04-26T20:25:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.454+0000 7f03af400640 1 -- 172.21.15.131:0/1440043550 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc00fe40 con 0x559fa96cd5c0 2024-04-26T20:25:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03af400640 1 -- 172.21.15.131:0/1440043550 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03bc003c90 con 0x559fa96cd5c0 2024-04-26T20:25:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03d7a56740 10 client.26432 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03d7a56740 10 client.26432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03d7a56740 20 client.26432 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa9748ed0 0x559fa974b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03d7a56740 4 client.26432 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fa9638e00 con 0x559fa9748ed0 2024-04-26T20:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03cde00640 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa9748ed0 0x559fa974b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03cde00640 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa9748ed0 0x559fa974b2c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3694ea2cfae1734b server_cookie=4243e11b63d12d5f in_seq=0 out_seq=0 2024-04-26T20:25:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.455+0000 7f03af400640 10 client.26432 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03af400640 1 -- 172.21.15.131:0/1440043550 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fa9638e00 con 0x559fa9748ed0 2024-04-26T20:25:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03af400640 10 client.26432 handle_command_reply: tid=0 2024-04-26T20:25:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 client.26432 shutdown 2024-04-26T20:25:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 msgr2=0x559fa96dadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96dadb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f03bc01c040 tx=0x7f03bc0093f0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 shutdown_connections 2024-04-26T20:25:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fa9748ed0 0x559fa974b2c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fa96d2730 0x559fa96dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa96cd5c0 0x559fa96dadb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 --2- 172.21.15.131:0/1440043550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fa96ccbd0 0x559fa96da870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 >> 172.21.15.131:0/1440043550 conn(0x559fa96c8240 msgr2=0x559fa96d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03af400640 0 client.26432 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 shutdown_connections 2024-04-26T20:25:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.459+0000 7f03d7a56740 1 -- 172.21.15.131:0/1440043550 wait complete. 2024-04-26T20:25:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d7a56740 20 client.26432 trim_cache size 0 max 16384 2024-04-26T20:25:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03b40a3cf0 msgr2=0x7f03b40a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03b40a3cf0 0x7f03b40a61b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005e30 tx=0x7f03c8005da0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d0108fc0 msgr2=0x7f03d019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d0108fc0 0x7f03d019f0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f03d010a010 tx=0x7f03c000dde0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 shutdown_connections 2024-04-26T20:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f03b40a3cf0 0x7f03b40a61b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d010ab70 0x7f03d01a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d010a1b0 0x7f03d019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 --2- 172.21.15.131:0/667296295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d0108fc0 0x7f03d019f0c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 >> 172.21.15.131:0/667296295 conn(0x7f03d0077f80 msgr2=0x7f03d010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.460+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 shutdown_connections 2024-04-26T20:25:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:26.461+0000 7f03d4c00640 1 -- 172.21.15.131:0/667296295 wait complete. 2024-04-26T20:25:23.877 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:27 smithi131 ceph-mon[27919]: pgmap v843: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:23.913 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:29 smithi131 ceph-mon[27919]: pgmap v844: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:25:23.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:31 smithi131 ceph-mon[27919]: pgmap v845: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:23.929 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:33 smithi131 ceph-mon[27919]: pgmap v846: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:23.937 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:35 smithi131 ceph-mon[27919]: pgmap v847: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:23.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:37 smithi131 ceph-mon[27919]: pgmap v848: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:23.975 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:19 smithi157 ceph-mon[32703]: pgmap v869: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:23.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:21 smithi157 ceph-mon[32703]: pgmap v870: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:23.991 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:19 smithi148 ceph-mon[32281]: pgmap v869: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:24.003 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:21 smithi148 ceph-mon[32281]: pgmap v870: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 -- 172.21.15.131:0/2791084869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c103b80 msgr2=0x7f944c104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 --2- 172.21.15.131:0/2791084869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c103b80 0x7f944c104000 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9438009900 tx=0x7f943802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:25:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 -- 172.21.15.131:0/2791084869 shutdown_connections 2024-04-26T20:25:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 --2- 172.21.15.131:0/2791084869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c104540 0x7f944c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 --2- 172.21.15.131:0/2791084869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c103b80 0x7f944c104000 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 --2- 172.21.15.131:0/2791084869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 0x7f944c102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 -- 172.21.15.131:0/2791084869 >> 172.21.15.131:0/2791084869 conn(0x7f944c0fdf20 msgr2=0x7f944c100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 -- 172.21.15.131:0/2791084869 shutdown_connections 2024-04-26T20:25:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.605+0000 7f9450c00640 1 -- 172.21.15.131:0/2791084869 wait complete. 2024-04-26T20:25:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.606+0000 7f9450c00640 1 Processor -- start 2024-04-26T20:25:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9450c00640 1 -- start start 2024-04-26T20:25:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9450c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 0x7f944c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9450c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c103b80 0x7f944c074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9450c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c104540 0x7f944c075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9450c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f944c10e510 con 0x7f944c103b80 2024-04-26T20:25:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9450c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f944c10e390 con 0x7f944c104540 2024-04-26T20:25:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9450c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f944c10e690 con 0x7f944c102990 2024-04-26T20:25:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9449e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 0x7f944c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9449e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 0x7f944c0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56154/0 (socket says 172.21.15.131:56154) 2024-04-26T20:25:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9449e00640 1 -- 172.21.15.131:0/1486071184 learned_addr learned my addr 172.21.15.131:0/1486071184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.607+0000 7f9449400640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c103b80 0x7f944c074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f944a800640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c104540 0x7f944c075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f9449e00640 1 -- 172.21.15.131:0/1486071184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c104540 msgr2=0x7f944c075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f9449e00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c104540 0x7f944c075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f9449e00640 1 -- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c103b80 msgr2=0x7f944c074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f9449e00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c103b80 0x7f944c074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f9449e00640 1 -- 172.21.15.131:0/1486071184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c075af0 con 0x7f944c102990 2024-04-26T20:25:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f944a800640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c104540 0x7f944c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f9449e00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 0x7f944c0766e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f943c00e9c0 tx=0x7f943c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.608+0000 7f9442a00640 1 -- 172.21.15.131:0/1486071184 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c00cd80 con 0x7f944c102990 2024-04-26T20:25:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.609+0000 7f9442a00640 1 -- 172.21.15.131:0/1486071184 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c0035f0 con 0x7f944c102990 2024-04-26T20:25:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.609+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c1a2c10 con 0x7f944c102990 2024-04-26T20:25:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.609+0000 7f9442a00640 1 -- 172.21.15.131:0/1486071184 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c010690 con 0x7f944c102990 2024-04-26T20:25:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.609+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1a30e0 con 0x7f944c102990 2024-04-26T20:25:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 Processor -- start 2024-04-26T20:25:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 -- start start 2024-04-26T20:25:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f9442a00640 1 -- 172.21.15.131:0/1486071184 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f943c004000 con 0x7f944c102990 2024-04-26T20:25:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17efff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55adc1754380 con 0x55adc17f05c0 2024-04-26T20:25:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55adc16f0c30 con 0x55adc17efbd0 2024-04-26T20:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.612+0000 7f945397c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55adc1750340 con 0x55adc17f5730 2024-04-26T20:25:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56160/0 (socket says 172.21.15.131:56160) 2024-04-26T20:25:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 -- 172.21.15.131:0/2287870542 learned_addr learned my addr 172.21.15.131:0/2287870542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f944a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 -- 172.21.15.131:0/2287870542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 msgr2=0x55adc17efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 --2- 172.21.15.131:0/2287870542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 -- 172.21.15.131:0/2287870542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 msgr2=0x55adc17f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 --2- 172.21.15.131:0/2287870542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 -- 172.21.15.131:0/2287870542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc1712940 con 0x55adc17f5730 2024-04-26T20:25:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.613+0000 7f9449400640 1 --2- 172.21.15.131:0/2287870542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f7bc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9438006010 tx=0x7f9438004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4bfdfc3eefbc7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f9437000640 1 -- 172.21.15.131:0/2287870542 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438035030 con 0x55adc17f5730 2024-04-26T20:25:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f9437000640 1 -- 172.21.15.131:0/2287870542 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9438040070 con 0x55adc17f5730 2024-04-26T20:25:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f9437000640 1 -- 172.21.15.131:0/2287870542 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438031920 con 0x55adc17f5730 2024-04-26T20:25:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f9442a00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f94300a3cf0 0x7f94300a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f9442a00640 1 -- 172.21.15.131:0/1486071184 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f943c0d0ff0 con 0x7f944c102990 2024-04-26T20:25:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f944a800640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f94300a3cf0 0x7f94300a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f945397c740 1 -- 172.21.15.131:0/2287870542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 msgr2=0x55adc17f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.614+0000 7f945397c740 1 --2- 172.21.15.131:0/2287870542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f7bc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9438006010 tx=0x7f9438004040 comp rx=0 tx=0).stop 2024-04-26T20:25:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 -- 172.21.15.131:0/2287870542 shutdown_connections 2024-04-26T20:25:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 --2- 172.21.15.131:0/2287870542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f7bc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 --2- 172.21.15.131:0/2287870542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 --2- 172.21.15.131:0/2287870542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 -- 172.21.15.131:0/2287870542 >> 172.21.15.131:0/2287870542 conn(0x55adc17eb240 msgr2=0x55adc17eb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 -- 172.21.15.131:0/2287870542 shutdown_connections 2024-04-26T20:25:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 -- 172.21.15.131:0/2287870542 wait complete. 2024-04-26T20:25:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f944a800640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f94300a3cf0 0x7f94300a61b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55adc17f0e20 tx=0x7f9444009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 -- start start 2024-04-26T20:25:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.615+0000 7f945397c740 1 Processor -- start 2024-04-26T20:25:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f945397c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f945397c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f945397c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc1801300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:25.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f945397c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55adc17f8680 con 0x55adc17f05c0 2024-04-26T20:25:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f945397c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55adc16dbc60 con 0x55adc17efbd0 2024-04-26T20:25:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f945397c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55adc17f8800 con 0x55adc17f5730 2024-04-26T20:25:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f945397c740 10 client.0 _ll_register_callbacks cb 0x55adc1707e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f9449400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f9449400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17fa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33032/0 (socket says 172.21.15.131:33032) 2024-04-26T20:25:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f9449400640 1 -- 172.21.15.131:0/3113767093 learned_addr learned my addr 172.21.15.131:0/3113767093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f944a800640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc1801300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.616+0000 7f9449e00640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f9449400640 1 -- 172.21.15.131:0/3113767093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 msgr2=0x55adc1801300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f9449400640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc1801300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f9449400640 1 -- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 msgr2=0x55adc17f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f9449400640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17f9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f9449400640 1 -- 172.21.15.131:0/3113767093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc16ea510 con 0x55adc17f05c0 2024-04-26T20:25:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f944a800640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc1801300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f9449400640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17fa2c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9438005ab0 tx=0x7f94380044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c880b143d1697a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f942b400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f942b400640 1 -- 172.21.15.131:0/3113767093 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438035070 con 0x55adc17f05c0 2024-04-26T20:25:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f942b400640 1 -- 172.21.15.131:0/3113767093 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9438003a00 con 0x55adc17f05c0 2024-04-26T20:25:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.617+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9438040070 con 0x55adc17f05c0 2024-04-26T20:25:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f942b400640 1 -- 172.21.15.131:0/3113767093 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f943804a070 con 0x55adc17f05c0 2024-04-26T20:25:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 10 client.16329 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adc1466210 con 0x55adc17f05c0 2024-04-26T20:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f942b400640 1 -- 172.21.15.131:0/3113767093 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9438040070 con 0x55adc17f05c0 2024-04-26T20:25:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f942b400640 1 -- 172.21.15.131:0/3113767093 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f94380455e0 con 0x55adc17f05c0 2024-04-26T20:25:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 10 client.16329 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 20 client.16329 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc1805460 0x55adc1807850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 4 client.16329 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.618+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc1761f90 con 0x55adc1805460 2024-04-26T20:25:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.619+0000 7f9449e00640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc1805460 0x55adc1807850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.619+0000 7f9449e00640 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc1805460 0x55adc1807850 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6572235b3b17799b server_cookie=4964cf23cd25a4f in_seq=0 out_seq=0 2024-04-26T20:25:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.619+0000 7f942b400640 10 client.16329 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f942b400640 1 -- 172.21.15.131:0/3113767093 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adc1761f90 con 0x55adc1805460 2024-04-26T20:25:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f942b400640 10 client.16329 handle_command_reply: tid=0 2024-04-26T20:25:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 client.16329 shutdown 2024-04-26T20:25:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 msgr2=0x55adc17fa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17fa2c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9438005ab0 tx=0x7f94380044e0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 shutdown_connections 2024-04-26T20:25:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc1805460 0x55adc1807850 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc1801300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc17fa2c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f942b400640 0 client.16329 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 --2- 172.21.15.131:0/3113767093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 >> 172.21.15.131:0/3113767093 conn(0x55adc17eb240 msgr2=0x55adc17ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 shutdown_connections 2024-04-26T20:25:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.620+0000 7f945397c740 1 -- 172.21.15.131:0/3113767093 wait complete. 2024-04-26T20:25:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.621+0000 7f945397c740 20 client.16329 trim_cache size 0 max 16384 2024-04-26T20:25:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 Processor -- start 2024-04-26T20:25:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 -- start start 2024-04-26T20:25:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc18012c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1aff60 con 0x55adc17f05c0 2024-04-26T20:25:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1ad830 con 0x55adc17efbd0 2024-04-26T20:25:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.637+0000 7f945397c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1ab680 con 0x55adc17f5730 2024-04-26T20:25:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc18012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc18012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33036/0 (socket says 172.21.15.131:33036) 2024-04-26T20:25:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f944a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449400640 1 -- 172.21.15.131:0/2651924224 learned_addr learned my addr 172.21.15.131:0/2651924224 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449e00640 1 -- 172.21.15.131:0/2651924224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 msgr2=0x55adc17f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449e00640 1 --2- 172.21.15.131:0/2651924224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449e00640 1 -- 172.21.15.131:0/2651924224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 msgr2=0x55adc18012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449e00640 1 --2- 172.21.15.131:0/2651924224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc18012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449e00640 1 -- 172.21.15.131:0/2651924224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc1532620 con 0x55adc17efbd0 2024-04-26T20:25:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.638+0000 7f9449e00640 1 --2- 172.21.15.131:0/2651924224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17eb620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f943c0ce550 tx=0x7f943c0d4aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b0a6803e475061 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f9436600640 1 -- 172.21.15.131:0/2651924224 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c09caf0 con 0x55adc17efbd0 2024-04-26T20:25:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f9436600640 1 -- 172.21.15.131:0/2651924224 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c09d420 con 0x55adc17efbd0 2024-04-26T20:25:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f9436600640 1 -- 172.21.15.131:0/2651924224 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c09c680 con 0x55adc17efbd0 2024-04-26T20:25:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f945397c740 1 -- 172.21.15.131:0/2651924224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 msgr2=0x55adc17eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f945397c740 1 --2- 172.21.15.131:0/2651924224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17eb620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f943c0ce550 tx=0x7f943c0d4aa0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f945397c740 1 -- 172.21.15.131:0/2651924224 shutdown_connections 2024-04-26T20:25:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f945397c740 1 --2- 172.21.15.131:0/2651924224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f5730 0x55adc17f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f945397c740 1 --2- 172.21.15.131:0/2651924224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f05c0 0x55adc18012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f945397c740 1 --2- 172.21.15.131:0/2651924224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17eb620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.639+0000 7f945397c740 1 -- 172.21.15.131:0/2651924224 >> 172.21.15.131:0/2651924224 conn(0x55adc17eb240 msgr2=0x55adc17f4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.640+0000 7f945397c740 1 -- 172.21.15.131:0/2651924224 shutdown_connections 2024-04-26T20:25:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.640+0000 7f945397c740 1 -- 172.21.15.131:0/2651924224 wait complete. 2024-04-26T20:25:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.640+0000 7f945397c740 1 -- start start 2024-04-26T20:25:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.640+0000 7f945397c740 1 Processor -- start 2024-04-26T20:25:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f945397c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17fe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f945397c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f05c0 0x55adc17fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f945397c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f5730 0x55adc17ff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f945397c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1abcf0 con 0x55adc17f5730 2024-04-26T20:25:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f945397c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1ad160 con 0x55adc17efbd0 2024-04-26T20:25:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f945397c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1ac3c0 con 0x55adc17f05c0 2024-04-26T20:25:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f945397c740 10 client.0 _ll_register_callbacks cb 0x7f943c0d4970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17fe440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:37702/0 (socket says 172.21.15.131:37702) 2024-04-26T20:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 -- 172.21.15.131:0/3629058777 learned_addr learned my addr 172.21.15.131:0/3629058777 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f944a800640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f5730 0x55adc17ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449400640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f05c0 0x55adc17fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 -- 172.21.15.131:0/3629058777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f05c0 msgr2=0x55adc17fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f05c0 0x55adc17fe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 -- 172.21.15.131:0/3629058777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f5730 msgr2=0x55adc17ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f5730 0x55adc17ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449e00640 1 -- 172.21.15.131:0/3629058777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc170ecb0 con 0x55adc17efbd0 2024-04-26T20:25:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.641+0000 7f9449400640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f05c0 0x55adc17fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f9449e00640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17fe440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f943c09ccb0 tx=0x7f943c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f80cb35c35edd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f942b400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f942b400640 1 -- 172.21.15.131:0/3629058777 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c09c860 con 0x55adc17efbd0 2024-04-26T20:25:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943c09d420 con 0x55adc17efbd0 2024-04-26T20:25:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f942b400640 1 -- 172.21.15.131:0/3629058777 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f943c099390 con 0x55adc17efbd0 2024-04-26T20:25:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f942b400640 1 -- 172.21.15.131:0/3629058777 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f943c09caf0 con 0x55adc17efbd0 2024-04-26T20:25:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 10 client.26450 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adc18010c0 con 0x55adc17efbd0 2024-04-26T20:25:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f942b400640 1 -- 172.21.15.131:0/3629058777 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c09d420 con 0x55adc17efbd0 2024-04-26T20:25:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f942b400640 1 -- 172.21.15.131:0/3629058777 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f943c0ce7a0 con 0x55adc17efbd0 2024-04-26T20:25:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 10 client.26450 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 10 client.26450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 20 client.26450 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc186bb40 0x55adc186df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 4 client.26450 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.642+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55adc175be00 con 0x55adc186bb40 2024-04-26T20:25:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.643+0000 7f9449400640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc186bb40 0x55adc186df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.643+0000 7f9449400640 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc186bb40 0x55adc186df60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e30482eaf1c0b3 server_cookie=93eaf6537dfa51e8 in_seq=0 out_seq=0 2024-04-26T20:25:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.643+0000 7f942b400640 10 client.26450 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f942b400640 1 -- 172.21.15.131:0/3629058777 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55adc175be00 con 0x55adc186bb40 2024-04-26T20:25:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f942b400640 10 client.26450 handle_command_reply: tid=0 2024-04-26T20:25:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 client.26450 shutdown 2024-04-26T20:25:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 msgr2=0x55adc17fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17fe440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f943c09ccb0 tx=0x7f943c005740 comp rx=0 tx=0).stop 2024-04-26T20:25:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 shutdown_connections 2024-04-26T20:25:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55adc186bb40 0x55adc186df60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adc17f5730 0x55adc17ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc17f05c0 0x55adc17fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 --2- 172.21.15.131:0/3629058777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adc17efbd0 0x55adc17fe440 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 >> 172.21.15.131:0/3629058777 conn(0x55adc17eb240 msgr2=0x55adc17f4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f942b400640 0 client.26450 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 shutdown_connections 2024-04-26T20:25:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 1 -- 172.21.15.131:0/3629058777 wait complete. 2024-04-26T20:25:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.647+0000 7f945397c740 20 client.26450 trim_cache size 0 max 16384 2024-04-26T20:25:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f94300a3cf0 msgr2=0x7f94300a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f94300a3cf0 0x7f94300a61b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55adc17f0e20 tx=0x7f9444009500 comp rx=0 tx=0).stop 2024-04-26T20:25:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 msgr2=0x7f944c0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 0x7f944c0766e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f943c00e9c0 tx=0x7f943c00ee80 comp rx=0 tx=0).stop 2024-04-26T20:25:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 shutdown_connections 2024-04-26T20:25:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944c104540 0x7f944c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c103b80 0x7f944c074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f94300a3cf0 0x7f94300a61b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 --2- 172.21.15.131:0/1486071184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944c102990 0x7f944c0766e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 >> 172.21.15.131:0/1486071184 conn(0x7f944c0fdf20 msgr2=0x7f944c106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 shutdown_connections 2024-04-26T20:25:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:39.648+0000 7f9450c00640 1 -- 172.21.15.131:0/1486071184 wait complete. 2024-04-26T20:25:28.196 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:39 smithi131 ceph-mon[27919]: pgmap v849: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:28.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:41 smithi131 ceph-mon[27919]: pgmap v850: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:28.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:43 smithi131 ceph-mon[27919]: pgmap v851: 129 pgs: 129 active+clean; 697 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:28.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:45 smithi131 ceph-mon[27919]: pgmap v852: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:28.322 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:47 smithi131 ceph-mon[27919]: pgmap v853: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:28.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:49 smithi131 ceph-mon[27919]: pgmap v854: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:28.340 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:51 smithi131 ceph-mon[27919]: pgmap v855: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:28.350 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:23 smithi157 ceph-mon[32703]: pgmap v871: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:28.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:25 smithi157 ceph-mon[32703]: pgmap v872: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:28.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:27 smithi157 ceph-mon[32703]: pgmap v873: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:28.398 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:23 smithi148 ceph-mon[32281]: pgmap v871: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:28.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:25 smithi148 ceph-mon[32281]: pgmap v872: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:28.421 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:27 smithi148 ceph-mon[32281]: pgmap v873: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.794+0000 7f806fe00640 1 -- 172.21.15.131:0/3989956600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80701028a0 msgr2=0x7f8070102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.794+0000 7f806fe00640 1 --2- 172.21.15.131:0/3989956600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80701028a0 0x7f8070102d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8058009900 tx=0x7f805802a410 comp rx=0 tx=0).stop 2024-04-26T20:25:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.795+0000 7f806fe00640 1 -- 172.21.15.131:0/3989956600 shutdown_connections 2024-04-26T20:25:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.795+0000 7f806fe00640 1 --2- 172.21.15.131:0/3989956600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8070103260 0x7f8070109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.795+0000 7f806fe00640 1 --2- 172.21.15.131:0/3989956600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80701028a0 0x7f8070102d20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.795+0000 7f806fe00640 1 --2- 172.21.15.131:0/3989956600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701016b0 0x7f8070101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.795+0000 7f806fe00640 1 -- 172.21.15.131:0/3989956600 >> 172.21.15.131:0/3989956600 conn(0x7f80700fcc20 msgr2=0x7f80700ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.795+0000 7f806fe00640 1 -- 172.21.15.131:0/3989956600 shutdown_connections 2024-04-26T20:25:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.795+0000 7f806fe00640 1 -- 172.21.15.131:0/3989956600 wait complete. 2024-04-26T20:25:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 Processor -- start 2024-04-26T20:25:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 -- start start 2024-04-26T20:25:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80701016b0 0x7f8070198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701028a0 0x7f8070198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8070103260 0x7f807019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f807010e360 con 0x7f8070103260 2024-04-26T20:25:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f807010e1e0 con 0x7f80701016b0 2024-04-26T20:25:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.796+0000 7f806fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f807010e4e0 con 0x7f80701028a0 2024-04-26T20:25:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701028a0 0x7f8070198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f806f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8070103260 0x7f807019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701028a0 0x7f8070198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42552/0 (socket says 172.21.15.131:42552) 2024-04-26T20:25:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 -- 172.21.15.131:0/4242441505 learned_addr learned my addr 172.21.15.131:0/4242441505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f806ea00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80701016b0 0x7f8070198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 -- 172.21.15.131:0/4242441505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80701016b0 msgr2=0x7f8070198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80701016b0 0x7f8070198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 -- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8070103260 msgr2=0x7f807019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8070103260 0x7f807019fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 -- 172.21.15.131:0/4242441505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80701a0660 con 0x7f80701028a0 2024-04-26T20:25:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.797+0000 7f8067e00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701028a0 0x7f8070198fd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8058005ab0 tx=0x7f8058003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.798+0000 7f8067400640 1 -- 172.21.15.131:0/4242441505 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805803f070 con 0x7f80701028a0 2024-04-26T20:25:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.798+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80701a08f0 con 0x7f80701028a0 2024-04-26T20:25:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.798+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80701a0e30 con 0x7f80701028a0 2024-04-26T20:25:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.798+0000 7f8067400640 1 -- 172.21.15.131:0/4242441505 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8058031c50 con 0x7f80701028a0 2024-04-26T20:25:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.798+0000 7f8067400640 1 -- 172.21.15.131:0/4242441505 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058049060 con 0x7f80701028a0 2024-04-26T20:25:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8067400640 1 -- 172.21.15.131:0/4242441505 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8058042070 con 0x7f80701028a0 2024-04-26T20:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 Processor -- start 2024-04-26T20:25:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 -- start start 2024-04-26T20:25:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b075610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b07ad50 0x557b1b07d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557b1afd5380 con 0x557b1b0751f0 2024-04-26T20:25:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557b1af71c30 con 0x557b1b07ad50 2024-04-26T20:25:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.801+0000 7f8078967740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557b1afd1340 con 0x557b1b075be0 2024-04-26T20:25:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b075610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f8067e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b07ad50 0x557b1b07d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42560/0 (socket says 172.21.15.131:42560) 2024-04-26T20:25:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 -- 172.21.15.131:0/1239323164 learned_addr learned my addr 172.21.15.131:0/1239323164 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 -- 172.21.15.131:0/1239323164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b07ad50 msgr2=0x557b1b07d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 --2- 172.21.15.131:0/1239323164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b07ad50 0x557b1b07d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 -- 172.21.15.131:0/1239323164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 msgr2=0x557b1b075610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 --2- 172.21.15.131:0/1239323164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b075610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.802+0000 7f806ea00640 1 -- 172.21.15.131:0/1239323164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1af93940 con 0x557b1b075be0 2024-04-26T20:25:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.803+0000 7f806ea00640 1 --2- 172.21.15.131:0/1239323164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07a6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8060009ca0 tx=0x7f8060011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc88ff77da57b364 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.803+0000 7f8067400640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f80540a3c30 0x7f80540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.803+0000 7f805fe00640 1 -- 172.21.15.131:0/1239323164 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060004220 con 0x557b1b075be0 2024-04-26T20:25:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.803+0000 7f805fe00640 1 -- 172.21.15.131:0/1239323164 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80600043c0 con 0x557b1b075be0 2024-04-26T20:25:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.803+0000 7f805fe00640 1 -- 172.21.15.131:0/1239323164 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060019830 con 0x557b1b075be0 2024-04-26T20:25:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.803+0000 7f8067400640 1 -- 172.21.15.131:0/4242441505 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f80580f0da0 con 0x7f80701028a0 2024-04-26T20:25:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.803+0000 7f806f400640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f80540a3c30 0x7f80540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 -- 172.21.15.131:0/1239323164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 msgr2=0x557b1b07a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:29.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 --2- 172.21.15.131:0/1239323164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07a6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8060009ca0 tx=0x7f8060011810 comp rx=0 tx=0).stop 2024-04-26T20:25:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 -- 172.21.15.131:0/1239323164 shutdown_connections 2024-04-26T20:25:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 --2- 172.21.15.131:0/1239323164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b07ad50 0x557b1b07d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 --2- 172.21.15.131:0/1239323164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07a6e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 --2- 172.21.15.131:0/1239323164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b075610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 -- 172.21.15.131:0/1239323164 >> 172.21.15.131:0/1239323164 conn(0x557b1b06c450 msgr2=0x557b1b06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 -- 172.21.15.131:0/1239323164 shutdown_connections 2024-04-26T20:25:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f806f400640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f80540a3c30 0x7f80540a60f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x557b1b076440 tx=0x7f8068009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 -- 172.21.15.131:0/1239323164 wait complete. 2024-04-26T20:25:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.804+0000 7f8078967740 1 -- start start 2024-04-26T20:25:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 1 Processor -- start 2024-04-26T20:25:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b07f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b07ad50 0x557b1b086970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557b1b07dd00 con 0x557b1b07ad50 2024-04-26T20:25:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557b1af5cc60 con 0x557b1b0751f0 2024-04-26T20:25:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557b1b07de80 con 0x557b1b075be0 2024-04-26T20:25:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8078967740 10 client.0 _ll_register_callbacks cb 0x557b1af88e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f806f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b07ad50 0x557b1b086970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f806f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b07ad50 0x557b1b086970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47292/0 (socket says 172.21.15.131:47292) 2024-04-26T20:25:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f8067e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f806f400640 1 -- 172.21.15.131:0/2371586617 learned_addr learned my addr 172.21.15.131:0/2371586617 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.805+0000 7f806ea00640 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b07f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f8067e00640 1 -- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 msgr2=0x557b1b07f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f8067e00640 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b07f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f8067e00640 1 -- 172.21.15.131:0/2371586617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b07ad50 msgr2=0x557b1b086970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f8067e00640 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b07ad50 0x557b1b086970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f8067e00640 1 -- 172.21.15.131:0/2371586617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1af6b510 con 0x557b1b075be0 2024-04-26T20:25:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f8067e00640 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07f9a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f80580ee300 tx=0x7f805802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=992d2bb0646c5b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f805cc00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:25:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f805cc00640 1 -- 172.21.15.131:0/2371586617 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580bc610 con 0x557b1b075be0 2024-04-26T20:25:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80600043c0 con 0x557b1b075be0 2024-04-26T20:25:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.806+0000 7f805cc00640 1 -- 172.21.15.131:0/2371586617 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f805803b460 con 0x557b1b075be0 2024-04-26T20:25:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f805cc00640 1 -- 172.21.15.131:0/2371586617 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80580bc430 con 0x557b1b075be0 2024-04-26T20:25:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 10 client.26440 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b1ace7210 con 0x557b1b075be0 2024-04-26T20:25:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f805cc00640 1 -- 172.21.15.131:0/2371586617 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80600043c0 con 0x557b1b075be0 2024-04-26T20:25:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f805cc00640 1 -- 172.21.15.131:0/2371586617 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f805804c060 con 0x557b1b075be0 2024-04-26T20:25:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 10 client.26440 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 10 client.26440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 20 client.26440 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b08afa0 0x557b1b08d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 4 client.26440 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.807+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b1afe2f90 con 0x557b1b08afa0 2024-04-26T20:25:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.808+0000 7f806ea00640 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b08afa0 0x557b1b08d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.808+0000 7f806ea00640 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b08afa0 0x557b1b08d3d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f708cfdeeaf7fcb server_cookie=e7a2be24bc5b63c9 in_seq=0 out_seq=0 2024-04-26T20:25:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.808+0000 7f805cc00640 10 client.26440 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f805cc00640 1 -- 172.21.15.131:0/2371586617 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b1afe2f90 con 0x557b1b08afa0 2024-04-26T20:25:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f805cc00640 10 client.26440 handle_command_reply: tid=0 2024-04-26T20:25:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 client.26440 shutdown 2024-04-26T20:25:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 msgr2=0x557b1b07f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07f9a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f80580ee300 tx=0x7f805802a410 comp rx=0 tx=0).stop 2024-04-26T20:25:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 shutdown_connections 2024-04-26T20:25:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b08afa0 0x557b1b08d3d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b07ad50 0x557b1b086970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b075be0 0x557b1b07f9a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 --2- 172.21.15.131:0/2371586617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b07f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 >> 172.21.15.131:0/2371586617 conn(0x557b1b06c450 msgr2=0x557b1b06b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f805cc00640 0 client.26440 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 shutdown_connections 2024-04-26T20:25:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 1 -- 172.21.15.131:0/2371586617 wait complete. 2024-04-26T20:25:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.809+0000 7f8078967740 20 client.26440 trim_cache size 0 max 16384 2024-04-26T20:25:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.825+0000 7f8078967740 1 Processor -- start 2024-04-26T20:25:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8078967740 1 -- start start 2024-04-26T20:25:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8078967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8078967740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b075be0 0x557b1b086930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8078967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b091cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8078967740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80701abad0 con 0x557b1b0751f0 2024-04-26T20:25:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8078967740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f80701a93a0 con 0x557b1b075be0 2024-04-26T20:25:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8078967740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80701a71f0 con 0x557b1b07ad50 2024-04-26T20:25:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b06fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47304/0 (socket says 172.21.15.131:47304) 2024-04-26T20:25:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 -- 172.21.15.131:0/2734260703 learned_addr learned my addr 172.21.15.131:0/2734260703 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f8067e00640 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b075be0 0x557b1b086930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806f400640 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b091cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 -- 172.21.15.131:0/2734260703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 msgr2=0x557b1b091cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b091cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 -- 172.21.15.131:0/2734260703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b075be0 msgr2=0x557b1b086930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b075be0 0x557b1b086930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.826+0000 7f806ea00640 1 -- 172.21.15.131:0/2734260703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1adb3620 con 0x557b1b0751f0 2024-04-26T20:25:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.827+0000 7f8067e00640 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b075be0 0x557b1b086930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.827+0000 7f806ea00640 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b06fb10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8060009250 tx=0x7f8060011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dd51c446d19ad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.827+0000 7f805f400640 1 -- 172.21.15.131:0/2734260703 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806002bb50 con 0x557b1b0751f0 2024-04-26T20:25:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.827+0000 7f805f400640 1 -- 172.21.15.131:0/2734260703 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806002b7f0 con 0x557b1b0751f0 2024-04-26T20:25:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.827+0000 7f805f400640 1 -- 172.21.15.131:0/2734260703 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060005370 con 0x557b1b0751f0 2024-04-26T20:25:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 -- 172.21.15.131:0/2734260703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 msgr2=0x557b1b06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b06fb10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8060009250 tx=0x7f8060011810 comp rx=0 tx=0).stop 2024-04-26T20:25:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 -- 172.21.15.131:0/2734260703 shutdown_connections 2024-04-26T20:25:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b091cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b075be0 0x557b1b086930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 --2- 172.21.15.131:0/2734260703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b0751f0 0x557b1b06fb10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 -- 172.21.15.131:0/2734260703 >> 172.21.15.131:0/2734260703 conn(0x557b1b06c450 msgr2=0x557b1b06f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 -- 172.21.15.131:0/2734260703 shutdown_connections 2024-04-26T20:25:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 -- 172.21.15.131:0/2734260703 wait complete. 2024-04-26T20:25:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.828+0000 7f8078967740 1 -- start start 2024-04-26T20:25:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 1 Processor -- start 2024-04-26T20:25:31.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b084120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b075be0 0x557b1b084660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b085620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80701a7860 con 0x557b1b075be0 2024-04-26T20:25:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f80701a8cd0 con 0x557b1b0751f0 2024-04-26T20:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80701a7f30 con 0x557b1b07ad50 2024-04-26T20:25:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8078967740 10 client.0 _ll_register_callbacks cb 0x7f80580f4540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8067e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b075be0 0x557b1b084660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8067e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b075be0 0x557b1b084660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47316/0 (socket says 172.21.15.131:47316) 2024-04-26T20:25:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f8067e00640 1 -- 172.21.15.131:0/1931618243 learned_addr learned my addr 172.21.15.131:0/1931618243 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f806ea00640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b084120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.829+0000 7f806f400640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b085620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f8067e00640 1 -- 172.21.15.131:0/1931618243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 msgr2=0x557b1b085620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f8067e00640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b085620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f8067e00640 1 -- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 msgr2=0x557b1b084120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f8067e00640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b084120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f8067e00640 1 -- 172.21.15.131:0/1931618243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1af8fcb0 con 0x557b1b075be0 2024-04-26T20:25:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f806ea00640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b084120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f8067e00640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b075be0 0x557b1b084660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f80580bcd30 tx=0x7f80580314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a43821a6318f0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f805cc00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f805cc00640 1 -- 172.21.15.131:0/1931618243 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580bc610 con 0x557b1b075be0 2024-04-26T20:25:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f805cc00640 1 -- 172.21.15.131:0/1931618243 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8058031e00 con 0x557b1b075be0 2024-04-26T20:25:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.830+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f806002b7f0 con 0x557b1b075be0 2024-04-26T20:25:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f805cc00640 1 -- 172.21.15.131:0/1931618243 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80580bcfc0 con 0x557b1b075be0 2024-04-26T20:25:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 10 client.16353 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b1b0f1550 con 0x557b1b075be0 2024-04-26T20:25:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f805cc00640 1 -- 172.21.15.131:0/1931618243 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806002b7f0 con 0x557b1b075be0 2024-04-26T20:25:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f805cc00640 1 -- 172.21.15.131:0/1931618243 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f80580d8020 con 0x557b1b075be0 2024-04-26T20:25:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 20 client.16353 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b0f1d30 0x557b1b0f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 4 client.16353 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b1afdce00 con 0x557b1b0f1d30 2024-04-26T20:25:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.831+0000 7f806ea00640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b0f1d30 0x557b1b0f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.832+0000 7f806ea00640 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b0f1d30 0x557b1b0f4150 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be2304d4e19fd28 server_cookie=8fc3dbad4809a7aa in_seq=0 out_seq=0 2024-04-26T20:25:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.832+0000 7f805cc00640 10 client.16353 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.835+0000 7f805cc00640 1 -- 172.21.15.131:0/1931618243 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b1afdce00 con 0x557b1b0f1d30 2024-04-26T20:25:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.835+0000 7f805cc00640 10 client.16353 handle_command_reply: tid=0 2024-04-26T20:25:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.835+0000 7f8078967740 1 client.16353 shutdown 2024-04-26T20:25:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b075be0 msgr2=0x557b1b084660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b075be0 0x557b1b084660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f80580bcd30 tx=0x7f80580314c0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 shutdown_connections 2024-04-26T20:25:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557b1b0f1d30 0x557b1b0f4150 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b1b07ad50 0x557b1b085620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b1b075be0 0x557b1b084660 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 --2- 172.21.15.131:0/1931618243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b1b0751f0 0x557b1b084120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f805cc00640 0 client.16353 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 >> 172.21.15.131:0/1931618243 conn(0x557b1b06c450 msgr2=0x557b1b06f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 shutdown_connections 2024-04-26T20:25:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 1 -- 172.21.15.131:0/1931618243 wait complete. 2024-04-26T20:25:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.836+0000 7f8078967740 20 client.16353 trim_cache size 0 max 16384 2024-04-26T20:25:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f80540a3c30 msgr2=0x7f80540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f80540a3c30 0x7f80540a60f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x557b1b076440 tx=0x7f8068009500 comp rx=0 tx=0).stop 2024-04-26T20:25:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701028a0 msgr2=0x7f8070198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701028a0 0x7f8070198fd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8058005ab0 tx=0x7f8058003b90 comp rx=0 tx=0).stop 2024-04-26T20:25:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 shutdown_connections 2024-04-26T20:25:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8070103260 0x7f807019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f80540a3c30 0x7f80540a60f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80701028a0 0x7f8070198fd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 --2- 172.21.15.131:0/4242441505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80701016b0 0x7f8070198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 >> 172.21.15.131:0/4242441505 conn(0x7f80700fcc20 msgr2=0x7f8070105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 shutdown_connections 2024-04-26T20:25:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:24:52.837+0000 7f806fe00640 1 -- 172.21.15.131:0/4242441505 wait complete. 2024-04-26T20:25:32.887 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:29 smithi148 ceph-mon[32281]: pgmap v874: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:32.898 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:31 smithi148 ceph-mon[32281]: pgmap v875: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:32.908 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:29 smithi157 ceph-mon[32703]: pgmap v874: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:32.917 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:31 smithi157 ceph-mon[32703]: pgmap v875: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:32.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:53 smithi131 ceph-mon[27919]: pgmap v856: 129 pgs: 129 active+clean; 698 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:33.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:55 smithi131 ceph-mon[27919]: pgmap v857: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T20:25:33.042 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:56 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:25:33.050 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:57 smithi131 ceph-mon[27919]: pgmap v858: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:33.057 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:25:33.158 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:25:33.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:25:33.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:24:59 smithi131 ceph-mon[27919]: pgmap v859: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:33.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:01 smithi131 ceph-mon[27919]: pgmap v860: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:33.189 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:03 smithi131 ceph-mon[27919]: pgmap v861: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 -- 172.21.15.131:0/3393210673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c0108de0 msgr2=0x7fa1c0109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 --2- 172.21.15.131:0/3393210673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c0108de0 0x7fa1c0109200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009860 tx=0x7fa1b402a340 comp rx=0 tx=0).stop 2024-04-26T20:25:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 -- 172.21.15.131:0/3393210673 shutdown_connections 2024-04-26T20:25:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 --2- 172.21.15.131:0/3393210673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c010a990 0x7fa1c0111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 --2- 172.21.15.131:0/3393210673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c0109fd0 0x7fa1c010a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 --2- 172.21.15.131:0/3393210673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c0108de0 0x7fa1c0109200 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 -- 172.21.15.131:0/3393210673 >> 172.21.15.131:0/3393210673 conn(0x7fa1c0077dc0 msgr2=0x7fa1c007a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 -- 172.21.15.131:0/3393210673 shutdown_connections 2024-04-26T20:25:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.980+0000 7fa1c6400640 1 -- 172.21.15.131:0/3393210673 wait complete. 2024-04-26T20:25:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.981+0000 7fa1c6400640 1 Processor -- start 2024-04-26T20:25:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.981+0000 7fa1c6400640 1 -- start start 2024-04-26T20:25:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c6400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c0108de0 0x7fa1c019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c6400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c0109fd0 0x7fa1c019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c6400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c010a990 0x7fa1c01a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c6400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c0114900 con 0x7fa1c0109fd0 2024-04-26T20:25:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c6400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c0114780 con 0x7fa1c010a990 2024-04-26T20:25:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c6400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c0114a80 con 0x7fa1c0108de0 2024-04-26T20:25:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c0109fd0 0x7fa1c019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c0109fd0 0x7fa1c019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33202/0 (socket says 172.21.15.131:33202) 2024-04-26T20:25:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1bfe00640 1 -- 172.21.15.131:0/1616961089 learned_addr learned my addr 172.21.15.131:0/1616961089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c5000640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c0108de0 0x7fa1c019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1c5a00640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c010a990 0x7fa1c01a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1bfe00640 1 -- 172.21.15.131:0/1616961089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c0108de0 msgr2=0x7fa1c019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c0108de0 0x7fa1c019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1bfe00640 1 -- 172.21.15.131:0/1616961089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c010a990 msgr2=0x7fa1c01a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.982+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c010a990 0x7fa1c01a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.983+0000 7fa1bfe00640 1 -- 172.21.15.131:0/1616961089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c01a6c50 con 0x7fa1c0109fd0 2024-04-26T20:25:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.983+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c0109fd0 0x7fa1c019f570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e9f0 tx=0x7fa1ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.983+0000 7fa1bd600640 1 -- 172.21.15.131:0/1616961089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00ccc0 con 0x7fa1c0109fd0 2024-04-26T20:25:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.983+0000 7fa1bd600640 1 -- 172.21.15.131:0/1616961089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ac0054d0 con 0x7fa1c0109fd0 2024-04-26T20:25:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.983+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c01a6f40 con 0x7fa1c0109fd0 2024-04-26T20:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.983+0000 7fa1bd600640 1 -- 172.21.15.131:0/1616961089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac010430 con 0x7fa1c0109fd0 2024-04-26T20:25:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.983+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c01a73b0 con 0x7fa1c0109fd0 2024-04-26T20:25:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1bd600640 1 -- 172.21.15.131:0/1616961089 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa1ac0046c0 con 0x7fa1c0109fd0 2024-04-26T20:25:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 Processor -- start 2024-04-26T20:25:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 -- start start 2024-04-26T20:25:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c4cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e393ba9380 con 0x55e393c44de0 2024-04-26T20:25:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e393b45c30 con 0x55e393c4a940 2024-04-26T20:25:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1ca845740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e393ba5340 con 0x55e393c457d0 2024-04-26T20:25:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1c5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1c5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c45200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33210/0 (socket says 172.21.15.131:33210) 2024-04-26T20:25:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1c5a00640 1 -- 172.21.15.131:0/3171644201 learned_addr learned my addr 172.21.15.131:0/3171644201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1c5000640 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.987+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c4cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1c5a00640 1 -- 172.21.15.131:0/3171644201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 msgr2=0x55e393c4a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1c5a00640 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1c5a00640 1 -- 172.21.15.131:0/3171644201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 msgr2=0x55e393c4cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1c5a00640 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c4cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1c5a00640 1 -- 172.21.15.131:0/3171644201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e393b67940 con 0x55e393c44de0 2024-04-26T20:25:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c4cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1c5a00640 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c45200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80039f0 tx=0x7fa1b8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe82dcb008edb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1bd600640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa1a80a3c30 0x7fa1a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1b2000640 1 -- 172.21.15.131:0/3171644201 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b801a470 con 0x55e393c44de0 2024-04-26T20:25:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1b2000640 1 -- 172.21.15.131:0/3171644201 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b8015070 con 0x55e393c44de0 2024-04-26T20:25:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1b2000640 1 -- 172.21.15.131:0/3171644201 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b80107e0 con 0x55e393c44de0 2024-04-26T20:25:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.988+0000 7fa1bd600640 1 -- 172.21.15.131:0/1616961089 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa1ac0d0bf0 con 0x7fa1c0109fd0 2024-04-26T20:25:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1c5000640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa1a80a3c30 0x7fa1a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 -- 172.21.15.131:0/3171644201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 msgr2=0x55e393c45200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c45200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80039f0 tx=0x7fa1b8003eb0 comp rx=0 tx=0).stop 2024-04-26T20:25:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1c5000640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa1a80a3c30 0x7fa1a80a60f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009830 tx=0x7fa1b402c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 -- 172.21.15.131:0/3171644201 shutdown_connections 2024-04-26T20:25:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c4cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3171644201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c45200 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 -- 172.21.15.131:0/3171644201 >> 172.21.15.131:0/3171644201 conn(0x55e393c40450 msgr2=0x55e393c40850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 -- 172.21.15.131:0/3171644201 shutdown_connections 2024-04-26T20:25:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.989+0000 7fa1ca845740 1 -- 172.21.15.131:0/3171644201 wait complete. 2024-04-26T20:25:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.990+0000 7fa1ca845740 1 -- start start 2024-04-26T20:25:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.990+0000 7fa1ca845740 1 Processor -- start 2024-04-26T20:25:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c56570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1ca845740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e393c4d8f0 con 0x55e393c44de0 2024-04-26T20:25:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1ca845740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e393b30c60 con 0x55e393c4a940 2024-04-26T20:25:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1ca845740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e393c4da70 con 0x55e393c457d0 2024-04-26T20:25:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1ca845740 10 client.0 _ll_register_callbacks cb 0x55e393b5ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33224/0 (socket says 172.21.15.131:33224) 2024-04-26T20:25:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 -- 172.21.15.131:0/3926803784 learned_addr learned my addr 172.21.15.131:0/3926803784 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5a00640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c56570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 -- 172.21.15.131:0/3926803784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 msgr2=0x55e393c4f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 -- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 msgr2=0x55e393c56570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c56570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1c5000640 1 -- 172.21.15.131:0/3926803784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e393b3f510 con 0x55e393c44de0 2024-04-26T20:25:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.991+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1c5000640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4eff0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40057c0 tx=0x7fa1b402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b58d2a8db7d54b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3926803784 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4037040 con 0x55e393c44de0 2024-04-26T20:25:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3926803784 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa1b40037f0 con 0x55e393c44de0 2024-04-26T20:25:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b8015070 con 0x55e393c44de0 2024-04-26T20:25:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3926803784 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1b4058050 con 0x55e393c44de0 2024-04-26T20:25:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1ca845740 10 client.16365 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3938bb210 con 0x55e393c44de0 2024-04-26T20:25:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.992+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3926803784 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b8015070 con 0x55e393c44de0 2024-04-26T20:25:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3926803784 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa1b404a880 con 0x55e393c44de0 2024-04-26T20:25:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1ca845740 10 client.16365 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1ca845740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1ca845740 20 client.16365 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393c5aac0 0x55e393c5cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1ca845740 4 client.16365 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e393bb6f90 con 0x55e393c5aac0 2024-04-26T20:25:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.993+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393c5aac0 0x55e393c5cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.994+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393c5aac0 0x55e393c5cf50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa2e8e850491957 server_cookie=cc855f535e58695 in_seq=0 out_seq=0 2024-04-26T20:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.994+0000 7fa1a6a00640 10 client.16365 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3926803784 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e393bb6f90 con 0x55e393c5aac0 2024-04-26T20:25:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1a6a00640 10 client.16365 handle_command_reply: tid=0 2024-04-26T20:25:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 client.16365 shutdown 2024-04-26T20:25:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 msgr2=0x55e393c4eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4eff0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40057c0 tx=0x7fa1b402a340 comp rx=0 tx=0).stop 2024-04-26T20:25:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 shutdown_connections 2024-04-26T20:25:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393c5aac0 0x55e393c5cf50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c4a940 0x55e393c56570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c457d0 0x55e393c4f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1a6a00640 0 client.16365 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3926803784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4eff0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 >> 172.21.15.131:0/3926803784 conn(0x55e393c40450 msgr2=0x55e393c3f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 shutdown_connections 2024-04-26T20:25:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 1 -- 172.21.15.131:0/3926803784 wait complete. 2024-04-26T20:25:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:05.995+0000 7fa1ca845740 20 client.16365 trim_cache size 0 max 16384 2024-04-26T20:25:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.011+0000 7fa1ca845740 1 Processor -- start 2024-04-26T20:25:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1ca845740 1 -- start start 2024-04-26T20:25:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c56530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1ca845740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c01b4360 con 0x55e393c44de0 2024-04-26T20:25:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1ca845740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c01b1c30 con 0x55e393c457d0 2024-04-26T20:25:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1ca845740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c01afa80 con 0x55e393c4a940 2024-04-26T20:25:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33228/0 (socket says 172.21.15.131:33228) 2024-04-26T20:25:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1c5000640 1 -- 172.21.15.131:0/2971203344 learned_addr learned my addr 172.21.15.131:0/2971203344 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c56530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.012+0000 7fa1c5a00640 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1c5000640 1 -- 172.21.15.131:0/2971203344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 msgr2=0x55e393c433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1c5000640 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1c5000640 1 -- 172.21.15.131:0/2971203344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 msgr2=0x55e393c56530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1c5000640 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c56530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1c5000640 1 -- 172.21.15.131:0/2971203344 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e393987620 con 0x55e393c44de0 2024-04-26T20:25:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c56530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1c5000640 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4f310 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa1b402a930 tx=0x7fa1b4030d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fc5c2ec823eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1b1600640 1 -- 172.21.15.131:0/2971203344 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4037040 con 0x55e393c44de0 2024-04-26T20:25:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1b1600640 1 -- 172.21.15.131:0/2971203344 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b4061070 con 0x55e393c44de0 2024-04-26T20:25:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.013+0000 7fa1b1600640 1 -- 172.21.15.131:0/2971203344 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b403e970 con 0x55e393c44de0 2024-04-26T20:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 -- 172.21.15.131:0/2971203344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 msgr2=0x55e393c4f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4f310 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa1b402a930 tx=0x7fa1b4030d20 comp rx=0 tx=0).stop 2024-04-26T20:25:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 -- 172.21.15.131:0/2971203344 shutdown_connections 2024-04-26T20:25:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c56530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 --2- 172.21.15.131:0/2971203344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c4f310 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 -- 172.21.15.131:0/2971203344 >> 172.21.15.131:0/2971203344 conn(0x55e393c40450 msgr2=0x55e393c4ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 -- 172.21.15.131:0/2971203344 shutdown_connections 2024-04-26T20:25:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.014+0000 7fa1ca845740 1 -- 172.21.15.131:0/2971203344 wait complete. 2024-04-26T20:25:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 -- start start 2024-04-26T20:25:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 Processor -- start 2024-04-26T20:25:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c53ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c54010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c55020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c01b00f0 con 0x55e393c44de0 2024-04-26T20:25:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c01b1560 con 0x55e393c457d0 2024-04-26T20:25:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c01b07c0 con 0x55e393c4a940 2024-04-26T20:25:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1ca845740 10 client.0 _ll_register_callbacks cb 0x7fa1b40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c53ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1c5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c53ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33230/0 (socket says 172.21.15.131:33230) 2024-04-26T20:25:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.015+0000 7fa1c5000640 1 -- 172.21.15.131:0/3304697099 learned_addr learned my addr 172.21.15.131:0/3304697099 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1c5a00640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c55020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c54010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1c5000640 1 -- 172.21.15.131:0/3304697099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 msgr2=0x55e393c55020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1c5000640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c55020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1c5000640 1 -- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 msgr2=0x55e393c54010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1c5000640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c54010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1c5000640 1 -- 172.21.15.131:0/3304697099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e393b63cb0 con 0x55e393c44de0 2024-04-26T20:25:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c54010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1c5000640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c53ad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1b404ade0 tx=0x7fa1b404a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6239305ba880e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3304697099 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b406a060 con 0x55e393c44de0 2024-04-26T20:25:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3304697099 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa1b4003800 con 0x55e393c44de0 2024-04-26T20:25:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.016+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b4061070 con 0x55e393c44de0 2024-04-26T20:25:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3304697099 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1b4071050 con 0x55e393c44de0 2024-04-26T20:25:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e393c56750 con 0x55e393c44de0 2024-04-26T20:25:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3304697099 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b4061070 con 0x55e393c44de0 2024-04-26T20:25:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3304697099 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa1b403e800 con 0x55e393c44de0 2024-04-26T20:25:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 20 client.16371 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393cc1630 0x55e393cc3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 4 client.16371 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.017+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e393bb0e00 con 0x55e393cc1630 2024-04-26T20:25:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.018+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393cc1630 0x55e393cc3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.018+0000 7fa1bfe00640 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393cc1630 0x55e393cc3a50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607ff5bd5fbeac9 server_cookie=bbb8e739e1c85283 in_seq=0 out_seq=0 2024-04-26T20:25:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.018+0000 7fa1a6a00640 10 client.16371 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1a6a00640 1 -- 172.21.15.131:0/3304697099 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e393bb0e00 con 0x55e393cc1630 2024-04-26T20:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1a6a00640 10 client.16371 handle_command_reply: tid=0 2024-04-26T20:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 client.16371 shutdown 2024-04-26T20:25:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 msgr2=0x55e393c53ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c53ad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1b404ade0 tx=0x7fa1b404a6f0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 shutdown_connections 2024-04-26T20:25:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e393cc1630 0x55e393cc3a50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e393c4a940 0x55e393c55020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e393c457d0 0x55e393c54010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1a6a00640 0 client.16371 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 --2- 172.21.15.131:0/3304697099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e393c44de0 0x55e393c53ad0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 >> 172.21.15.131:0/3304697099 conn(0x55e393c40450 msgr2=0x55e393c433d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.022+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 shutdown_connections 2024-04-26T20:25:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1ca845740 1 -- 172.21.15.131:0/3304697099 wait complete. 2024-04-26T20:25:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1ca845740 20 client.16371 trim_cache size 0 max 16384 2024-04-26T20:25:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa1a80a3c30 msgr2=0x7fa1a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa1a80a3c30 0x7fa1a80a60f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009830 tx=0x7fa1b402c040 comp rx=0 tx=0).stop 2024-04-26T20:25:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c0109fd0 msgr2=0x7fa1c019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c0109fd0 0x7fa1c019f570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e9f0 tx=0x7fa1ac00eeb0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 shutdown_connections 2024-04-26T20:25:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c010a990 0x7fa1c01a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c0109fd0 0x7fa1c019f570 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa1a80a3c30 0x7fa1a80a60f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 --2- 172.21.15.131:0/1616961089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c0108de0 0x7fa1c019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.023+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 >> 172.21.15.131:0/1616961089 conn(0x7fa1c0077dc0 msgr2=0x7fa1c010d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.024+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 shutdown_connections 2024-04-26T20:25:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:06.024+0000 7fa1c6400640 1 -- 172.21.15.131:0/1616961089 wait complete. 2024-04-26T20:25:37.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:33 smithi157 ceph-mon[32703]: pgmap v876: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:37.811 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:35 smithi157 ceph-mon[32703]: pgmap v877: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:25:37.845 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:33 smithi148 ceph-mon[32281]: pgmap v876: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:37.909 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:35 smithi148 ceph-mon[32281]: pgmap v877: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:25:37.919 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:05 smithi131 ceph-mon[27919]: pgmap v862: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:37.926 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:07 smithi131 ceph-mon[27919]: pgmap v863: 129 pgs: 129 active+clean; 699 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:37.936 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:09 smithi131 ceph-mon[27919]: pgmap v864: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:37.982 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:10 smithi131 ceph-mon[27919]: pgmap v865: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:37.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:13 smithi131 ceph-mon[27919]: pgmap v866: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:38.000 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:15 smithi131 ceph-mon[27919]: pgmap v867: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:38.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:17 smithi131 ceph-mon[27919]: pgmap v868: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 -- 172.21.15.131:0/2904736156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541032a0 msgr2=0x7f1954109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 --2- 172.21.15.131:0/2904736156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541032a0 0x7f1954109b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f194c009ca0 tx=0x7f194c02a8c0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 -- 172.21.15.131:0/2904736156 shutdown_connections 2024-04-26T20:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 --2- 172.21.15.131:0/2904736156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541032a0 0x7f1954109b30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 --2- 172.21.15.131:0/2904736156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19541028e0 0x7f1954102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 --2- 172.21.15.131:0/2904736156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19541016f0 0x7f1954101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 -- 172.21.15.131:0/2904736156 >> 172.21.15.131:0/2904736156 conn(0x7f19540fcc60 msgr2=0x7f19540ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 -- 172.21.15.131:0/2904736156 shutdown_connections 2024-04-26T20:25:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.166+0000 7f195ac00640 1 -- 172.21.15.131:0/2904736156 wait complete. 2024-04-26T20:25:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 Processor -- start 2024-04-26T20:25:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 -- start start 2024-04-26T20:25:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541016f0 0x7f1954198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19541028e0 0x7f1954198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19541032a0 0x7f195419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f195410d250 con 0x7f19541032a0 2024-04-26T20:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f195410d0d0 con 0x7f19541028e0 2024-04-26T20:25:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.167+0000 7f195ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f195410d3d0 con 0x7f19541016f0 2024-04-26T20:25:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541016f0 0x7f1954198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f195a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19541032a0 0x7f195419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1958e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19541028e0 0x7f1954198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541016f0 0x7f1954198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52198/0 (socket says 172.21.15.131:52198) 2024-04-26T20:25:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 -- 172.21.15.131:0/3772495369 learned_addr learned my addr 172.21.15.131:0/3772495369 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 -- 172.21.15.131:0/3772495369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19541028e0 msgr2=0x7f1954198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19541028e0 0x7f1954198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 -- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19541032a0 msgr2=0x7f195419ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19541032a0 0x7f195419ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 -- 172.21.15.131:0/3772495369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19541a05b0 con 0x7f19541016f0 2024-04-26T20:25:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.168+0000 7f1959800640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541016f0 0x7f1954198990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f194800e9c0 tx=0x7f194800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.169+0000 7f1952000640 1 -- 172.21.15.131:0/3772495369 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194800cd80 con 0x7f19541016f0 2024-04-26T20:25:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.169+0000 7f1952000640 1 -- 172.21.15.131:0/3772495369 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19480035f0 con 0x7f19541016f0 2024-04-26T20:25:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.169+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19541a08a0 con 0x7f19541016f0 2024-04-26T20:25:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.169+0000 7f1952000640 1 -- 172.21.15.131:0/3772495369 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948010690 con 0x7f19541016f0 2024-04-26T20:25:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.169+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19541a0e40 con 0x7f19541016f0 2024-04-26T20:25:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 Processor -- start 2024-04-26T20:25:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 -- start start 2024-04-26T20:25:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9d6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649fa933380 con 0x5649fa9cf7d0 2024-04-26T20:25:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5649fa8cfc30 con 0x5649fa9d4940 2024-04-26T20:25:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f195f037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649fa92f340 con 0x5649fa9cede0 2024-04-26T20:25:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.173+0000 7f1952000640 1 -- 172.21.15.131:0/3772495369 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f19480037a0 con 0x7f19541016f0 2024-04-26T20:25:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f1958e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9d6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52202/0 (socket says 172.21.15.131:52202) 2024-04-26T20:25:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 -- 172.21.15.131:0/554869972 learned_addr learned my addr 172.21.15.131:0/554869972 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f1959800640 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 -- 172.21.15.131:0/554869972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 msgr2=0x5649fa9d6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9d6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 -- 172.21.15.131:0/554869972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cf7d0 msgr2=0x5649fa9d42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f195a200640 1 -- 172.21.15.131:0/554869972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649fa8f1940 con 0x5649fa9cede0 2024-04-26T20:25:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f1958e00640 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9d6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.174+0000 7f1959800640 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f195a200640 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9cf200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f194c009ca0 tx=0x7f194c0093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a21c420ad1f92a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f1946a00640 1 -- 172.21.15.131:0/554869972 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c031650 con 0x5649fa9cede0 2024-04-26T20:25:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f1946a00640 1 -- 172.21.15.131:0/554869972 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f194c036030 con 0x5649fa9cede0 2024-04-26T20:25:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f1946a00640 1 -- 172.21.15.131:0/554869972 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c004680 con 0x5649fa9cede0 2024-04-26T20:25:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f1952000640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f193c0a3cb0 0x7f193c0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f1952000640 1 -- 172.21.15.131:0/3772495369 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f19480d0cc0 con 0x7f19541016f0 2024-04-26T20:25:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f195f037740 1 -- 172.21.15.131:0/554869972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 msgr2=0x5649fa9cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.175+0000 7f195f037740 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9cf200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f194c009ca0 tx=0x7f194c0093c0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f1959800640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f193c0a3cb0 0x7f193c0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 -- 172.21.15.131:0/554869972 shutdown_connections 2024-04-26T20:25:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9d6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 --2- 172.21.15.131:0/554869972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9cf200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 -- 172.21.15.131:0/554869972 >> 172.21.15.131:0/554869972 conn(0x5649fa9ca450 msgr2=0x5649fa9ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 -- 172.21.15.131:0/554869972 shutdown_connections 2024-04-26T20:25:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 -- 172.21.15.131:0/554869972 wait complete. 2024-04-26T20:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f1959800640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f193c0a3cb0 0x7f193c0a6170 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5649fa9cfbd0 tx=0x7f1948098ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.176+0000 7f195f037740 1 -- start start 2024-04-26T20:25:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 1 Processor -- start 2024-04-26T20:25:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9d4940 0x5649fa9e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649fa9d7710 con 0x5649fa9d4940 2024-04-26T20:25:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5649fa8bac60 con 0x5649fa9cf7d0 2024-04-26T20:25:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649fa9d7890 con 0x5649fa9cede0 2024-04-26T20:25:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195f037740 10 client.0 _ll_register_callbacks cb 0x5649fa8e6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9d4940 0x5649fa9e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f1958e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9d4940 0x5649fa9e03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47938/0 (socket says 172.21.15.131:47938) 2024-04-26T20:25:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195a200640 1 -- 172.21.15.131:0/1670173875 learned_addr learned my addr 172.21.15.131:0/1670173875 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.177+0000 7f195a200640 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f1958e00640 1 -- 172.21.15.131:0/1670173875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 msgr2=0x5649fa9d8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f1958e00640 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9d8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f1958e00640 1 -- 172.21.15.131:0/1670173875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9d4940 msgr2=0x5649fa9e03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f1958e00640 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9d4940 0x5649fa9e03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f1958e00640 1 -- 172.21.15.131:0/1670173875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649fa8c9510 con 0x5649fa9cf7d0 2024-04-26T20:25:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f195a200640 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f1958e00640 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d93b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1954199ea0 tx=0x7f1940012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=285cc996a5faf5b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f193b400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f193b400640 1 -- 172.21.15.131:0/1670173875 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940004030 con 0x5649fa9cf7d0 2024-04-26T20:25:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f194c036030 con 0x5649fa9cf7d0 2024-04-26T20:25:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.178+0000 7f193b400640 1 -- 172.21.15.131:0/1670173875 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f19400041d0 con 0x5649fa9cf7d0 2024-04-26T20:25:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f193b400640 1 -- 172.21.15.131:0/1670173875 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1940025050 con 0x5649fa9cf7d0 2024-04-26T20:25:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 10 client.26498 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649fa645210 con 0x5649fa9cf7d0 2024-04-26T20:25:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f193b400640 1 -- 172.21.15.131:0/1670173875 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f194c036030 con 0x5649fa9cf7d0 2024-04-26T20:25:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f193b400640 1 -- 172.21.15.131:0/1670173875 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1940003930 con 0x5649fa9cf7d0 2024-04-26T20:25:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 10 client.26498 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 10 client.26498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 20 client.26498 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649fa9e49b0 0x5649fa9e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 4 client.26498 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.179+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649fa940f90 con 0x5649fa9e49b0 2024-04-26T20:25:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.180+0000 7f195a200640 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649fa9e49b0 0x5649fa9e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.180+0000 7f195a200640 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649fa9e49b0 0x5649fa9e6dd0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72d78a897e6d979 server_cookie=b643b8d157296935 in_seq=0 out_seq=0 2024-04-26T20:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.180+0000 7f193b400640 10 client.26498 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f193b400640 1 -- 172.21.15.131:0/1670173875 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649fa940f90 con 0x5649fa9e49b0 2024-04-26T20:25:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f193b400640 10 client.26498 handle_command_reply: tid=0 2024-04-26T20:25:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 client.26498 shutdown 2024-04-26T20:25:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 msgr2=0x5649fa9d93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d93b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1954199ea0 tx=0x7f1940012b60 comp rx=0 tx=0).stop 2024-04-26T20:25:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 shutdown_connections 2024-04-26T20:25:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649fa9e49b0 0x5649fa9e6dd0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9d4940 0x5649fa9e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9d93b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 --2- 172.21.15.131:0/1670173875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cede0 0x5649fa9d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f193b400640 0 client.26498 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 >> 172.21.15.131:0/1670173875 conn(0x5649fa9ca450 msgr2=0x5649fa9c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 shutdown_connections 2024-04-26T20:25:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 1 -- 172.21.15.131:0/1670173875 wait complete. 2024-04-26T20:25:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.181+0000 7f195f037740 20 client.26498 trim_cache size 0 max 16384 2024-04-26T20:25:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 Processor -- start 2024-04-26T20:25:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 -- start start 2024-04-26T20:25:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9e03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9d4940 0x5649fa9d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f19541adca0 con 0x5649fa9cede0 2024-04-26T20:25:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f19541ab570 con 0x5649fa9cf7d0 2024-04-26T20:25:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195f037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19541a93c0 con 0x5649fa9d4940 2024-04-26T20:25:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47942/0 (socket says 172.21.15.131:47942) 2024-04-26T20:25:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f195a200640 1 -- 172.21.15.131:0/1399869218 learned_addr learned my addr 172.21.15.131:0/1399869218 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.198+0000 7f1958e00640 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9e03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f195a200640 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9d4940 0x5649fa9d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1958e00640 1 -- 172.21.15.131:0/1399869218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9d4940 msgr2=0x5649fa9d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1958e00640 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9d4940 0x5649fa9d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1958e00640 1 -- 172.21.15.131:0/1399869218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 msgr2=0x5649fa9d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1958e00640 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1958e00640 1 -- 172.21.15.131:0/1399869218 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649fa711620 con 0x5649fa9cf7d0 2024-04-26T20:25:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f195a200640 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1958e00640 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9e03b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1940000c00 tx=0x7f1940010640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2412686958c091a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1946000640 1 -- 172.21.15.131:0/1399869218 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940022040 con 0x5649fa9cf7d0 2024-04-26T20:25:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1946000640 1 -- 172.21.15.131:0/1399869218 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1940003aa0 con 0x5649fa9cf7d0 2024-04-26T20:25:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.199+0000 7f1946000640 1 -- 172.21.15.131:0/1399869218 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940010a30 con 0x5649fa9cf7d0 2024-04-26T20:25:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 -- 172.21.15.131:0/1399869218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 msgr2=0x5649fa9e03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9e03b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1940000c00 tx=0x7f1940010640 comp rx=0 tx=0).stop 2024-04-26T20:25:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 -- 172.21.15.131:0/1399869218 shutdown_connections 2024-04-26T20:25:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9d4940 0x5649fa9d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9cf7d0 0x5649fa9e03b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 --2- 172.21.15.131:0/1399869218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 -- 172.21.15.131:0/1399869218 >> 172.21.15.131:0/1399869218 conn(0x5649fa9ca450 msgr2=0x5649fa9ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 -- 172.21.15.131:0/1399869218 shutdown_connections 2024-04-26T20:25:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.200+0000 7f195f037740 1 -- 172.21.15.131:0/1399869218 wait complete. 2024-04-26T20:25:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 -- start start 2024-04-26T20:25:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 Processor -- start 2024-04-26T20:25:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cf7d0 0x5649fa9de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f19541a9a30 con 0x5649fa9cede0 2024-04-26T20:25:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f19541aaea0 con 0x5649fa9d4940 2024-04-26T20:25:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19541aa100 con 0x5649fa9cf7d0 2024-04-26T20:25:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195f037740 10 client.0 _ll_register_callbacks cb 0x7f1940009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.201+0000 7f195a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9ddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47956/0 (socket says 172.21.15.131:47956) 2024-04-26T20:25:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 -- 172.21.15.131:0/44420936 learned_addr learned my addr 172.21.15.131:0/44420936 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f1958e00640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cf7d0 0x5649fa9de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 -- 172.21.15.131:0/44420936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cf7d0 msgr2=0x5649fa9de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cf7d0 0x5649fa9de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 -- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 msgr2=0x5649fa9df0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9df0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 -- 172.21.15.131:0/44420936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649fa8edcb0 con 0x5649fa9cede0 2024-04-26T20:25:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f1958e00640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cf7d0 0x5649fa9de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f195a200640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9ddb70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f194c006010 tx=0x7f194c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8432eec446e9e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f193b400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.202+0000 7f193b400640 1 -- 172.21.15.131:0/44420936 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c063060 con 0x5649fa9cede0 2024-04-26T20:25:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f193b400640 1 -- 172.21.15.131:0/44420936 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f194c003d80 con 0x5649fa9cede0 2024-04-26T20:25:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1940003aa0 con 0x5649fa9cede0 2024-04-26T20:25:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f193b400640 1 -- 172.21.15.131:0/44420936 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f194c0040c0 con 0x5649fa9cede0 2024-04-26T20:25:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 10 client.16389 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649faa4af50 con 0x5649fa9cede0 2024-04-26T20:25:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f193b400640 1 -- 172.21.15.131:0/44420936 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1940003aa0 con 0x5649fa9cede0 2024-04-26T20:25:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f193b400640 1 -- 172.21.15.131:0/44420936 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f194c066080 con 0x5649fa9cede0 2024-04-26T20:25:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 10 client.16389 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 20 client.16389 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649faa4b640 0x5649faa4da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 4 client.16389 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.203+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649fa93ae00 con 0x5649faa4b640 2024-04-26T20:25:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.204+0000 7f1958e00640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649faa4b640 0x5649faa4da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.204+0000 7f1958e00640 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649faa4b640 0x5649faa4da60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edadcc24a3b775fc server_cookie=b9edf48782ecd193 in_seq=0 out_seq=0 2024-04-26T20:25:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.204+0000 7f193b400640 10 client.16389 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f193b400640 1 -- 172.21.15.131:0/44420936 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5649fa93ae00 con 0x5649faa4b640 2024-04-26T20:25:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f193b400640 10 client.16389 handle_command_reply: tid=0 2024-04-26T20:25:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 client.16389 shutdown 2024-04-26T20:25:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 msgr2=0x5649fa9ddb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9ddb70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f194c006010 tx=0x7f194c031af0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 shutdown_connections 2024-04-26T20:25:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5649faa4b640 0x5649faa4da60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fa9d4940 0x5649fa9df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fa9cf7d0 0x5649fa9de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 --2- 172.21.15.131:0/44420936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fa9cede0 0x5649fa9ddb70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f193b400640 0 client.16389 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 >> 172.21.15.131:0/44420936 conn(0x5649fa9ca450 msgr2=0x5649fa9d8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 shutdown_connections 2024-04-26T20:25:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 1 -- 172.21.15.131:0/44420936 wait complete. 2024-04-26T20:25:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.208+0000 7f195f037740 20 client.16389 trim_cache size 0 max 16384 2024-04-26T20:25:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f193c0a3cb0 msgr2=0x7f193c0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f193c0a3cb0 0x7f193c0a6170 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5649fa9cfbd0 tx=0x7f1948098ce0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541016f0 msgr2=0x7f1954198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541016f0 0x7f1954198990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f194800e9c0 tx=0x7f194800ee80 comp rx=0 tx=0).stop 2024-04-26T20:25:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 shutdown_connections 2024-04-26T20:25:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19541032a0 0x7f195419ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19541028e0 0x7f1954198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f193c0a3cb0 0x7f193c0a6170 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 --2- 172.21.15.131:0/3772495369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19541016f0 0x7f1954198990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 >> 172.21.15.131:0/3772495369 conn(0x7f19540fcc60 msgr2=0x7f19540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 shutdown_connections 2024-04-26T20:25:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:19.209+0000 7f195ac00640 1 -- 172.21.15.131:0/3772495369 wait complete. 2024-04-26T20:25:41.933 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:19 smithi131 ceph-mon[27919]: pgmap v869: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:41.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:21 smithi131 ceph-mon[27919]: pgmap v870: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:41.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:23 smithi131 ceph-mon[27919]: pgmap v871: 129 pgs: 129 active+clean; 700 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:42.029 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:25 smithi131 ceph-mon[27919]: pgmap v872: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:42.039 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-04-26T20:25:42.048 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-04-26T20:25:42.054 DEBUG:teuthology.orchestra.run.smithi131:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-04-26T20:25:42.152 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:37 smithi157 ceph-mon[32703]: pgmap v878: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:42.159 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:39 smithi157 ceph-mon[32703]: pgmap v879: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:42.166 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:37 smithi148 ceph-mon[32281]: pgmap v878: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:42.172 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:39 smithi148 ceph-mon[32281]: pgmap v879: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:42.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:27 smithi131 ceph-mon[27919]: pgmap v873: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:42.189 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:29 smithi131 ceph-mon[27919]: pgmap v874: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:42.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:31 smithi131 ceph-mon[27919]: pgmap v875: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:42.283 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:41 smithi157 ceph-mon[32703]: pgmap v880: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:42.292 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:41 smithi148 ceph-mon[32281]: pgmap v880: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.349+0000 7f69af200640 1 -- 172.21.15.131:0/3782159511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01028c0 msgr2=0x7f69b0102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.349+0000 7f69af200640 1 --2- 172.21.15.131:0/3782159511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01028c0 0x7f69b0102d40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6998009900 tx=0x7f699802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.349+0000 7f69af200640 1 -- 172.21.15.131:0/3782159511 shutdown_connections 2024-04-26T20:25:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.349+0000 7f69af200640 1 --2- 172.21.15.131:0/3782159511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0103280 0x7f69b0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.349+0000 7f69af200640 1 --2- 172.21.15.131:0/3782159511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01028c0 0x7f69b0102d40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.349+0000 7f69af200640 1 --2- 172.21.15.131:0/3782159511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69b01016d0 0x7f69b0101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.349+0000 7f69af200640 1 -- 172.21.15.131:0/3782159511 >> 172.21.15.131:0/3782159511 conn(0x7f69b00fcc60 msgr2=0x7f69b00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.350+0000 7f69af200640 1 -- 172.21.15.131:0/3782159511 shutdown_connections 2024-04-26T20:25:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.350+0000 7f69af200640 1 -- 172.21.15.131:0/3782159511 wait complete. 2024-04-26T20:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 Processor -- start 2024-04-26T20:25:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 -- start start 2024-04-26T20:25:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01016d0 0x7f69b019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69b01028c0 0x7f69b019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0103280 0x7f69b01a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010e380 con 0x7f69b01016d0 2024-04-26T20:25:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010e200 con 0x7f69b0103280 2024-04-26T20:25:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69af200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010e500 con 0x7f69b01028c0 2024-04-26T20:25:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69ade00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01016d0 0x7f69b019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69ade00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01016d0 0x7f69b019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54248/0 (socket says 172.21.15.131:54248) 2024-04-26T20:25:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69ade00640 1 -- 172.21.15.131:0/1316250845 learned_addr learned my addr 172.21.15.131:0/1316250845 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.351+0000 7f69a7e00640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69b01028c0 0x7f69b019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ae800640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0103280 0x7f69b01a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ade00640 1 -- 172.21.15.131:0/1316250845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69b01028c0 msgr2=0x7f69b019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ade00640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69b01028c0 0x7f69b019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ade00640 1 -- 172.21.15.131:0/1316250845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0103280 msgr2=0x7f69b01a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ade00640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0103280 0x7f69b01a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ade00640 1 -- 172.21.15.131:0/1316250845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b01a2880 con 0x7f69b01016d0 2024-04-26T20:25:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69a7e00640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69b01028c0 0x7f69b019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ae800640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0103280 0x7f69b01a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69ade00640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01016d0 0x7f69b019ac70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69a000e9c0 tx=0x7f69a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69a6a00640 1 -- 172.21.15.131:0/1316250845 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a000cd80 con 0x7f69b01016d0 2024-04-26T20:25:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b01a2b70 con 0x7f69b01016d0 2024-04-26T20:25:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.352+0000 7f69a6a00640 1 -- 172.21.15.131:0/1316250845 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a00035f0 con 0x7f69b01016d0 2024-04-26T20:25:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.353+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b01a3070 con 0x7f69b01016d0 2024-04-26T20:25:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.353+0000 7f69a6a00640 1 -- 172.21.15.131:0/1316250845 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0010630 con 0x7f69b01016d0 2024-04-26T20:25:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.356+0000 7f69a6a00640 1 -- 172.21.15.131:0/1316250845 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f69a0004000 con 0x7f69b01016d0 2024-04-26T20:25:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.356+0000 7f69b7a58740 1 Processor -- start 2024-04-26T20:25:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.356+0000 7f69b7a58740 1 -- start start 2024-04-26T20:25:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb57200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb5c940 0x5610dfb5edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69b7a58740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5610dfabb380 con 0x5610dfb56de0 2024-04-26T20:25:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69b7a58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5610dfa57c30 con 0x5610dfb5c940 2024-04-26T20:25:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69b7a58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610dfab7340 con 0x5610dfb577d0 2024-04-26T20:25:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb57200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ade00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb57200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54254/0 (socket says 172.21.15.131:54254) 2024-04-26T20:25:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69a7e00640 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb5c940 0x5610dfb5edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ae800640 1 -- 172.21.15.131:0/854625985 learned_addr learned my addr 172.21.15.131:0/854625985 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ade00640 1 -- 172.21.15.131:0/854625985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb5c940 msgr2=0x5610dfb5edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ade00640 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb5c940 0x5610dfb5edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ade00640 1 -- 172.21.15.131:0/854625985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 msgr2=0x5610dfb57200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ade00640 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb57200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.357+0000 7f69ade00640 1 -- 172.21.15.131:0/854625985 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610dfa79940 con 0x5610dfb577d0 2024-04-26T20:25:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.358+0000 7f69ade00640 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb5c2d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f69a00ce540 tx=0x7f69a00d9570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ef9a91155e49c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.358+0000 7f69a6a00640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69940a3c30 0x7f69940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.358+0000 7f69ae800640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69940a3c30 0x7f69940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.358+0000 7f69a6a00640 1 -- 172.21.15.131:0/1316250845 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f69a00d0f80 con 0x7f69b01016d0 2024-04-26T20:25:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.358+0000 7f699f000640 1 -- 172.21.15.131:0/854625985 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a00e4070 con 0x5610dfb577d0 2024-04-26T20:25:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.358+0000 7f699f000640 1 -- 172.21.15.131:0/854625985 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a009d410 con 0x5610dfb577d0 2024-04-26T20:25:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.358+0000 7f699f000640 1 -- 172.21.15.131:0/854625985 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a009c670 con 0x5610dfb577d0 2024-04-26T20:25:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 -- 172.21.15.131:0/854625985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 msgr2=0x5610dfb5c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69ae800640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69940a3c30 0x7f69940a60f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69a80099a0 tx=0x7f69a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb5c2d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f69a00ce540 tx=0x7f69a00d9570 comp rx=0 tx=0).stop 2024-04-26T20:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 -- 172.21.15.131:0/854625985 shutdown_connections 2024-04-26T20:25:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb5c940 0x5610dfb5edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb5c2d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 --2- 172.21.15.131:0/854625985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb57200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 -- 172.21.15.131:0/854625985 >> 172.21.15.131:0/854625985 conn(0x5610dfb52450 msgr2=0x5610dfb52850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 -- 172.21.15.131:0/854625985 shutdown_connections 2024-04-26T20:25:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.359+0000 7f69b7a58740 1 -- 172.21.15.131:0/854625985 wait complete. 2024-04-26T20:25:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 -- start start 2024-04-26T20:25:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 Processor -- start 2024-04-26T20:25:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfb68540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5610dfb5f8f0 con 0x5610dfb5c940 2024-04-26T20:25:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5610dfa42c60 con 0x5610dfb56de0 2024-04-26T20:25:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.360+0000 7f69b7a58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610dfb5fa70 con 0x5610dfb577d0 2024-04-26T20:25:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69b7a58740 10 client.0 _ll_register_callbacks cb 0x5610dfa6ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfb68540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfb68540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54266/0 (socket says 172.21.15.131:54266) 2024-04-26T20:25:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 -- 172.21.15.131:0/2616796230 learned_addr learned my addr 172.21.15.131:0/2616796230 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ade00640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69a7e00640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 -- 172.21.15.131:0/2616796230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 msgr2=0x5610dfb61500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 -- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb56de0 msgr2=0x5610dfb60fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ae800640 1 -- 172.21.15.131:0/2616796230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610dfa51510 con 0x5610dfb5c940 2024-04-26T20:25:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.361+0000 7f69ade00640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f69ae800640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfb68540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005f10 tx=0x7f69a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e256a7cbc50dad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f6993400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f6993400640 1 -- 172.21.15.131:0/2616796230 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a802c060 con 0x5610dfb5c940 2024-04-26T20:25:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f6993400640 1 -- 172.21.15.131:0/2616796230 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f69a8017030 con 0x5610dfb5c940 2024-04-26T20:25:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a009d410 con 0x5610dfb5c940 2024-04-26T20:25:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f6993400640 1 -- 172.21.15.131:0/2616796230 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69a8033050 con 0x5610dfb5c940 2024-04-26T20:25:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f69b7a58740 10 client.16401 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610df7cd210 con 0x5610dfb5c940 2024-04-26T20:25:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.362+0000 7f6993400640 1 -- 172.21.15.131:0/2616796230 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a009d410 con 0x5610dfb5c940 2024-04-26T20:25:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f6993400640 1 -- 172.21.15.131:0/2616796230 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f69a801c3f0 con 0x5610dfb5c940 2024-04-26T20:25:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f69b7a58740 10 client.16401 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f69b7a58740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f69b7a58740 20 client.16401 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfb6caf0 0x5610dfb6ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f69b7a58740 4 client.16401 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610dfac8f90 con 0x5610dfb6caf0 2024-04-26T20:25:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.363+0000 7f69ade00640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfb6caf0 0x5610dfb6ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.364+0000 7f69ade00640 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfb6caf0 0x5610dfb6ef40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8836c5bf67320b server_cookie=e737c8e8e1f2cc03 in_seq=0 out_seq=0 2024-04-26T20:25:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.364+0000 7f6993400640 10 client.16401 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f6993400640 1 -- 172.21.15.131:0/2616796230 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610dfac8f90 con 0x5610dfb6caf0 2024-04-26T20:25:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f6993400640 10 client.16401 handle_command_reply: tid=0 2024-04-26T20:25:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 client.16401 shutdown 2024-04-26T20:25:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 msgr2=0x5610dfb68540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfb68540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005f10 tx=0x7f69a8004ab0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 shutdown_connections 2024-04-26T20:25:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfb6caf0 0x5610dfb6ef40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfb68540 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2616796230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f6993400640 0 client.16401 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 >> 172.21.15.131:0/2616796230 conn(0x5610dfb52450 msgr2=0x5610dfb51650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 shutdown_connections 2024-04-26T20:25:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 1 -- 172.21.15.131:0/2616796230 wait complete. 2024-04-26T20:25:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.365+0000 7f69b7a58740 20 client.16401 trim_cache size 0 max 16384 2024-04-26T20:25:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.381+0000 7f69b7a58740 1 Processor -- start 2024-04-26T20:25:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.381+0000 7f69b7a58740 1 -- start start 2024-04-26T20:25:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.381+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb5c940 0x5610dfb68460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69b7a58740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01b0040 con 0x5610dfb56de0 2024-04-26T20:25:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69b7a58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01ad910 con 0x5610dfb577d0 2024-04-26T20:25:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69b7a58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01ab760 con 0x5610dfb5c940 2024-04-26T20:25:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54270/0 (socket says 172.21.15.131:54270) 2024-04-26T20:25:44.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 -- 172.21.15.131:0/2710435538 learned_addr learned my addr 172.21.15.131:0/2710435538 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ae800640 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb5c940 0x5610dfb68460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69a7e00640 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 -- 172.21.15.131:0/2710435538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb5c940 msgr2=0x5610dfb68460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb5c940 0x5610dfb68460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 -- 172.21.15.131:0/2710435538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 msgr2=0x5610dfb61500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ade00640 1 -- 172.21.15.131:0/2710435538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610df899620 con 0x5610dfb56de0 2024-04-26T20:25:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.382+0000 7f69ae800640 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb5c940 0x5610dfb68460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.383+0000 7f69ade00640 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f69a00ceac0 tx=0x7f69a0003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c01c30b1f52b936 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.383+0000 7f699e600640 1 -- 172.21.15.131:0/2710435538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a00d4bc0 con 0x5610dfb56de0 2024-04-26T20:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.383+0000 7f699e600640 1 -- 172.21.15.131:0/2710435538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a009d410 con 0x5610dfb56de0 2024-04-26T20:25:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.383+0000 7f699e600640 1 -- 172.21.15.131:0/2710435538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a009ae50 con 0x5610dfb56de0 2024-04-26T20:25:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 -- 172.21.15.131:0/2710435538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 msgr2=0x5610dfb60fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f69a00ceac0 tx=0x7f69a0003e70 comp rx=0 tx=0).stop 2024-04-26T20:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 -- 172.21.15.131:0/2710435538 shutdown_connections 2024-04-26T20:25:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb5c940 0x5610dfb68460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb61500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 --2- 172.21.15.131:0/2710435538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb56de0 0x5610dfb60fc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 -- 172.21.15.131:0/2710435538 >> 172.21.15.131:0/2710435538 conn(0x5610dfb52450 msgr2=0x5610dfb52830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 -- 172.21.15.131:0/2710435538 shutdown_connections 2024-04-26T20:25:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 -- 172.21.15.131:0/2710435538 wait complete. 2024-04-26T20:25:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 -- start start 2024-04-26T20:25:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.384+0000 7f69b7a58740 1 Processor -- start 2024-04-26T20:25:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb56de0 0x5610dfb654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb65a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69b7a58740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfbd10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69b7a58740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01abdd0 con 0x5610dfb5c940 2024-04-26T20:25:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69b7a58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01ad240 con 0x5610dfb577d0 2024-04-26T20:25:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69b7a58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01ac4a0 con 0x5610dfb56de0 2024-04-26T20:25:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69b7a58740 10 client.0 _ll_register_callbacks cb 0x7f69a8009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69ae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfbd10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69ade00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb56de0 0x5610dfb654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb65a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69ade00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb56de0 0x5610dfb654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:39764/0 (socket says 172.21.15.131:39764) 2024-04-26T20:25:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.385+0000 7f69ade00640 1 -- 172.21.15.131:0/3056524236 learned_addr learned my addr 172.21.15.131:0/3056524236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69a7e00640 1 -- 172.21.15.131:0/3056524236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb56de0 msgr2=0x5610dfb654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69a7e00640 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb56de0 0x5610dfb654c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69a7e00640 1 -- 172.21.15.131:0/3056524236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 msgr2=0x5610dfbd10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69a7e00640 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfbd10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69a7e00640 1 -- 172.21.15.131:0/3056524236 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610dfa75cb0 con 0x5610dfb577d0 2024-04-26T20:25:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69ade00640 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb56de0 0x5610dfb654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69ae800640 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfbd10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69a7e00640 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb65a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f699802a9e0 tx=0x7f69980039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dfb7d335bc5f8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f6993400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f6993400640 1 -- 172.21.15.131:0/3056524236 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998035030 con 0x5610dfb577d0 2024-04-26T20:25:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f6993400640 1 -- 172.21.15.131:0/3056524236 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f69980315a0 con 0x5610dfb577d0 2024-04-26T20:25:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.386+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a009d410 con 0x5610dfb577d0 2024-04-26T20:25:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f6993400640 1 -- 172.21.15.131:0/3056524236 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6998054050 con 0x5610dfb577d0 2024-04-26T20:25:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 10 client.26522 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610dfbd2740 con 0x5610dfb577d0 2024-04-26T20:25:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f6993400640 1 -- 172.21.15.131:0/3056524236 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a009d410 con 0x5610dfb577d0 2024-04-26T20:25:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f6993400640 1 -- 172.21.15.131:0/3056524236 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f699804f030 con 0x5610dfb577d0 2024-04-26T20:25:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 10 client.26522 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 10 client.26522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 20 client.26522 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfbd3170 0x5610dfbd5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 4 client.26522 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.387+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610dfac2e00 con 0x5610dfbd3170 2024-04-26T20:25:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.388+0000 7f69ade00640 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfbd3170 0x5610dfbd5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.388+0000 7f6993400640 10 client.26522 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.388+0000 7f69ade00640 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfbd3170 0x5610dfbd5590 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5710ce76c4878a7 server_cookie=a20da6dbd457831 in_seq=0 out_seq=0 2024-04-26T20:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f6993400640 1 -- 172.21.15.131:0/3056524236 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610dfac2e00 con 0x5610dfbd3170 2024-04-26T20:25:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f6993400640 10 client.26522 handle_command_reply: tid=0 2024-04-26T20:25:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 client.26522 shutdown 2024-04-26T20:25:45.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 msgr2=0x5610dfb65a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb65a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f699802a9e0 tx=0x7f69980039e0 comp rx=0 tx=0).stop 2024-04-26T20:25:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 shutdown_connections 2024-04-26T20:25:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5610dfbd3170 0x5610dfbd5590 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610dfb5c940 0x5610dfbd10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610dfb577d0 0x5610dfb65a00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 --2- 172.21.15.131:0/3056524236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610dfb56de0 0x5610dfb654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f6993400640 0 client.26522 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 >> 172.21.15.131:0/3056524236 conn(0x5610dfb52450 msgr2=0x5610dfb60f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 shutdown_connections 2024-04-26T20:25:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 1 -- 172.21.15.131:0/3056524236 wait complete. 2024-04-26T20:25:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.392+0000 7f69b7a58740 20 client.26522 trim_cache size 0 max 16384 2024-04-26T20:25:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69940a3c30 msgr2=0x7f69940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69940a3c30 0x7f69940a60f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69a80099a0 tx=0x7f69a8009500 comp rx=0 tx=0).stop 2024-04-26T20:25:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01016d0 msgr2=0x7f69b019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01016d0 0x7f69b019ac70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69a000e9c0 tx=0x7f69a000ee80 comp rx=0 tx=0).stop 2024-04-26T20:25:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 shutdown_connections 2024-04-26T20:25:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0103280 0x7f69b01a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f69940a3c30 0x7f69940a60f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69b01028c0 0x7f69b019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 --2- 172.21.15.131:0/1316250845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69b01016d0 0x7f69b019ac70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 >> 172.21.15.131:0/1316250845 conn(0x7f69b00fcc60 msgr2=0x7f69b010a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.393+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 shutdown_connections 2024-04-26T20:25:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:32.394+0000 7f69af200640 1 -- 172.21.15.131:0/1316250845 wait complete. 2024-04-26T20:25:46.292 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:33 smithi131 ceph-mon[27919]: pgmap v876: 129 pgs: 129 active+clean; 701 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:46.347 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:35 smithi131 ceph-mon[27919]: pgmap v877: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:25:46.359 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:43 smithi157 ceph-mon[32703]: pgmap v881: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:46.368 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:45 smithi157 ceph-mon[32703]: pgmap v882: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:46.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:43 smithi148 ceph-mon[32281]: pgmap v881: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:46.384 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:45 smithi148 ceph-mon[32281]: pgmap v882: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:46.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 -- 172.21.15.131:0/2726711784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c103a30 msgr2=0x7f370c103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:46.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 --2- 172.21.15.131:0/2726711784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c103a30 0x7f370c103eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009860 tx=0x7f36f802a340 comp rx=0 tx=0).stop 2024-04-26T20:25:46.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 -- 172.21.15.131:0/2726711784 shutdown_connections 2024-04-26T20:25:46.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 --2- 172.21.15.131:0/2726711784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f370c1043f0 0x7f370c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 --2- 172.21.15.131:0/2726711784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c103a30 0x7f370c103eb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 --2- 172.21.15.131:0/2726711784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c102840 0x7f370c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 -- 172.21.15.131:0/2726711784 >> 172.21.15.131:0/2726711784 conn(0x7f370c0fddd0 msgr2=0x7f370c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:46.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 -- 172.21.15.131:0/2726711784 shutdown_connections 2024-04-26T20:25:46.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.131+0000 7f3713000640 1 -- 172.21.15.131:0/2726711784 wait complete. 2024-04-26T20:25:46.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 Processor -- start 2024-04-26T20:25:46.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 -- start start 2024-04-26T20:25:46.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f370c102840 0x7f370c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c103a30 0x7f370c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:46.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c1043f0 0x7f370c1a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:46.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10e370 con 0x7f370c103a30 2024-04-26T20:25:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10e1f0 con 0x7f370c1043f0 2024-04-26T20:25:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3713000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10e4f0 con 0x7f370c102840 2024-04-26T20:25:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c103a30 0x7f370c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c103a30 0x7f370c19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52750/0 (socket says 172.21.15.131:52750) 2024-04-26T20:25:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.133+0000 7f3711200640 1 -- 172.21.15.131:0/2131653897 learned_addr learned my addr 172.21.15.131:0/2131653897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711c00640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f370c102840 0x7f370c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:46.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3712600640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c1043f0 0x7f370c1a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:46.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711200640 1 -- 172.21.15.131:0/2131653897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f370c102840 msgr2=0x7f370c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:46.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711200640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f370c102840 0x7f370c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711200640 1 -- 172.21.15.131:0/2131653897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c1043f0 msgr2=0x7f370c1a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:46.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711200640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c1043f0 0x7f370c1a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711200640 1 -- 172.21.15.131:0/2131653897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c1a28c0 con 0x7f370c103a30 2024-04-26T20:25:46.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711c00640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f370c102840 0x7f370c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:46.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.134+0000 7f3711200640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c103a30 0x7f370c19b1c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005940 tx=0x7f36f802ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:47.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.135+0000 7f370a000640 1 -- 172.21.15.131:0/2131653897 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8040020 con 0x7f370c103a30 2024-04-26T20:25:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.135+0000 7f370a000640 1 -- 172.21.15.131:0/2131653897 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f8030e50 con 0x7f370c103a30 2024-04-26T20:25:47.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.135+0000 7f370a000640 1 -- 172.21.15.131:0/2131653897 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8037040 con 0x7f370c103a30 2024-04-26T20:25:47.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.135+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c1a2b50 con 0x7f370c103a30 2024-04-26T20:25:47.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.135+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c1a2ff0 con 0x7f370c103a30 2024-04-26T20:25:47.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 Processor -- start 2024-04-26T20:25:47.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 -- start start 2024-04-26T20:25:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f370a000640 1 -- 172.21.15.131:0/2131653897 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f36f8043020 con 0x7f370c103a30 2024-04-26T20:25:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b760299470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:47.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299a40 0x55b76029e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:47.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b76029ebb0 0x55b7602a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:47.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b7601fcc70 con 0x55b76029ebb0 2024-04-26T20:25:47.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b7601b5980 con 0x55b760299a40 2024-04-26T20:25:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.139+0000 7f3717453740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7601f8c30 con 0x55b760299050 2024-04-26T20:25:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b76029ebb0 0x55b7602a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b76029ebb0 0x55b7602a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52758/0 (socket says 172.21.15.131:52758) 2024-04-26T20:25:47.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299a40 0x55b76029e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:47.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 -- 172.21.15.131:0/3426375617 learned_addr learned my addr 172.21.15.131:0/3426375617 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:47.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3712600640 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b760299470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:47.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 -- 172.21.15.131:0/3426375617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 msgr2=0x55b760299470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:47.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b760299470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:47.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 -- 172.21.15.131:0/3426375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299a40 msgr2=0x55b76029e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:47.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299a40 0x55b76029e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 -- 172.21.15.131:0/3426375617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b76018d7b0 con 0x55b76029ebb0 2024-04-26T20:25:47.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711c00640 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299a40 0x55b76029e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:47.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.140+0000 7f3711200640 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b76029ebb0 0x55b7602a1040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f36f80f64e0 tx=0x7f36f80f65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622b695e17793c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.141+0000 7f370a000640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36f40a3cc0 0x7f36f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:47.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.141+0000 7f36fea00640 1 -- 172.21.15.131:0/3426375617 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80bb7c0 con 0x55b76029ebb0 2024-04-26T20:25:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.141+0000 7f36fea00640 1 -- 172.21.15.131:0/3426375617 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f80fd030 con 0x55b76029ebb0 2024-04-26T20:25:47.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.141+0000 7f36fea00640 1 -- 172.21.15.131:0/3426375617 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80bbaf0 con 0x55b76029ebb0 2024-04-26T20:25:47.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.141+0000 7f370a000640 1 -- 172.21.15.131:0/2131653897 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f36f80f0530 con 0x7f370c103a30 2024-04-26T20:25:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.141+0000 7f3712600640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36f40a3cc0 0x7f36f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:47.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 -- 172.21.15.131:0/3426375617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b76029ebb0 msgr2=0x55b7602a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b76029ebb0 0x55b7602a1040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f36f80f64e0 tx=0x7f36f80f65d0 comp rx=0 tx=0).stop 2024-04-26T20:25:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3712600640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36f40a3cc0 0x7f36f40a6180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55b76029a2a0 tx=0x7f3704009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:47.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 -- 172.21.15.131:0/3426375617 shutdown_connections 2024-04-26T20:25:47.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b76029ebb0 0x55b7602a1040 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:47.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299a40 0x55b76029e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:47.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 --2- 172.21.15.131:0/3426375617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b760299470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:47.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 -- 172.21.15.131:0/3426375617 >> 172.21.15.131:0/3426375617 conn(0x55b7602946c0 msgr2=0x55b760294ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:47.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 -- 172.21.15.131:0/3426375617 shutdown_connections 2024-04-26T20:25:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 -- 172.21.15.131:0/3426375617 wait complete. 2024-04-26T20:25:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.142+0000 7f3717453740 1 -- start start 2024-04-26T20:25:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 1 Processor -- start 2024-04-26T20:25:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:47.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b7602aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b7601fbf80 con 0x55b760299a40 2024-04-26T20:25:47.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b7601a9260 con 0x55b76029ebb0 2024-04-26T20:25:47.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7602012a0 con 0x55b760299050 2024-04-26T20:25:47.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3717453740 10 client.0 _ll_register_callbacks cb 0x55b7601f3960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:47.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:47.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52770/0 (socket says 172.21.15.131:52770) 2024-04-26T20:25:47.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3711200640 1 -- 172.21.15.131:0/1621894582 learned_addr learned my addr 172.21.15.131:0/1621894582 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:47.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3712600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b7602aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:47.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.143+0000 7f3711c00640 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:47.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3712600640 1 -- 172.21.15.131:0/1621894582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 msgr2=0x55b7602a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:48.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3712600640 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:48.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3712600640 1 -- 172.21.15.131:0/1621894582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 msgr2=0x55b7602a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3712600640 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3712600640 1 -- 172.21.15.131:0/1621894582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b76018b1c0 con 0x55b76029ebb0 2024-04-26T20:25:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3711200640 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3712600640 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b7602aa1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3704005fd0 tx=0x7f3704004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955ed4c3344befc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:48.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f36f3400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:25:48.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f36f3400640 1 -- 172.21.15.131:0/1621894582 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370400f870 con 0x55b76029ebb0 2024-04-26T20:25:48.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f80fd030 con 0x55b76029ebb0 2024-04-26T20:25:48.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.144+0000 7f36f3400640 1 -- 172.21.15.131:0/1621894582 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f370400fa10 con 0x55b76029ebb0 2024-04-26T20:25:48.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f36f3400640 1 -- 172.21.15.131:0/1621894582 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f370400fc10 con 0x55b76029ebb0 2024-04-26T20:25:48.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 10 client.26534 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:48.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b75ff0e3f0 con 0x55b76029ebb0 2024-04-26T20:25:48.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f36f3400640 1 -- 172.21.15.131:0/1621894582 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f80fd030 con 0x55b76029ebb0 2024-04-26T20:25:48.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f36f3400640 1 -- 172.21.15.131:0/1621894582 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3704026070 con 0x55b76029ebb0 2024-04-26T20:25:48.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 10 client.26534 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:48.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:48.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 20 client.26534 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b7602ae770 0x55b7602b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:48.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 4 client.26534 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:48.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.145+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b76020a880 con 0x55b7602ae770 2024-04-26T20:25:48.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.146+0000 7f3711c00640 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b7602ae770 0x55b7602b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:48.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.146+0000 7f3711c00640 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b7602ae770 0x55b7602b0bf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a5ac375304876d server_cookie=6d46f0fbe3d4552e in_seq=0 out_seq=0 2024-04-26T20:25:48.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.146+0000 7f36f3400640 10 client.26534 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.147+0000 7f36f3400640 1 -- 172.21.15.131:0/1621894582 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b76020a880 con 0x55b7602ae770 2024-04-26T20:25:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.147+0000 7f36f3400640 10 client.26534 handle_command_reply: tid=0 2024-04-26T20:25:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.147+0000 7f3717453740 1 client.26534 shutdown 2024-04-26T20:25:48.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.147+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 msgr2=0x55b7602aa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:48.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.147+0000 7f3717453740 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b7602aa1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3704005fd0 tx=0x7f3704004ab0 comp rx=0 tx=0).stop 2024-04-26T20:25:48.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 shutdown_connections 2024-04-26T20:25:48.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b7602ae770 0x55b7602b0bf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:48.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b7602aa1a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:48.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:48.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 --2- 172.21.15.131:0/1621894582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:48.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f36f3400640 0 client.26534 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:48.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 >> 172.21.15.131:0/1621894582 conn(0x55b7602946c0 msgr2=0x55b7602937b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:48.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 shutdown_connections 2024-04-26T20:25:48.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 1 -- 172.21.15.131:0/1621894582 wait complete. 2024-04-26T20:25:48.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.148+0000 7f3717453740 20 client.26534 trim_cache size 0 max 16384 2024-04-26T20:25:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.162+0000 7f3717453740 1 Processor -- start 2024-04-26T20:25:48.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.162+0000 7f3717453740 1 -- start start 2024-04-26T20:25:48.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3717453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3717453740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602aa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:48.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3717453740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b760297b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:48.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3717453740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1aff80 con 0x55b760299a40 2024-04-26T20:25:48.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3717453740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1ad850 con 0x55b76029ebb0 2024-04-26T20:25:48.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3717453740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1ab6a0 con 0x55b760299050 2024-04-26T20:25:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602aa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52776/0 (socket says 172.21.15.131:52776) 2024-04-26T20:25:48.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 -- 172.21.15.131:0/1331470417 learned_addr learned my addr 172.21.15.131:0/1331470417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:48.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711c00640 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3712600640 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b760297b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 -- 172.21.15.131:0/1331470417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 msgr2=0x55b7602975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:49.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 -- 172.21.15.131:0/1331470417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 msgr2=0x55b760297b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b760297b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.163+0000 7f3711200640 1 -- 172.21.15.131:0/1331470417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75ffd87d0 con 0x55b760299a40 2024-04-26T20:25:49.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.164+0000 7f3711200640 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602aa160 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f36f8004650 tx=0x7f36f8033f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe79336396d0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.164+0000 7f36fe000640 1 -- 172.21.15.131:0/1331470417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80bbcd0 con 0x55b760299a40 2024-04-26T20:25:49.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.164+0000 7f36fe000640 1 -- 172.21.15.131:0/1331470417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f80fd070 con 0x55b760299a40 2024-04-26T20:25:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.164+0000 7f36fe000640 1 -- 172.21.15.131:0/1331470417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80bbeb0 con 0x55b760299a40 2024-04-26T20:25:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 -- 172.21.15.131:0/1331470417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 msgr2=0x55b7602aa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602aa160 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f36f8004650 tx=0x7f36f8033f90 comp rx=0 tx=0).stop 2024-04-26T20:25:49.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 -- 172.21.15.131:0/1331470417 shutdown_connections 2024-04-26T20:25:49.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b76029ebb0 0x55b760297b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602aa160 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 --2- 172.21.15.131:0/1331470417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b760299050 0x55b7602975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 -- 172.21.15.131:0/1331470417 >> 172.21.15.131:0/1331470417 conn(0x55b7602946c0 msgr2=0x55b7602937b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:49.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 -- 172.21.15.131:0/1331470417 shutdown_connections 2024-04-26T20:25:49.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 -- 172.21.15.131:0/1331470417 wait complete. 2024-04-26T20:25:49.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 -- start start 2024-04-26T20:25:49.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.165+0000 7f3717453740 1 Processor -- start 2024-04-26T20:25:49.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3717453740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299050 0x55b7602a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3717453740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3717453740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b76029ebb0 0x55b7602a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3717453740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1abd10 con 0x55b760299a40 2024-04-26T20:25:49.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3717453740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1ad180 con 0x55b760299050 2024-04-26T20:25:49.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3717453740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1ac3e0 con 0x55b76029ebb0 2024-04-26T20:25:49.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3717453740 10 client.0 _ll_register_callbacks cb 0x7f3704009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:49.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52784/0 (socket says 172.21.15.131:52784) 2024-04-26T20:25:49.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 -- 172.21.15.131:0/1974030540 learned_addr learned my addr 172.21.15.131:0/1974030540 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:49.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711c00640 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299050 0x55b7602a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3712600640 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b76029ebb0 0x55b7602a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 -- 172.21.15.131:0/1974030540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b76029ebb0 msgr2=0x55b7602a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:49.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b76029ebb0 0x55b7602a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 -- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299050 msgr2=0x55b7602a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299050 0x55b7602a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.166+0000 7f3711200640 1 -- 172.21.15.131:0/1974030540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7601a32a0 con 0x55b760299a40 2024-04-26T20:25:49.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f3711200640 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a7b10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f36f80c4ab0 tx=0x7f36f80f0830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8833e2996bf66420 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f36f3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:49.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f36f3400640 1 -- 172.21.15.131:0/1974030540 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80d5060 con 0x55b760299a40 2024-04-26T20:25:49.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f36f3400640 1 -- 172.21.15.131:0/1974030540 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f36f80ba670 con 0x55b760299a40 2024-04-26T20:25:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f80fd070 con 0x55b760299a40 2024-04-26T20:25:49.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f36f3400640 1 -- 172.21.15.131:0/1974030540 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36f80bc140 con 0x55b760299a40 2024-04-26T20:25:49.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f3717453740 10 client.16425 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:49.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7602aa240 con 0x55b760299a40 2024-04-26T20:25:49.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.167+0000 7f36f3400640 1 -- 172.21.15.131:0/1974030540 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f80fd070 con 0x55b760299a40 2024-04-26T20:25:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f36f3400640 1 -- 172.21.15.131:0/1974030540 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f36f804d060 con 0x55b760299a40 2024-04-26T20:25:49.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f3717453740 10 client.16425 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:49.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f3717453740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:49.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f3717453740 20 client.16425 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:50.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f3717453740 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b76031d5e0 0x55b76031f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f3717453740 4 client.16425 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:25:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7602a2a10 con 0x55b76031d5e0 2024-04-26T20:25:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.168+0000 7f3711c00640 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b76031d5e0 0x55b76031f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3711c00640 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b76031d5e0 0x55b76031f9d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1eeca38809ba8d server_cookie=938cc972b1ca357d in_seq=0 out_seq=0 2024-04-26T20:25:50.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f36f3400640 10 client.16425 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:50.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f36f3400640 1 -- 172.21.15.131:0/1974030540 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7602a2a10 con 0x55b76031d5e0 2024-04-26T20:25:50.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f36f3400640 10 client.16425 handle_command_reply: tid=0 2024-04-26T20:25:50.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 client.16425 shutdown 2024-04-26T20:25:50.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 msgr2=0x55b7602a7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a7b10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f36f80c4ab0 tx=0x7f36f80f0830 comp rx=0 tx=0).stop 2024-04-26T20:25:50.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 shutdown_connections 2024-04-26T20:25:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b76031d5e0 0x55b76031f9d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b76029ebb0 0x55b7602a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b760299a40 0x55b7602a7b10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f36f3400640 0 client.16425 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:50.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 --2- 172.21.15.131:0/1974030540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b760299050 0x55b7602a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 >> 172.21.15.131:0/1974030540 conn(0x55b7602946c0 msgr2=0x55b76029a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:50.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 shutdown_connections 2024-04-26T20:25:50.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.169+0000 7f3717453740 1 -- 172.21.15.131:0/1974030540 wait complete. 2024-04-26T20:25:50.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3717453740 20 client.16425 trim_cache size 0 max 16384 2024-04-26T20:25:50.470 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:25:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36f40a3cc0 msgr2=0x7f36f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36f40a3cc0 0x7f36f40a6180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55b76029a2a0 tx=0x7f3704009500 comp rx=0 tx=0).stop 2024-04-26T20:25:50.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c103a30 msgr2=0x7f370c19b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c103a30 0x7f370c19b1c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005940 tx=0x7f36f802ae20 comp rx=0 tx=0).stop 2024-04-26T20:25:50.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 shutdown_connections 2024-04-26T20:25:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f370c1043f0 0x7f370c1a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c103a30 0x7f370c19b1c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f36f40a3cc0 0x7f36f40a6180 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 --2- 172.21.15.131:0/2131653897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f370c102840 0x7f370c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 >> 172.21.15.131:0/2131653897 conn(0x7f370c0fddd0 msgr2=0x7f370c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:50.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 shutdown_connections 2024-04-26T20:25:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:38.170+0000 7f3713000640 1 -- 172.21.15.131:0/2131653897 wait complete. 2024-04-26T20:25:50.765 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:37 smithi131 ceph-mon[27919]: pgmap v878: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:50.774 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:39 smithi131 ceph-mon[27919]: pgmap v879: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:50.781 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:41 smithi131 ceph-mon[27919]: pgmap v880: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:50.788 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:43 smithi131 ceph-mon[27919]: pgmap v881: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:50.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:47 smithi148 ceph-mon[32281]: pgmap v883: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:50.803 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:48 smithi148 ceph-mon[32281]: pgmap v884: 129 pgs: 129 active+clean; 703 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:50.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:47 smithi157 ceph-mon[32703]: pgmap v883: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:50.910 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:48 smithi157 ceph-mon[32703]: pgmap v884: 129 pgs: 129 active+clean; 703 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 -- 172.21.15.131:0/1724478526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98075390 msgr2=0x7f1a98075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1724478526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98075390 0x7f1a98075810 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009930 tx=0x7f1a8002a440 comp rx=0 tx=0).stop 2024-04-26T20:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 -- 172.21.15.131:0/1724478526 shutdown_connections 2024-04-26T20:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1724478526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9810eda0 0x7f1a98111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1724478526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98075390 0x7f1a98075810 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1724478526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a98076970 0x7f1a98074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 -- 172.21.15.131:0/1724478526 >> 172.21.15.131:0/1724478526 conn(0x7f1a980fdd40 msgr2=0x7f1a98100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 -- 172.21.15.131:0/1724478526 shutdown_connections 2024-04-26T20:25:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.537+0000 7f1a97e00640 1 -- 172.21.15.131:0/1724478526 wait complete. 2024-04-26T20:25:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.538+0000 7f1a97e00640 1 Processor -- start 2024-04-26T20:25:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a97e00640 1 -- start start 2024-04-26T20:25:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a98075390 0x7f1a9819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98076970 0x7f1a9819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9810eda0 0x7f1a981a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a97e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981148f0 con 0x7f1a98076970 2024-04-26T20:25:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a97e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98114770 con 0x7f1a9810eda0 2024-04-26T20:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a97e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98114a70 con 0x7f1a98075390 2024-04-26T20:25:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98076970 0x7f1a9819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98076970 0x7f1a9819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38230/0 (socket says 172.21.15.131:38230) 2024-04-26T20:25:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a8fe00640 1 -- 172.21.15.131:0/1552660866 learned_addr learned my addr 172.21.15.131:0/1552660866 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.539+0000 7f1a96a00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a98075390 0x7f1a9819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a97400640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9810eda0 0x7f1a981a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a8fe00640 1 -- 172.21.15.131:0/1552660866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a98075390 msgr2=0x7f1a9819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a98075390 0x7f1a9819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a8fe00640 1 -- 172.21.15.131:0/1552660866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9810eda0 msgr2=0x7f1a981a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9810eda0 0x7f1a981a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a8fe00640 1 -- 172.21.15.131:0/1552660866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a981a6cb0 con 0x7f1a98076970 2024-04-26T20:25:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a96a00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a98075390 0x7f1a9819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a97400640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9810eda0 0x7f1a981a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.540+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98076970 0x7f1a9819f670 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009a50 tx=0x7f1a800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.541+0000 7f1a8f400640 1 -- 172.21.15.131:0/1552660866 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8003f030 con 0x7f1a98076970 2024-04-26T20:25:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.541+0000 7f1a8f400640 1 -- 172.21.15.131:0/1552660866 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a80031910 con 0x7f1a98076970 2024-04-26T20:25:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.541+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a981a6f40 con 0x7f1a98076970 2024-04-26T20:25:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.541+0000 7f1a8f400640 1 -- 172.21.15.131:0/1552660866 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80035070 con 0x7f1a98076970 2024-04-26T20:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.541+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a981a73a0 con 0x7f1a98076970 2024-04-26T20:25:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.545+0000 7f1aa058b740 1 Processor -- start 2024-04-26T20:25:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.545+0000 7f1aa058b740 1 -- start start 2024-04-26T20:25:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.545+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6764d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764d7d0 0x562c676522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c67652940 0x562c67654dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1aa058b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562c675b1380 con 0x562c67652940 2024-04-26T20:25:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1aa058b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c6754dc30 con 0x562c6764d7d0 2024-04-26T20:25:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1aa058b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562c675ad340 con 0x562c6764cde0 2024-04-26T20:25:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a8f400640 1 -- 172.21.15.131:0/1552660866 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1a80042030 con 0x7f1a98076970 2024-04-26T20:25:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6764d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c67652940 0x562c67654dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6764d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40846/0 (socket says 172.21.15.131:40846) 2024-04-26T20:25:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 -- 172.21.15.131:0/2499040015 learned_addr learned my addr 172.21.15.131:0/2499040015 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764d7d0 0x562c676522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 -- 172.21.15.131:0/2499040015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764d7d0 msgr2=0x562c676522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764d7d0 0x562c676522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 -- 172.21.15.131:0/2499040015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c67652940 msgr2=0x562c67654dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c67652940 0x562c67654dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a97400640 1 -- 172.21.15.131:0/2499040015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6756f940 con 0x562c6764cde0 2024-04-26T20:25:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.546+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764d7d0 0x562c676522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.547+0000 7f1a97400640 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6764d200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1a90011810 tx=0x7f1a90011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5107180b0d5bede4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.547+0000 7f1a87e00640 1 -- 172.21.15.131:0/2499040015 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90011310 con 0x562c6764cde0 2024-04-26T20:25:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.547+0000 7f1a87e00640 1 -- 172.21.15.131:0/2499040015 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9000f9a0 con 0x562c6764cde0 2024-04-26T20:25:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.547+0000 7f1a87e00640 1 -- 172.21.15.131:0/2499040015 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a900055f0 con 0x562c6764cde0 2024-04-26T20:25:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.547+0000 7f1a8f400640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a7c0a3c30 0x7f1a7c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.547+0000 7f1a8f400640 1 -- 172.21.15.131:0/1552660866 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f1a800f0ec0 con 0x7f1a98076970 2024-04-26T20:25:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 -- 172.21.15.131:0/2499040015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 msgr2=0x562c6764d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6764d200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1a90011810 tx=0x7f1a90011cd0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1a96a00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a7c0a3c30 0x7f1a7c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 -- 172.21.15.131:0/2499040015 shutdown_connections 2024-04-26T20:25:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c67652940 0x562c67654dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764d7d0 0x562c676522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2499040015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6764d200 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 -- 172.21.15.131:0/2499040015 >> 172.21.15.131:0/2499040015 conn(0x562c67648450 msgr2=0x562c67648850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 -- 172.21.15.131:0/2499040015 shutdown_connections 2024-04-26T20:25:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 -- 172.21.15.131:0/2499040015 wait complete. 2024-04-26T20:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1aa058b740 1 -- start start 2024-04-26T20:25:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.548+0000 7f1a96a00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a7c0a3c30 0x7f1a7c0a60f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x562c676493c0 tx=0x7f1a88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 1 Processor -- start 2024-04-26T20:25:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c67656fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c67657500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562c676558f0 con 0x562c6764d7d0 2024-04-26T20:25:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c67538c60 con 0x562c67652940 2024-04-26T20:25:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562c67655a70 con 0x562c6764cde0 2024-04-26T20:25:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1aa058b740 10 client.0 _ll_register_callbacks cb 0x562c67564e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1a96a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c67656fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c67657500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1a96a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c67656fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40860/0 (socket says 172.21.15.131:40860) 2024-04-26T20:25:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.549+0000 7f1a96a00640 1 -- 172.21.15.131:0/2434991252 learned_addr learned my addr 172.21.15.131:0/2434991252 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a97400640 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a96a00640 1 -- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 msgr2=0x562c6765e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a96a00640 1 -- 172.21.15.131:0/2434991252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 msgr2=0x562c67657500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c67657500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a96a00640 1 -- 172.21.15.131:0/2434991252 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c67547510 con 0x562c6764cde0 2024-04-26T20:25:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a97400640 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c67656fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a8801eab0 tx=0x7f1a88004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec853a928babc62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a84c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:25:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1a84c00640 1 -- 172.21.15.131:0/2434991252 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8802e060 con 0x562c6764cde0 2024-04-26T20:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.550+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a9000f9a0 con 0x562c6764cde0 2024-04-26T20:25:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.551+0000 7f1a84c00640 1 -- 172.21.15.131:0/2434991252 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f1a88018030 con 0x562c6764cde0 2024-04-26T20:25:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.551+0000 7f1a84c00640 1 -- 172.21.15.131:0/2434991252 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a88035050 con 0x562c6764cde0 2024-04-26T20:25:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.551+0000 7f1aa058b740 10 client.26530 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.551+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c672c3210 con 0x562c6764cde0 2024-04-26T20:25:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.551+0000 7f1a84c00640 1 -- 172.21.15.131:0/2434991252 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9000f9a0 con 0x562c6764cde0 2024-04-26T20:25:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.551+0000 7f1a84c00640 1 -- 172.21.15.131:0/2434991252 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1a8801c740 con 0x562c6764cde0 2024-04-26T20:25:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.552+0000 7f1aa058b740 10 client.26530 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.552+0000 7f1aa058b740 10 client.26530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.552+0000 7f1aa058b740 20 client.26530 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.552+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c67662af0 0x562c67664f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.552+0000 7f1aa058b740 4 client.26530 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.552+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c675bef90 con 0x562c67662af0 2024-04-26T20:25:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.552+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c67662af0 0x562c67664f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.553+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c67662af0 0x562c67664f80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a6d7ece2b8aaff4 server_cookie=dc895ea00b24acf4 in_seq=0 out_seq=0 2024-04-26T20:25:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.553+0000 7f1a84c00640 10 client.26530 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1a84c00640 1 -- 172.21.15.131:0/2434991252 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c675bef90 con 0x562c67662af0 2024-04-26T20:25:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1a84c00640 10 client.26530 handle_command_reply: tid=0 2024-04-26T20:25:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 client.26530 shutdown 2024-04-26T20:25:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 msgr2=0x562c67656fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c67656fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a8801eab0 tx=0x7f1a88004730 comp rx=0 tx=0).stop 2024-04-26T20:25:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 shutdown_connections 2024-04-26T20:25:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c67662af0 0x562c67664f80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c67657500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2434991252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c67656fc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1a84c00640 0 client.26530 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 >> 172.21.15.131:0/2434991252 conn(0x562c67648450 msgr2=0x562c67647650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 shutdown_connections 2024-04-26T20:25:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 1 -- 172.21.15.131:0/2434991252 wait complete. 2024-04-26T20:25:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.554+0000 7f1aa058b740 20 client.26530 trim_cache size 0 max 16384 2024-04-26T20:25:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.570+0000 7f1aa058b740 1 Processor -- start 2024-04-26T20:25:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.570+0000 7f1aa058b740 1 -- start start 2024-04-26T20:25:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764cde0 0x562c67656bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c67652940 0x562c67657110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1aa058b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981b4370 con 0x562c6764d7d0 2024-04-26T20:25:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1aa058b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981b1c40 con 0x562c6764cde0 2024-04-26T20:25:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1aa058b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981afa90 con 0x562c67652940 2024-04-26T20:25:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1a96a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764cde0 0x562c67656bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1a96a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764cde0 0x562c67656bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55446/0 (socket says 172.21.15.131:55446) 2024-04-26T20:25:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1a96a00640 1 -- 172.21.15.131:0/1549341837 learned_addr learned my addr 172.21.15.131:0/1549341837 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1a97400640 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c67652940 0x562c67657110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.571+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a96a00640 1 -- 172.21.15.131:0/1549341837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c67652940 msgr2=0x562c67657110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a96a00640 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c67652940 0x562c67657110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a96a00640 1 -- 172.21.15.131:0/1549341837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 msgr2=0x562c6765e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a96a00640 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a96a00640 1 -- 172.21.15.131:0/1549341837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6738f620 con 0x562c6764cde0 2024-04-26T20:25:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a96a00640 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764cde0 0x562c67656bd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a88019040 tx=0x7f1a88009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9a2851b13366c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a87400640 1 -- 172.21.15.131:0/1549341837 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8801ce50 con 0x562c6764cde0 2024-04-26T20:25:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a87400640 1 -- 172.21.15.131:0/1549341837 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a8803e070 con 0x562c6764cde0 2024-04-26T20:25:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.572+0000 7f1a87400640 1 -- 172.21.15.131:0/1549341837 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88005040 con 0x562c6764cde0 2024-04-26T20:25:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 -- 172.21.15.131:0/1549341837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764cde0 msgr2=0x562c67656bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764cde0 0x562c67656bd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a88019040 tx=0x7f1a88009e30 comp rx=0 tx=0).stop 2024-04-26T20:25:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 -- 172.21.15.131:0/1549341837 shutdown_connections 2024-04-26T20:25:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c67652940 0x562c67657110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 --2- 172.21.15.131:0/1549341837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c6764cde0 0x562c67656bd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 -- 172.21.15.131:0/1549341837 >> 172.21.15.131:0/1549341837 conn(0x562c67648450 msgr2=0x562c67651f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 -- 172.21.15.131:0/1549341837 shutdown_connections 2024-04-26T20:25:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 -- 172.21.15.131:0/1549341837 wait complete. 2024-04-26T20:25:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.573+0000 7f1aa058b740 1 -- start start 2024-04-26T20:25:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 1 Processor -- start 2024-04-26T20:25:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6765ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981b0100 con 0x562c6764d7d0 2024-04-26T20:25:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981b1570 con 0x562c67652940 2024-04-26T20:25:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981b07d0 con 0x562c6764cde0 2024-04-26T20:25:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1aa058b740 10 client.0 _ll_register_callbacks cb 0x7f1a8801e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38256/0 (socket says 172.21.15.131:38256) 2024-04-26T20:25:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1a8fe00640 1 -- 172.21.15.131:0/2391782398 learned_addr learned my addr 172.21.15.131:0/2391782398 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1a97400640 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.574+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6765ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a8fe00640 1 -- 172.21.15.131:0/2391782398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 msgr2=0x562c6765ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6765ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a8fe00640 1 -- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 msgr2=0x562c6765cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a8fe00640 1 -- 172.21.15.131:0/2391782398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6756bcb0 con 0x562c6764d7d0 2024-04-26T20:25:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a8fe00640 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765bf70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1a800dd010 tx=0x7f1a800dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ab5c2a35b59b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a84c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a84c00640 1 -- 172.21.15.131:0/2391782398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80049040 con 0x562c6764d7d0 2024-04-26T20:25:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a84c00640 1 -- 172.21.15.131:0/2391782398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f1a800ee670 con 0x562c6764d7d0 2024-04-26T20:25:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8803e070 con 0x562c6764d7d0 2024-04-26T20:25:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1a84c00640 1 -- 172.21.15.131:0/2391782398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a800044e0 con 0x562c6764d7d0 2024-04-26T20:25:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1aa058b740 10 client.16443 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.575+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c6765e6e0 con 0x562c6764d7d0 2024-04-26T20:25:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1a84c00640 1 -- 172.21.15.131:0/2391782398 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a8803e070 con 0x562c6764d7d0 2024-04-26T20:25:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1a84c00640 1 -- 172.21.15.131:0/2391782398 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1a800dd0b0 con 0x562c6764d7d0 2024-04-26T20:25:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1aa058b740 10 client.16443 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1aa058b740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1aa058b740 20 client.16443 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c676c96d0 0x562c676cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1aa058b740 4 client.16443 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:25:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c675b8e00 con 0x562c676c96d0 2024-04-26T20:25:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.576+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c676c96d0 0x562c676cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.577+0000 7f1a96a00640 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c676c96d0 0x562c676cbaf0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c307b69fd2b1c125 server_cookie=2a9e930c77169621 in_seq=0 out_seq=0 2024-04-26T20:25:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.577+0000 7f1a84c00640 10 client.16443 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1a84c00640 1 -- 172.21.15.131:0/2391782398 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c675b8e00 con 0x562c676c96d0 2024-04-26T20:25:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1a84c00640 10 client.16443 handle_command_reply: tid=0 2024-04-26T20:25:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 client.16443 shutdown 2024-04-26T20:25:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 msgr2=0x562c6765bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765bf70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1a800dd010 tx=0x7f1a800dd040 comp rx=0 tx=0).stop 2024-04-26T20:25:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 shutdown_connections 2024-04-26T20:25:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x562c676c96d0 0x562c676cbaf0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c67652940 0x562c6765cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c6764d7d0 0x562c6765bf70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1a84c00640 0 client.16443 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 --2- 172.21.15.131:0/2391782398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562c6764cde0 0x562c6765ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 >> 172.21.15.131:0/2391782398 conn(0x562c67648450 msgr2=0x562c67656bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 shutdown_connections 2024-04-26T20:25:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 1 -- 172.21.15.131:0/2391782398 wait complete. 2024-04-26T20:25:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.581+0000 7f1aa058b740 20 client.16443 trim_cache size 0 max 16384 2024-04-26T20:25:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:25:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:25:55.512 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T20:25:55.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:25:55.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T20:25:55.535 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:51 smithi148 ceph-mon[32281]: pgmap v885: 129 pgs: 129 active+clean; 703 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:55.543 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:53 smithi148 ceph-mon[32281]: pgmap v886: 129 pgs: 129 active+clean; 703 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:55.589 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:51 smithi157 ceph-mon[32703]: pgmap v885: 129 pgs: 129 active+clean; 703 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:55.640 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:53 smithi157 ceph-mon[32703]: pgmap v886: 129 pgs: 129 active+clean; 703 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:55.648 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:55 smithi157 ceph-mon[32703]: pgmap v887: 129 pgs: 129 active+clean; 13 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.0 MiB/s wr, 7 op/s 2024-04-26T20:25:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a7c0a3c30 msgr2=0x7f1a7c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a7c0a3c30 0x7f1a7c0a60f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x562c676493c0 tx=0x7f1a88005f90 comp rx=0 tx=0).stop 2024-04-26T20:25:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98076970 msgr2=0x7f1a9819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98076970 0x7f1a9819f670 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009a50 tx=0x7f1a800042f0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 shutdown_connections 2024-04-26T20:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9810eda0 0x7f1a981a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98076970 0x7f1a9819f670 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f1a7c0a3c30 0x7f1a7c0a60f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 --2- 172.21.15.131:0/1552660866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a98075390 0x7f1a9819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 >> 172.21.15.131:0/1552660866 conn(0x7f1a980fdd40 msgr2=0x7f1a98102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 shutdown_connections 2024-04-26T20:25:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:45.582+0000 7f1a97e00640 1 -- 172.21.15.131:0/1552660866 wait complete. 2024-04-26T20:25:55.843 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:45 smithi131 ceph-mon[27919]: pgmap v882: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:55.852 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:47 smithi131 ceph-mon[27919]: pgmap v883: 129 pgs: 129 active+clean; 702 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T20:25:55.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:48 smithi131 ceph-mon[27919]: pgmap v884: 129 pgs: 129 active+clean; 703 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:25:55.946 INFO:tasks.workunit.client.0.smithi131.stdout:test large (16MB) dio write 2024-04-26T20:25:55.953 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test large (16MB) dio write' 2024-04-26T20:25:55.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-04-26T20:25:55.969 INFO:tasks.workunit.client.0.smithi131.stderr:1+0 records in 2024-04-26T20:25:56.036 INFO:tasks.workunit.client.0.smithi131.stderr:1+0 records out 2024-04-26T20:25:56.045 INFO:tasks.workunit.client.0.smithi131.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.301598 s, 55.6 MB/s 2024-04-26T20:25:56.054 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo OK 2024-04-26T20:25:56.065 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:55 smithi148 ceph-mon[32281]: pgmap v887: 129 pgs: 129 active+clean; 13 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.0 MiB/s wr, 7 op/s 2024-04-26T20:25:56.075 INFO:tasks.workunit.client.0.smithi131.stdout:OK 2024-04-26T20:25:56.084 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T20:25:56.152 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T20:25:56.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:51 smithi131 ceph-mon[27919]: pgmap v885: 129 pgs: 129 active+clean; 703 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:56.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:53 smithi131 ceph-mon[27919]: pgmap v886: 129 pgs: 129 active+clean; 703 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:25:56.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:55 smithi131 ceph-mon[27919]: pgmap v887: 129 pgs: 129 active+clean; 13 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.0 MiB/s wr, 7 op/s 2024-04-26T20:25:56.278 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-04-26T20:25:56.285 DEBUG:teuthology.orchestra.run.smithi131:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-04-26T20:25:56.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.382+0000 7fb79ec00640 1 -- 172.21.15.131:0/2012468566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798103a50 msgr2=0x7fb798103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.382+0000 7fb79ec00640 1 --2- 172.21.15.131:0/2012468566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798103a50 0x7fb798103ed0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb784009900 tx=0x7fb78402a410 comp rx=0 tx=0).stop 2024-04-26T20:25:56.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.382+0000 7fb79ec00640 1 -- 172.21.15.131:0/2012468566 shutdown_connections 2024-04-26T20:25:56.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.382+0000 7fb79ec00640 1 --2- 172.21.15.131:0/2012468566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798104410 0x7fb79810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.382+0000 7fb79ec00640 1 --2- 172.21.15.131:0/2012468566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798103a50 0x7fb798103ed0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.382+0000 7fb79ec00640 1 --2- 172.21.15.131:0/2012468566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb798102860 0x7fb798102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.382+0000 7fb79ec00640 1 -- 172.21.15.131:0/2012468566 >> 172.21.15.131:0/2012468566 conn(0x7fb7980fddd0 msgr2=0x7fb798100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.383+0000 7fb79ec00640 1 -- 172.21.15.131:0/2012468566 shutdown_connections 2024-04-26T20:25:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.383+0000 7fb79ec00640 1 -- 172.21.15.131:0/2012468566 wait complete. 2024-04-26T20:25:56.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 Processor -- start 2024-04-26T20:25:56.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 -- start start 2024-04-26T20:25:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798102860 0x7fb79819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb798103a50 0x7fb79819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798104410 0x7fb7981a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb79810e3e0 con 0x7fb798102860 2024-04-26T20:25:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb79810e260 con 0x7fb798103a50 2024-04-26T20:25:56.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb79810e560 con 0x7fb798104410 2024-04-26T20:25:56.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798102860 0x7fb79819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798102860 0x7fb79819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52722/0 (socket says 172.21.15.131:52722) 2024-04-26T20:25:56.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79d800640 1 -- 172.21.15.131:0/3925403441 learned_addr learned my addr 172.21.15.131:0/3925403441 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:56.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79e200640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798104410 0x7fb7981a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.384+0000 7fb79ce00640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb798103a50 0x7fb79819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb79d800640 1 -- 172.21.15.131:0/3925403441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798104410 msgr2=0x7fb7981a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb79d800640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798104410 0x7fb7981a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb79d800640 1 -- 172.21.15.131:0/3925403441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb798103a50 msgr2=0x7fb79819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb79d800640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb798103a50 0x7fb79819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb79d800640 1 -- 172.21.15.131:0/3925403441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7981a2900 con 0x7fb798102860 2024-04-26T20:25:56.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb79d800640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798102860 0x7fb79819acc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d20 tx=0x7fb78c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb796000640 1 -- 172.21.15.131:0/3925403441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c010630 con 0x7fb798102860 2024-04-26T20:25:56.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb796000640 1 -- 172.21.15.131:0/3925403441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb78c010c80 con 0x7fb798102860 2024-04-26T20:25:56.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.385+0000 7fb796000640 1 -- 172.21.15.131:0/3925403441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c018d50 con 0x7fb798102860 2024-04-26T20:25:56.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.386+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7981a2bf0 con 0x7fb798102860 2024-04-26T20:25:56.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.386+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7981a30c0 con 0x7fb798102860 2024-04-26T20:25:56.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 Processor -- start 2024-04-26T20:25:56.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 -- start start 2024-04-26T20:25:57.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f61e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f66f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f675d0 0x561bd8f69a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561bd8ec6980 con 0x561bd8f675d0 2024-04-26T20:25:57.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561bd8e63220 con 0x561bd8f61a70 2024-04-26T20:25:57.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561bd8ec2940 con 0x561bd8f62460 2024-04-26T20:25:57.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.389+0000 7fb796000640 1 -- 172.21.15.131:0/3925403441 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb78c01f020 con 0x7fb798102860 2024-04-26T20:25:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f61e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f61e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55350/0 (socket says 172.21.15.131:55350) 2024-04-26T20:25:57.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 -- 172.21.15.131:0/3830207147 learned_addr learned my addr 172.21.15.131:0/3830207147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79ce00640 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f675d0 0x561bd8f69a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79d800640 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f66f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 -- 172.21.15.131:0/3830207147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 msgr2=0x561bd8f66f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f66f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 -- 172.21.15.131:0/3830207147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f675d0 msgr2=0x561bd8f69a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f675d0 0x561bd8f69a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 -- 172.21.15.131:0/3830207147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd8ca07e0 con 0x561bd8f61a70 2024-04-26T20:25:57.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79ce00640 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f675d0 0x561bd8f69a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:57.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.390+0000 7fb79e200640 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f61e90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb790009ca0 tx=0x7fb790011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ad65480af4cd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:57.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.391+0000 7fb78aa00640 1 -- 172.21.15.131:0/3830207147 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79001e070 con 0x561bd8f61a70 2024-04-26T20:25:57.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.391+0000 7fb78aa00640 1 -- 172.21.15.131:0/3830207147 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb790009e50 con 0x561bd8f61a70 2024-04-26T20:25:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.391+0000 7fb78aa00640 1 -- 172.21.15.131:0/3830207147 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79000ea10 con 0x561bd8f61a70 2024-04-26T20:25:57.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.391+0000 7fb796000640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7800a3c40 0x7fb7800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb796000640 1 -- 172.21.15.131:0/3925403441 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb78c0d0e30 con 0x7fb798102860 2024-04-26T20:25:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb79d800640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7800a3c40 0x7fb7800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3830207147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 msgr2=0x561bd8f61e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f61e90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb790009ca0 tx=0x7fb790011810 comp rx=0 tx=0).stop 2024-04-26T20:25:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3830207147 shutdown_connections 2024-04-26T20:25:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f675d0 0x561bd8f69a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f66f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3830207147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f61e90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3830207147 >> 172.21.15.131:0/3830207147 conn(0x561bd8f5d0e0 msgr2=0x561bd8f5d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3830207147 shutdown_connections 2024-04-26T20:25:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3830207147 wait complete. 2024-04-26T20:25:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.392+0000 7fb79d800640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7800a3c40 0x7fb7800a6100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x561bd8f62860 tx=0x7fb78c0187f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 -- start start 2024-04-26T20:25:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 Processor -- start 2024-04-26T20:25:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f61a70 0x561bd8f6bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f6c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561bd8f6a580 con 0x561bd8f62460 2024-04-26T20:25:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561bd8e4e170 con 0x561bd8f675d0 2024-04-26T20:25:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561bd8f6a700 con 0x561bd8f61a70 2024-04-26T20:25:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.393+0000 7fb7a2fb6740 10 client.0 _ll_register_callbacks cb 0x561bd8ebd770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f61a70 0x561bd8f6bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f61a70 0x561bd8f6bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54356/0 (socket says 172.21.15.131:54356) 2024-04-26T20:25:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 -- 172.21.15.131:0/2348094574 learned_addr learned my addr 172.21.15.131:0/2348094574 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:58.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79ce00640 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f6c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 -- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 msgr2=0x561bd8f734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:58.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 -- 172.21.15.131:0/2348094574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 msgr2=0x561bd8f6c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:58.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f6c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 -- 172.21.15.131:0/2348094574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd8d00f70 con 0x561bd8f61a70 2024-04-26T20:25:58.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb79e200640 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f61a70 0x561bd8f6bf20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb79000ec70 tx=0x7fb79000e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=403959a26973174d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:58.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.394+0000 7fb77f400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:25:58.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.395+0000 7fb77f400640 1 -- 172.21.15.131:0/2348094574 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79002d060 con 0x561bd8f61a70 2024-04-26T20:25:58.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.395+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb790009e50 con 0x561bd8f61a70 2024-04-26T20:25:58.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.395+0000 7fb77f400640 1 -- 172.21.15.131:0/2348094574 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb790004230 con 0x561bd8f61a70 2024-04-26T20:25:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.395+0000 7fb77f400640 1 -- 172.21.15.131:0/2348094574 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb79001e070 con 0x561bd8f61a70 2024-04-26T20:25:58.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.395+0000 7fb7a2fb6740 10 client.26548 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:58.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.395+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561bd8b98170 con 0x561bd8f61a70 2024-04-26T20:25:58.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.395+0000 7fb77f400640 1 -- 172.21.15.131:0/2348094574 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb790009e50 con 0x561bd8f61a70 2024-04-26T20:25:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb77f400640 1 -- 172.21.15.131:0/2348094574 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb7900035f0 con 0x561bd8f61a70 2024-04-26T20:25:58.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb7a2fb6740 10 client.26548 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:25:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb7a2fb6740 10 client.26548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:25:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb7a2fb6740 20 client.26548 populate_metadata read hostname 'smithi131' 2024-04-26T20:25:58.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8f77750 0x561bd8f79b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb7a2fb6740 4 client.26548 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:25:58.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd8ed4590 con 0x561bd8f77750 2024-04-26T20:25:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.396+0000 7fb79ce00640 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8f77750 0x561bd8f79b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.397+0000 7fb79ce00640 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8f77750 0x561bd8f79b40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac322f99751399 server_cookie=7a124efba7b78e4e in_seq=0 out_seq=0 2024-04-26T20:25:58.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.397+0000 7fb77f400640 10 client.26548 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.397+0000 7fb77f400640 1 -- 172.21.15.131:0/2348094574 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bd8ed4590 con 0x561bd8f77750 2024-04-26T20:25:58.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.397+0000 7fb77f400640 10 client.26548 handle_command_reply: tid=0 2024-04-26T20:25:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 client.26548 shutdown 2024-04-26T20:25:58.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f61a70 msgr2=0x561bd8f6bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f61a70 0x561bd8f6bf20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb79000ec70 tx=0x7fb79000e3b0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 shutdown_connections 2024-04-26T20:25:58.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8f77750 0x561bd8f79b40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f6c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/2348094574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f61a70 0x561bd8f6bf20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 >> 172.21.15.131:0/2348094574 conn(0x561bd8f5d0e0 msgr2=0x561bd8f5c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:58.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb77f400640 0 client.26548 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:25:58.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 shutdown_connections 2024-04-26T20:25:58.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/2348094574 wait complete. 2024-04-26T20:25:58.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.398+0000 7fb7a2fb6740 20 client.26548 trim_cache size 0 max 16384 2024-04-26T20:25:58.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 Processor -- start 2024-04-26T20:25:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 -- start start 2024-04-26T20:25:58.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f675d0 0x561bd8f5ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981b0010 con 0x561bd8f62460 2024-04-26T20:25:58.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981ad8e0 con 0x561bd8f61a70 2024-04-26T20:25:58.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981ab730 con 0x561bd8f675d0 2024-04-26T20:25:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb79ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb79ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f73460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52736/0 (socket says 172.21.15.131:52736) 2024-04-26T20:25:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.418+0000 7fb79ce00640 1 -- 172.21.15.131:0/1809130508 learned_addr learned my addr 172.21.15.131:0/1809130508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:59.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79e200640 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79e200640 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f675d0 0x561bd8f5ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79ce00640 1 -- 172.21.15.131:0/1809130508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f675d0 msgr2=0x561bd8f5ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:59.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79ce00640 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f675d0 0x561bd8f5ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79ce00640 1 -- 172.21.15.131:0/1809130508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 msgr2=0x561bd8f6c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:59.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79ce00640 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f6c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79ce00640 1 -- 172.21.15.131:0/1809130508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd8f66b60 con 0x561bd8f62460 2024-04-26T20:25:59.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.419+0000 7fb79ce00640 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f73460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb784006010 tx=0x7fb784004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec1d7659542dd48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb78a000640 1 -- 172.21.15.131:0/1809130508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784049020 con 0x561bd8f62460 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb78a000640 1 -- 172.21.15.131:0/1809130508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7840315a0 con 0x561bd8f62460 2024-04-26T20:25:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb78a000640 1 -- 172.21.15.131:0/1809130508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784035040 con 0x561bd8f62460 2024-04-26T20:25:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/1809130508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 msgr2=0x561bd8f73460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f73460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb784006010 tx=0x7fb784004510 comp rx=0 tx=0).stop 2024-04-26T20:25:59.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/1809130508 shutdown_connections 2024-04-26T20:25:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f675d0 0x561bd8f5ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f62460 0x561bd8f73460 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.420+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/1809130508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f61a70 0x561bd8f6c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.421+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/1809130508 >> 172.21.15.131:0/1809130508 conn(0x561bd8f5d0e0 msgr2=0x561bd8f6bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:59.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.421+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/1809130508 shutdown_connections 2024-04-26T20:25:59.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.421+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/1809130508 wait complete. 2024-04-26T20:25:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.421+0000 7fb7a2fb6740 1 -- start start 2024-04-26T20:25:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.421+0000 7fb7a2fb6740 1 Processor -- start 2024-04-26T20:25:59.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f61a70 0x561bd8f709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f70f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb7a2fb6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f71f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981abda0 con 0x561bd8f61a70 2024-04-26T20:25:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981ad210 con 0x561bd8f675d0 2024-04-26T20:25:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb7a2fb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981ac470 con 0x561bd8f62460 2024-04-26T20:25:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb7a2fb6740 10 client.0 _ll_register_callbacks cb 0x7fb79001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:25:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f61a70 0x561bd8f709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f61a70 0x561bd8f709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52746/0 (socket says 172.21.15.131:52746) 2024-04-26T20:25:59.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 -- 172.21.15.131:0/3890659761 learned_addr learned my addr 172.21.15.131:0/3890659761 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:25:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79ce00640 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f70f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f71f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 -- 172.21.15.131:0/3890659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 msgr2=0x561bd8f70f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:59.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f70f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 -- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 msgr2=0x561bd8f71f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:59.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f71f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.422+0000 7fb79e200640 1 -- 172.21.15.131:0/3890659761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd8f72530 con 0x561bd8f61a70 2024-04-26T20:25:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb79e200640 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f61a70 0x561bd8f709f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb790003a10 tx=0x7fb790009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b8d848fe1d90b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb77f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:25:59.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb77f400640 1 -- 172.21.15.131:0/3890659761 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79002d070 con 0x561bd8f61a70 2024-04-26T20:25:59.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb77f400640 1 -- 172.21.15.131:0/3890659761 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fb79000e620 con 0x561bd8f61a70 2024-04-26T20:25:59.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7840315a0 con 0x561bd8f61a70 2024-04-26T20:25:59.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb77f400640 1 -- 172.21.15.131:0/3890659761 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb79000e840 con 0x561bd8f61a70 2024-04-26T20:25:59.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb7a2fb6740 10 client.16461 fetch_fsmap learned FSMap version 27 2024-04-26T20:25:59.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.423+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561bd8fddc80 con 0x561bd8f61a70 2024-04-26T20:26:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb77f400640 1 -- 172.21.15.131:0/3890659761 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7840315a0 con 0x561bd8f61a70 2024-04-26T20:26:00.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb77f400640 1 -- 172.21.15.131:0/3890659761 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb790028400 con 0x561bd8f61a70 2024-04-26T20:26:00.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb7a2fb6740 10 client.16461 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:00.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb7a2fb6740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb7a2fb6740 20 client.16461 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:00.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8fde440 0x561bd8fe0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:00.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb7a2fb6740 4 client.16461 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T20:26:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561bd8ece400 con 0x561bd8fde440 2024-04-26T20:26:00.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.424+0000 7fb79ce00640 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8fde440 0x561bd8fe0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:00.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.425+0000 7fb77f400640 10 client.16461 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:00.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.425+0000 7fb79ce00640 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8fde440 0x561bd8fe0860 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5422c15c1b2c77a server_cookie=4ac259fc103b4a20 in_seq=0 out_seq=0 2024-04-26T20:26:00.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.426+0000 7fb77f400640 1 -- 172.21.15.131:0/3890659761 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561bd8ece400 con 0x561bd8fde440 2024-04-26T20:26:00.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.426+0000 7fb77f400640 10 client.16461 handle_command_reply: tid=0 2024-04-26T20:26:00.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.426+0000 7fb7a2fb6740 1 client.16461 shutdown 2024-04-26T20:26:00.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f61a70 msgr2=0x561bd8f709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:00.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f61a70 0x561bd8f709f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb790003a10 tx=0x7fb790009a50 comp rx=0 tx=0).stop 2024-04-26T20:26:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 shutdown_connections 2024-04-26T20:26:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x561bd8fde440 0x561bd8fe0860 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561bd8f675d0 0x561bd8f71f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bd8f62460 0x561bd8f70f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 --2- 172.21.15.131:0/3890659761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd8f61a70 0x561bd8f709f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb77f400640 0 client.16461 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:00.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 >> 172.21.15.131:0/3890659761 conn(0x561bd8f5d0e0 msgr2=0x561bd8f5ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 shutdown_connections 2024-04-26T20:26:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 1 -- 172.21.15.131:0/3890659761 wait complete. 2024-04-26T20:26:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.427+0000 7fb7a2fb6740 20 client.16461 trim_cache size 0 max 16384 2024-04-26T20:26:00.508 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:26:00.514 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-04-26T20:26:00.587 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "cf31b544-c766-4f1b-9ea7-c67589c06d36", 2024-04-26T20:26:00.594 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-04-26T20:26:00.600 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:26:00.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7800a3c40 msgr2=0x7fb7800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7800a3c40 0x7fb7800a6100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x561bd8f62860 tx=0x7fb78c0187f0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798102860 msgr2=0x7fb79819acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:00.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798102860 0x7fb79819acc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d20 tx=0x7fb78c00c6a0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 shutdown_connections 2024-04-26T20:26:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fb7800a3c40 0x7fb7800a6100 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798104410 0x7fb7981a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb798103a50 0x7fb79819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 --2- 172.21.15.131:0/3925403441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb798102860 0x7fb79819acc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:00.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 >> 172.21.15.131:0/3925403441 conn(0x7fb7980fddd0 msgr2=0x7fb798074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:00.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 shutdown_connections 2024-04-26T20:26:00.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:25:56.428+0000 7fb79ec00640 1 -- 172.21.15.131:0/3925403441 wait complete. 2024-04-26T20:26:00.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T20:26:00.908 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T20:26:00.994 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:57 smithi131 ceph-mon[27919]: pgmap v888: 129 pgs: 129 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-04-26T20:26:01.002 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:57 smithi148 ceph-mon[32281]: pgmap v888: 129 pgs: 129 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-04-26T20:26:01.010 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:58 smithi148 ceph-mon[32281]: scrub queued for path: / 2024-04-26T20:26:01.019 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:58 smithi148 ceph-mon[32281]: scrub summary: idle+waiting paths [/] 2024-04-26T20:26:01.028 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:58 smithi148 ceph-mon[32281]: scrub summary: active paths [/] 2024-04-26T20:26:01.036 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:58 smithi148 ceph-mon[32281]: scrub summary: idle+waiting paths [/] 2024-04-26T20:26:01.093 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:58 smithi148 ceph-mon[32281]: scrub summary: idle 2024-04-26T20:26:01.100 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:58 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:26:01.108 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:25:59 smithi148 ceph-mon[32281]: pgmap v889: 129 pgs: 129 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-26T20:26:01.117 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.126 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.134 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.208 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.266 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:26:01.274 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:26:01.285 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:26:01.292 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:26:01.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:26:01.368 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:57 smithi157 ceph-mon[32703]: pgmap v888: 129 pgs: 129 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-04-26T20:26:01.375 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:58 smithi157 ceph-mon[32703]: scrub queued for path: / 2024-04-26T20:26:01.382 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:58 smithi157 ceph-mon[32703]: scrub summary: idle+waiting paths [/] 2024-04-26T20:26:01.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:58 smithi157 ceph-mon[32703]: scrub summary: active paths [/] 2024-04-26T20:26:01.397 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:58 smithi157 ceph-mon[32703]: scrub summary: idle+waiting paths [/] 2024-04-26T20:26:01.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:58 smithi157 ceph-mon[32703]: scrub summary: idle 2024-04-26T20:26:01.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:58 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:26:01.466 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:25:59 smithi157 ceph-mon[32703]: pgmap v889: 129 pgs: 129 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-26T20:26:01.473 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.481 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.520 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:01.579 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:26:01.586 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:26:01.594 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:26:01.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:26:01.667 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:26:01.674 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:58 smithi131 ceph-mon[27919]: scrub queued for path: / 2024-04-26T20:26:01.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:58 smithi131 ceph-mon[27919]: scrub summary: idle+waiting paths [/] 2024-04-26T20:26:01.688 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:58 smithi131 ceph-mon[27919]: scrub summary: active paths [/] 2024-04-26T20:26:01.695 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:58 smithi131 ceph-mon[27919]: scrub summary: idle+waiting paths [/] 2024-04-26T20:26:01.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:58 smithi131 ceph-mon[27919]: scrub summary: idle 2024-04-26T20:26:01.732 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:58 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:26:01.740 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:01 smithi148 ceph-mon[32281]: pgmap v890: 129 pgs: 129 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 271 op/s 2024-04-26T20:26:01.748 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:01 smithi157 ceph-mon[32703]: pgmap v890: 129 pgs: 129 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 271 op/s 2024-04-26T20:26:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 -- 172.21.15.131:0/545758799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc104550 msgr2=0x7f43fc10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 --2- 172.21.15.131:0/545758799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc104550 0x7f43fc10ade0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009be0 tx=0x7f43f402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 -- 172.21.15.131:0/545758799 shutdown_connections 2024-04-26T20:26:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 --2- 172.21.15.131:0/545758799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc104550 0x7f43fc10ade0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 --2- 172.21.15.131:0/545758799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43fc103b90 0x7f43fc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 --2- 172.21.15.131:0/545758799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc1029a0 0x7f43fc102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 -- 172.21.15.131:0/545758799 >> 172.21.15.131:0/545758799 conn(0x7f43fc0fdef0 msgr2=0x7f43fc100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 -- 172.21.15.131:0/545758799 shutdown_connections 2024-04-26T20:26:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.719+0000 7f4403600640 1 -- 172.21.15.131:0/545758799 wait complete. 2024-04-26T20:26:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.720+0000 7f4403600640 1 Processor -- start 2024-04-26T20:26:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4403600640 1 -- start start 2024-04-26T20:26:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4403600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43fc1029a0 0x7f43fc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4403600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc103b90 0x7f43fc199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4403600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc104550 0x7f43fc1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4403600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10e4d0 con 0x7f43fc1029a0 2024-04-26T20:26:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4403600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10e350 con 0x7f43fc103b90 2024-04-26T20:26:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4403600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10e650 con 0x7f43fc104550 2024-04-26T20:26:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc103b90 0x7f43fc199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f4400c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43fc1029a0 0x7f43fc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc103b90 0x7f43fc199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55412/0 (socket says 172.21.15.131:55412) 2024-04-26T20:26:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.721+0000 7f43fbe00640 1 -- 172.21.15.131:0/3394516389 learned_addr learned my addr 172.21.15.131:0/3394516389 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f4401600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc104550 0x7f43fc1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f43fbe00640 1 -- 172.21.15.131:0/3394516389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc104550 msgr2=0x7f43fc1a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f43fbe00640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc104550 0x7f43fc1a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f43fbe00640 1 -- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43fc1029a0 msgr2=0x7f43fc198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f43fbe00640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43fc1029a0 0x7f43fc198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f43fbe00640 1 -- 172.21.15.131:0/3394516389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc1a0710 con 0x7f43fc103b90 2024-04-26T20:26:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f43fbe00640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc103b90 0x7f43fc199120 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9c0 tx=0x7f43e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.722+0000 7f43f9600640 1 -- 172.21.15.131:0/3394516389 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e800cde0 con 0x7f43fc103b90 2024-04-26T20:26:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.723+0000 7f43f9600640 1 -- 172.21.15.131:0/3394516389 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e80035f0 con 0x7f43fc103b90 2024-04-26T20:26:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.723+0000 7f43f9600640 1 -- 172.21.15.131:0/3394516389 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8010690 con 0x7f43fc103b90 2024-04-26T20:26:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.723+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc1a0a00 con 0x7f43fc103b90 2024-04-26T20:26:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.723+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc1a0fa0 con 0x7f43fc103b90 2024-04-26T20:26:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.726+0000 7f43f9600640 1 -- 172.21.15.131:0/3394516389 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f43e8004000 con 0x7f43fc103b90 2024-04-26T20:26:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.727+0000 7f43f9600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f43e40a3d70 0x7f43e40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.727+0000 7f43f9600640 1 -- 172.21.15.131:0/3394516389 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f43e80d0c70 con 0x7f43fc103b90 2024-04-26T20:26:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 Processor -- start 2024-04-26T20:26:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4400c00640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f43e40a3d70 0x7f43e40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 -- start start 2024-04-26T20:26:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e6fe0 0x55fbe15e7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 0x55fbe15ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fbe1547380 con 0x55fbe15e79d0 2024-04-26T20:26:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fbe14e3c30 con 0x55fbe15e6fe0 2024-04-26T20:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.728+0000 7f4406477740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fbe1543340 con 0x55fbe15ecb40 2024-04-26T20:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4401600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4401600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15eefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54404/0 (socket says 172.21.15.131:54404) 2024-04-26T20:26:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4401600640 1 -- 172.21.15.131:0/4217398910 learned_addr learned my addr 172.21.15.131:0/4217398910 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4400c00640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f43e40a3d70 0x7f43e40a6230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43fc1039f0 tx=0x7f43f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4400c00640 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e6fe0 0x55fbe15e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4400c00640 1 -- 172.21.15.131:0/4217398910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 msgr2=0x55fbe15eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4400c00640 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4400c00640 1 -- 172.21.15.131:0/4217398910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 msgr2=0x55fbe15ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4400c00640 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 0x55fbe15ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.729+0000 7f4400c00640 1 -- 172.21.15.131:0/4217398910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe1505940 con 0x55fbe15e6fe0 2024-04-26T20:26:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.730+0000 7f4400c00640 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e6fe0 0x55fbe15e7400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f43f00118d0 tx=0x7f43f0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb94291d0a16613 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.730+0000 7f43edc00640 1 -- 172.21.15.131:0/4217398910 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f000ee60 con 0x55fbe15e6fe0 2024-04-26T20:26:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.730+0000 7f43edc00640 1 -- 172.21.15.131:0/4217398910 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f00035f0 con 0x55fbe15e6fe0 2024-04-26T20:26:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.730+0000 7f43edc00640 1 -- 172.21.15.131:0/4217398910 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f00266e0 con 0x55fbe15e6fe0 2024-04-26T20:26:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 -- 172.21.15.131:0/4217398910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e6fe0 msgr2=0x55fbe15e7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e6fe0 0x55fbe15e7400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f43f00118d0 tx=0x7f43f0014000 comp rx=0 tx=0).stop 2024-04-26T20:26:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 -- 172.21.15.131:0/4217398910 shutdown_connections 2024-04-26T20:26:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 0x55fbe15ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 --2- 172.21.15.131:0/4217398910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e6fe0 0x55fbe15e7400 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 -- 172.21.15.131:0/4217398910 >> 172.21.15.131:0/4217398910 conn(0x55fbe15e2650 msgr2=0x55fbe15e2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 -- 172.21.15.131:0/4217398910 shutdown_connections 2024-04-26T20:26:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.731+0000 7f4406477740 1 -- 172.21.15.131:0/4217398910 wait complete. 2024-04-26T20:26:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 -- start start 2024-04-26T20:26:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 Processor -- start 2024-04-26T20:26:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fbe15efaf0 con 0x55fbe15e6fe0 2024-04-26T20:26:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fbe14cec60 con 0x55fbe15e79d0 2024-04-26T20:26:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fbe15efc70 con 0x55fbe15ecb40 2024-04-26T20:26:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.732+0000 7f4406477740 10 client.0 _ll_register_callbacks cb 0x55fbe14fae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54406/0 (socket says 172.21.15.131:54406) 2024-04-26T20:26:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 -- 172.21.15.131:0/2037352052 learned_addr learned my addr 172.21.15.131:0/2037352052 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f43fbe00640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4400c00640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 -- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 msgr2=0x55fbe15f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 -- 172.21.15.131:0/2037352052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 msgr2=0x55fbe15f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 -- 172.21.15.131:0/2037352052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe14dd510 con 0x55fbe15ecb40 2024-04-26T20:26:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f43fbe00640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f4401600640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f8800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43f4034040 tx=0x7f43f40305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed6ea48a404de74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.733+0000 7f43e2000640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:26:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.734+0000 7f43e2000640 1 -- 172.21.15.131:0/2037352052 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4036040 con 0x55fbe15ecb40 2024-04-26T20:26:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.734+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f00035f0 con 0x55fbe15ecb40 2024-04-26T20:26:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.734+0000 7f43e2000640 1 -- 172.21.15.131:0/2037352052 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f43f4030c70 con 0x55fbe15ecb40 2024-04-26T20:26:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.734+0000 7f43e2000640 1 -- 172.21.15.131:0/2037352052 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43f4050050 con 0x55fbe15ecb40 2024-04-26T20:26:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.734+0000 7f4406477740 10 client.26566 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.734+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbe1259210 con 0x55fbe15ecb40 2024-04-26T20:26:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.734+0000 7f43e2000640 1 -- 172.21.15.131:0/2037352052 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f00035f0 con 0x55fbe15ecb40 2024-04-26T20:26:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f43e2000640 1 -- 172.21.15.131:0/2037352052 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f43f4004510 con 0x55fbe15ecb40 2024-04-26T20:26:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f4406477740 10 client.26566 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f4406477740 10 client.26566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f4406477740 20 client.26566 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f4406477740 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe15fcdc0 0x55fbe15ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f4406477740 4 client.26566 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbe1554f90 con 0x55fbe15fcdc0 2024-04-26T20:26:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.735+0000 7f4400c00640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe15fcdc0 0x55fbe15ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.736+0000 7f43e2000640 10 client.26566 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.736+0000 7f4400c00640 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe15fcdc0 0x55fbe15ff1b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f16e6c3d4b6bada server_cookie=4043fa9b5ce50a52 in_seq=0 out_seq=0 2024-04-26T20:26:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.736+0000 7f43e2000640 1 -- 172.21.15.131:0/2037352052 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbe1554f90 con 0x55fbe15fcdc0 2024-04-26T20:26:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.736+0000 7f43e2000640 10 client.26566 handle_command_reply: tid=0 2024-04-26T20:26:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 client.26566 shutdown 2024-04-26T20:26:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 msgr2=0x55fbe15f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f8800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43f4034040 tx=0x7f43f40305d0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 shutdown_connections 2024-04-26T20:26:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe15fcdc0 0x55fbe15ff1b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f8800 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f43e2000640 0 client.26566 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 --2- 172.21.15.131:0/2037352052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 >> 172.21.15.131:0/2037352052 conn(0x55fbe15e2650 msgr2=0x55fbe15dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 shutdown_connections 2024-04-26T20:26:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 1 -- 172.21.15.131:0/2037352052 wait complete. 2024-04-26T20:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.737+0000 7f4406477740 20 client.26566 trim_cache size 0 max 16384 2024-04-26T20:26:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 Processor -- start 2024-04-26T20:26:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 -- start start 2024-04-26T20:26:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 0x55fbe15f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15ecb40 0x55fbe15f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc1abbf0 con 0x55fbe15e79d0 2024-04-26T20:26:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc1a94c0 con 0x55fbe15ecb40 2024-04-26T20:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.756+0000 7f4406477740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc1a7310 con 0x55fbe15e6fe0 2024-04-26T20:26:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15ecb40 0x55fbe15f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4400c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15ecb40 0x55fbe15f17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55430/0 (socket says 172.21.15.131:55430) 2024-04-26T20:26:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4400c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54408/0 (socket says 172.21.15.131:54408) 2024-04-26T20:26:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 -- 172.21.15.131:0/3945565073 learned_addr learned my addr 172.21.15.131:0/3945565073 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f43fbe00640 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 0x55fbe15f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 -- 172.21.15.131:0/3945565073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15e6fe0 msgr2=0x55fbe15f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:04.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 -- 172.21.15.131:0/3945565073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 msgr2=0x55fbe15f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 0x55fbe15f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 -- 172.21.15.131:0/3945565073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe15ff520 con 0x55fbe15ecb40 2024-04-26T20:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4400c00640 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.757+0000 7f4401600640 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15ecb40 0x55fbe15f17c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f43f4004960 tx=0x7f43f4005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01dfa34bfb6342a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.758+0000 7f43ed200640 1 -- 172.21.15.131:0/3945565073 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4059070 con 0x55fbe15ecb40 2024-04-26T20:26:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.758+0000 7f43ed200640 1 -- 172.21.15.131:0/3945565073 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f40475e0 con 0x55fbe15ecb40 2024-04-26T20:26:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.758+0000 7f43ed200640 1 -- 172.21.15.131:0/3945565073 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4036070 con 0x55fbe15ecb40 2024-04-26T20:26:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.758+0000 7f4406477740 1 -- 172.21.15.131:0/3945565073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15ecb40 msgr2=0x55fbe15f17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.758+0000 7f4406477740 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15ecb40 0x55fbe15f17c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f43f4004960 tx=0x7f43f4005f70 comp rx=0 tx=0).stop 2024-04-26T20:26:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 -- 172.21.15.131:0/3945565073 shutdown_connections 2024-04-26T20:26:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15ecb40 0x55fbe15f17c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e79d0 0x55fbe15f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 --2- 172.21.15.131:0/3945565073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 -- 172.21.15.131:0/3945565073 >> 172.21.15.131:0/3945565073 conn(0x55fbe15e2650 msgr2=0x55fbe15e8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 -- 172.21.15.131:0/3945565073 shutdown_connections 2024-04-26T20:26:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 -- 172.21.15.131:0/3945565073 wait complete. 2024-04-26T20:26:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 -- start start 2024-04-26T20:26:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.759+0000 7f4406477740 1 Processor -- start 2024-04-26T20:26:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4406477740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4406477740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4406477740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4406477740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc1a7980 con 0x55fbe15e6fe0 2024-04-26T20:26:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4406477740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc1a8df0 con 0x55fbe15e79d0 2024-04-26T20:26:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4406477740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc1a8050 con 0x55fbe15ecb40 2024-04-26T20:26:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4406477740 10 client.0 _ll_register_callbacks cb 0x7f43f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4401600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4401600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54422/0 (socket says 172.21.15.131:54422) 2024-04-26T20:26:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4401600640 1 -- 172.21.15.131:0/3953699586 learned_addr learned my addr 172.21.15.131:0/3953699586 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.760+0000 7f4400c00640 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43fbe00640 1 -- 172.21.15.131:0/3953699586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 msgr2=0x55fbe15f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43fbe00640 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43fbe00640 1 -- 172.21.15.131:0/3953699586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 msgr2=0x55fbe15f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43fbe00640 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43fbe00640 1 -- 172.21.15.131:0/3953699586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe1325620 con 0x55fbe15e79d0 2024-04-26T20:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f4401600640 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43fbe00640 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f6450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ce1d0 tx=0x7f43e809c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f62f0a093d14a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43e2000640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:26:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43e2000640 1 -- 172.21.15.131:0/3953699586 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e809c9a0 con 0x55fbe15e79d0 2024-04-26T20:26:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43e2000640 1 -- 172.21.15.131:0/3953699586 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f43e80037a0 con 0x55fbe15e79d0 2024-04-26T20:26:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f40475e0 con 0x55fbe15e79d0 2024-04-26T20:26:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f43e2000640 1 -- 172.21.15.131:0/3953699586 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43e80a0d50 con 0x55fbe15e79d0 2024-04-26T20:26:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f4406477740 10 client.26594 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.761+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbe1501cb0 con 0x55fbe15e79d0 2024-04-26T20:26:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f43e2000640 1 -- 172.21.15.131:0/3953699586 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f40475e0 con 0x55fbe15e79d0 2024-04-26T20:26:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f43e2000640 1 -- 172.21.15.131:0/3953699586 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f43e80986f0 con 0x55fbe15e79d0 2024-04-26T20:26:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f4406477740 10 client.26594 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f4406477740 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f4406477740 20 client.26594 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f4406477740 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe16639c0 0x55fbe1665de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f4406477740 4 client.26594 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:26:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbe154ee00 con 0x55fbe16639c0 2024-04-26T20:26:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.762+0000 7f4400c00640 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe16639c0 0x55fbe1665de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.763+0000 7f4400c00640 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe16639c0 0x55fbe1665de0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c163497cb5514a58 server_cookie=88f59201c0b550f6 in_seq=0 out_seq=0 2024-04-26T20:26:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.763+0000 7f43e2000640 10 client.26594 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f43e2000640 1 -- 172.21.15.131:0/3953699586 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fbe154ee00 con 0x55fbe16639c0 2024-04-26T20:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f43e2000640 10 client.26594 handle_command_reply: tid=0 2024-04-26T20:26:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 client.26594 shutdown 2024-04-26T20:26:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 msgr2=0x55fbe15f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f6450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ce1d0 tx=0x7f43e809c880 comp rx=0 tx=0).stop 2024-04-26T20:26:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 shutdown_connections 2024-04-26T20:26:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55fbe16639c0 0x55fbe1665de0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbe15ecb40 0x55fbe15f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fbe15e79d0 0x55fbe15f6450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f43e2000640 0 client.26594 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 --2- 172.21.15.131:0/3953699586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fbe15e6fe0 0x55fbe15f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.775+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 >> 172.21.15.131:0/3953699586 conn(0x55fbe15e2650 msgr2=0x55fbe15f1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.776+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 shutdown_connections 2024-04-26T20:26:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.776+0000 7f4406477740 1 -- 172.21.15.131:0/3953699586 wait complete. 2024-04-26T20:26:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.776+0000 7f4406477740 20 client.26594 trim_cache size 0 max 16384 2024-04-26T20:26:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:26:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:26:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:26:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:26:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.776+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f43e40a3d70 msgr2=0x7f43e40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.776+0000 7f4403600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f43e40a3d70 0x7f43e40a6230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43fc1039f0 tx=0x7f43f0005f90 comp rx=0 tx=0).stop 2024-04-26T20:26:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.776+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc103b90 msgr2=0x7f43fc199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.776+0000 7f4403600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc103b90 0x7f43fc199120 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9c0 tx=0x7f43e800ee80 comp rx=0 tx=0).stop 2024-04-26T20:26:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 shutdown_connections 2024-04-26T20:26:06.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f43e40a3d70 0x7f43e40a6230 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc104550 0x7f43fc1a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43fc103b90 0x7f43fc199120 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 --2- 172.21.15.131:0/3394516389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43fc1029a0 0x7f43fc198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 >> 172.21.15.131:0/3394516389 conn(0x7f43fc0fdef0 msgr2=0x7f43fc106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 shutdown_connections 2024-04-26T20:26:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:25:58.777+0000 7f4403600640 1 -- 172.21.15.131:0/3394516389 wait complete. 2024-04-26T20:26:06.435 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:25:59 smithi131 ceph-mon[27919]: pgmap v889: 129 pgs: 129 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-26T20:26:06.442 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:03 smithi157 ceph-mon[32703]: pgmap v891: 129 pgs: 129 active+clean; 13 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 274 op/s 2024-04-26T20:26:06.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:05 smithi157 ceph-mon[32703]: pgmap v892: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T20:26:06.479 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:03 smithi148 ceph-mon[32281]: pgmap v891: 129 pgs: 129 active+clean; 13 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 274 op/s 2024-04-26T20:26:06.558 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:05 smithi148 ceph-mon[32281]: pgmap v892: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T20:26:06.569 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:06.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:06.589 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:06.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:26:06.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:26:06.692 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:26:06.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:26:06.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:26:06.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:26:06.726 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:01 smithi131 ceph-mon[27919]: pgmap v890: 129 pgs: 129 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 271 op/s 2024-04-26T20:26:06.736 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:03 smithi131 ceph-mon[27919]: pgmap v891: 129 pgs: 129 active+clean; 13 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 274 op/s 2024-04-26T20:26:06.799 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:05 smithi131 ceph-mon[27919]: pgmap v892: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T20:26:06.846 INFO:tasks.workunit.client.0.smithi131.stdout:.......... 2024-04-26T20:26:06.855 INFO:tasks.workunit.client.0.smithi131.stdout:do_o_direct_reads: SUCCESS 2024-04-26T20:26:06.864 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T20:26:06.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T20:26:06.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.938+0000 7f4de3e00640 1 -- 172.21.15.131:0/1394770116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc0772f0 msgr2=0x7f4ddc077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.938+0000 7f4de3e00640 1 --2- 172.21.15.131:0/1394770116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc0772f0 0x7f4ddc077710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009ca0 tx=0x7f4dd402a8b0 comp rx=0 tx=0).stop 2024-04-26T20:26:07.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.939+0000 7f4de3e00640 1 -- 172.21.15.131:0/1394770116 shutdown_connections 2024-04-26T20:26:07.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.939+0000 7f4de3e00640 1 --2- 172.21.15.131:0/1394770116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc078ed0 0x7f4ddc079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:07.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.939+0000 7f4de3e00640 1 --2- 172.21.15.131:0/1394770116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ddc078530 0x7f4ddc078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:07.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.939+0000 7f4de3e00640 1 --2- 172.21.15.131:0/1394770116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc0772f0 0x7f4ddc077710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:07.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.939+0000 7f4de3e00640 1 -- 172.21.15.131:0/1394770116 >> 172.21.15.131:0/1394770116 conn(0x7f4ddc100120 msgr2=0x7f4ddc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:07.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.939+0000 7f4de3e00640 1 -- 172.21.15.131:0/1394770116 shutdown_connections 2024-04-26T20:26:07.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.939+0000 7f4de3e00640 1 -- 172.21.15.131:0/1394770116 wait complete. 2024-04-26T20:26:07.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 Processor -- start 2024-04-26T20:26:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 -- start start 2024-04-26T20:26:07.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ddc0772f0 0x7f4ddc19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc078530 0x7f4ddc19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:07.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc078ed0 0x7f4ddc1a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:07.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc114a90 con 0x7f4ddc078ed0 2024-04-26T20:26:07.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc114910 con 0x7f4ddc0772f0 2024-04-26T20:26:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.940+0000 7f4de3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc114c10 con 0x7f4ddc078530 2024-04-26T20:26:07.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc078ed0 0x7f4ddc1a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:07.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc078ed0 0x7f4ddc1a6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56970/0 (socket says 172.21.15.131:56970) 2024-04-26T20:26:07.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 -- 172.21.15.131:0/2133747413 learned_addr learned my addr 172.21.15.131:0/2133747413 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:07.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc078530 0x7f4ddc19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:07.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1400640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ddc0772f0 0x7f4ddc19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 -- 172.21.15.131:0/2133747413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc078530 msgr2=0x7f4ddc19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:07.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc078530 0x7f4ddc19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:07.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 -- 172.21.15.131:0/2133747413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ddc0772f0 msgr2=0x7f4ddc19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:07.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ddc0772f0 0x7f4ddc19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 -- 172.21.15.131:0/2133747413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc1a6df0 con 0x7f4ddc078ed0 2024-04-26T20:26:07.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4de1e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc078ed0 0x7f4ddc1a6780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e970 tx=0x7f4dcc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:07.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.941+0000 7f4dda000640 1 -- 172.21.15.131:0/2133747413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00ccc0 con 0x7f4ddc078ed0 2024-04-26T20:26:07.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.942+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc1a70e0 con 0x7f4ddc078ed0 2024-04-26T20:26:07.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.942+0000 7f4dda000640 1 -- 172.21.15.131:0/2133747413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dcc00ce60 con 0x7f4ddc078ed0 2024-04-26T20:26:07.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.942+0000 7f4dda000640 1 -- 172.21.15.131:0/2133747413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc018430 con 0x7f4ddc078ed0 2024-04-26T20:26:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.942+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc1a75e0 con 0x7f4ddc078ed0 2024-04-26T20:26:07.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 Processor -- start 2024-04-26T20:26:07.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4dda000640 1 -- 172.21.15.131:0/2133747413 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4dcc004f70 con 0x7f4ddc078ed0 2024-04-26T20:26:07.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 -- start start 2024-04-26T20:26:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:07.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c06c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c09780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:07.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557c95b12030 con 0x557c95c02180 2024-04-26T20:26:07.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c95b2d7f0 con 0x557c95c01790 2024-04-26T20:26:07.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.945+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c95ae87e0 con 0x557c95c072f0 2024-04-26T20:26:07.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c06c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c06c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56986/0 (socket says 172.21.15.131:56986) 2024-04-26T20:26:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 -- 172.21.15.131:0/737229717 learned_addr learned my addr 172.21.15.131:0/737229717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:07.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de0a00640 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de0a00640 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c09780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 -- 172.21.15.131:0/737229717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 msgr2=0x557c95c09780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c09780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:07.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 -- 172.21.15.131:0/737229717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 msgr2=0x557c95c01bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:07.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c01bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:08.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 -- 172.21.15.131:0/737229717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c95af4050 con 0x557c95c02180 2024-04-26T20:26:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de0a00640 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c09780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.946+0000 7f4de1400640 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c06c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4004a80 tx=0x7f4dd4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2a979f2fe83a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:08.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.947+0000 7f4dda000640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4dc00a3c40 0x7f4dc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:08.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.947+0000 7f4dd2600640 1 -- 172.21.15.131:0/737229717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4030900 con 0x557c95c02180 2024-04-26T20:26:08.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.947+0000 7f4dd2600640 1 -- 172.21.15.131:0/737229717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd4036030 con 0x557c95c02180 2024-04-26T20:26:08.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.947+0000 7f4dd2600640 1 -- 172.21.15.131:0/737229717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4004500 con 0x557c95c02180 2024-04-26T20:26:08.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.947+0000 7f4dda000640 1 -- 172.21.15.131:0/2133747413 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4dcc0d0ef0 con 0x7f4ddc078ed0 2024-04-26T20:26:08.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.947+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4dc00a3c40 0x7f4dc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:08.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 -- 172.21.15.131:0/737229717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 msgr2=0x557c95c06c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:08.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c06c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4004a80 tx=0x7f4dd4004ab0 comp rx=0 tx=0).stop 2024-04-26T20:26:08.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 -- 172.21.15.131:0/737229717 shutdown_connections 2024-04-26T20:26:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c09780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:08.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c06c80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:08.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4dc00a3c40 0x7f4dc00a6100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x557c95c029e0 tx=0x7f4dc80096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/737229717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:08.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 -- 172.21.15.131:0/737229717 >> 172.21.15.131:0/737229717 conn(0x557c95bfcdb0 msgr2=0x557c95bfd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 -- 172.21.15.131:0/737229717 shutdown_connections 2024-04-26T20:26:08.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 -- 172.21.15.131:0/737229717 wait complete. 2024-04-26T20:26:08.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 -- start start 2024-04-26T20:26:08.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.948+0000 7f4de6bcf740 1 Processor -- start 2024-04-26T20:26:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c0b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c0baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c12ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557c95b6a200 con 0x557c95c02180 2024-04-26T20:26:08.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c95b64ee0 con 0x557c95c01790 2024-04-26T20:26:08.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c95b71e70 con 0x557c95c072f0 2024-04-26T20:26:08.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de6bcf740 10 client.0 _ll_register_callbacks cb 0x557c95b5b270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:08.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c0baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:08.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de1400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c0b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c0baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56990/0 (socket says 172.21.15.131:56990) 2024-04-26T20:26:08.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de0a00640 1 -- 172.21.15.131:0/2817297190 learned_addr learned my addr 172.21.15.131:0/2817297190 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:08.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.949+0000 7f4de1e00640 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c12ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de0a00640 1 -- 172.21.15.131:0/2817297190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 msgr2=0x557c95c12ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c12ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:08.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de0a00640 1 -- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 msgr2=0x557c95c0b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:08.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c0b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:08.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de0a00640 1 -- 172.21.15.131:0/2817297190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c959417d0 con 0x557c95c02180 2024-04-26T20:26:08.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de1400640 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c0b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:08.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c0baa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8004610 tx=0x7f4dc8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37119f046f780096 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:08.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4dc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:26:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2817297190 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc801e640 con 0x557c95c02180 2024-04-26T20:26:08.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd4036030 con 0x557c95c02180 2024-04-26T20:26:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.950+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2817297190 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4dc801e7e0 con 0x557c95c02180 2024-04-26T20:26:08.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2817297190 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dc801e9e0 con 0x557c95c02180 2024-04-26T20:26:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 10 client.16488 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:09.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c958773f0 con 0x557c95c02180 2024-04-26T20:26:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2817297190 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd4036030 con 0x557c95c02180 2024-04-26T20:26:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2817297190 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4dc8026070 con 0x557c95c02180 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 10 client.16488 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:09.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 20 client.16488 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:09.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c17040 0x557c95c194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 4 client.16488 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.951+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c95b737e0 con 0x557c95c17040 2024-04-26T20:26:09.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.952+0000 7f4de1400640 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c17040 0x557c95c194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:09.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.952+0000 7f4de1400640 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c17040 0x557c95c194b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cbf07b290621fe server_cookie=3f53d7c8b5a8ff5c in_seq=0 out_seq=0 2024-04-26T20:26:09.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.952+0000 7f4dc6a00640 10 client.16488 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:09.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.953+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2817297190 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c95b737e0 con 0x557c95c17040 2024-04-26T20:26:09.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.953+0000 7f4dc6a00640 10 client.16488 handle_command_reply: tid=0 2024-04-26T20:26:09.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.953+0000 7f4de6bcf740 1 client.16488 shutdown 2024-04-26T20:26:09.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.953+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 msgr2=0x557c95c0baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:09.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.953+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c0baa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8004610 tx=0x7f4dc8005f90 comp rx=0 tx=0).stop 2024-04-26T20:26:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 shutdown_connections 2024-04-26T20:26:09.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c17040 0x557c95c194b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c072f0 0x557c95c12ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:09.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c0baa0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:09.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2817297190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c0b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:09.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4dc6a00640 0 client.16488 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:09.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 >> 172.21.15.131:0/2817297190 conn(0x557c95bfcdb0 msgr2=0x557c95bfc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:09.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 shutdown_connections 2024-04-26T20:26:09.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2817297190 wait complete. 2024-04-26T20:26:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.954+0000 7f4de6bcf740 20 client.16488 trim_cache size 0 max 16384 2024-04-26T20:26:09.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 Processor -- start 2024-04-26T20:26:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 -- start start 2024-04-26T20:26:09.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c01790 0x557c95bfd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:09.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c12aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:09.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c072f0 0x557c95bffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1b1e00 con 0x557c95c02180 2024-04-26T20:26:09.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1afc50 con 0x557c95c072f0 2024-04-26T20:26:09.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1b1730 con 0x557c95c01790 2024-04-26T20:26:09.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c01790 0x557c95bfd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:09.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c01790 0x557c95bfd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33706/0 (socket says 172.21.15.131:33706) 2024-04-26T20:26:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de1400640 1 -- 172.21.15.131:0/1411869998 learned_addr learned my addr 172.21.15.131:0/1411869998 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:09.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.973+0000 7f4de0a00640 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c12aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:09.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1e00640 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c072f0 0x557c95bffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1400640 1 -- 172.21.15.131:0/1411869998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c072f0 msgr2=0x557c95bffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1400640 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c072f0 0x557c95bffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1400640 1 -- 172.21.15.131:0/1411869998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 msgr2=0x557c95c12aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:09.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1400640 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c12aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:09.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1400640 1 -- 172.21.15.131:0/1411869998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c958370f0 con 0x557c95c01790 2024-04-26T20:26:09.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1e00640 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c072f0 0x557c95bffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:09.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4de1400640 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c01790 0x557c95bfd190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009620 tx=0x7f4dd4005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae339222beb991e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:09.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4dd1c00640 1 -- 172.21.15.131:0/1411869998 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4054020 con 0x557c95c01790 2024-04-26T20:26:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4dd1c00640 1 -- 172.21.15.131:0/1411869998 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd4036070 con 0x557c95c01790 2024-04-26T20:26:09.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.974+0000 7f4dd1c00640 1 -- 172.21.15.131:0/1411869998 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4055480 con 0x557c95c01790 2024-04-26T20:26:09.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 -- 172.21.15.131:0/1411869998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c01790 msgr2=0x557c95bfd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c01790 0x557c95bfd190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009620 tx=0x7f4dd4005fb0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 -- 172.21.15.131:0/1411869998 shutdown_connections 2024-04-26T20:26:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c072f0 0x557c95bffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c02180 0x557c95c12aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/1411869998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c01790 0x557c95bfd190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 -- 172.21.15.131:0/1411869998 >> 172.21.15.131:0/1411869998 conn(0x557c95bfcdb0 msgr2=0x557c95bfc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:10.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 -- 172.21.15.131:0/1411869998 shutdown_connections 2024-04-26T20:26:10.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.975+0000 7f4de6bcf740 1 -- 172.21.15.131:0/1411869998 wait complete. 2024-04-26T20:26:10.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 -- start start 2024-04-26T20:26:10.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 Processor -- start 2024-04-26T20:26:10.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c10240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c02180 0x557c95c10780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c072f0 0x557c95c11790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1b0990 con 0x557c95c072f0 2024-04-26T20:26:10.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1b02c0 con 0x557c95c01790 2024-04-26T20:26:10.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1b1060 con 0x557c95c02180 2024-04-26T20:26:10.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de6bcf740 10 client.0 _ll_register_callbacks cb 0x7f4dc8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:10.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.976+0000 7f4de1400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c10240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c02180 0x557c95c10780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c072f0 0x557c95c11790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c10240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:48404/0 (socket says 172.21.15.131:48404) 2024-04-26T20:26:10.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 -- 172.21.15.131:0/2057856940 learned_addr learned my addr 172.21.15.131:0/2057856940 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:10.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 -- 172.21.15.131:0/2057856940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c02180 msgr2=0x557c95c10780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:10.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c02180 0x557c95c10780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 -- 172.21.15.131:0/2057856940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c072f0 msgr2=0x557c95c11790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:10.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c072f0 0x557c95c11790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 -- 172.21.15.131:0/2057856940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c95b2da30 con 0x557c95c01790 2024-04-26T20:26:10.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de1400640 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c10240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009a60 tx=0x7f4dd4003bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346271cb04b07eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4dc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:26:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2057856940 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4054030 con 0x557c95c01790 2024-04-26T20:26:10.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd4036070 con 0x557c95c01790 2024-04-26T20:26:10.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.977+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2057856940 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4dd4055220 con 0x557c95c01790 2024-04-26T20:26:10.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2057856940 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dd4030cc0 con 0x557c95c01790 2024-04-26T20:26:10.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4de6bcf740 10 client.26612 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:10.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c95c7d630 con 0x557c95c01790 2024-04-26T20:26:10.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2057856940 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd4036070 con 0x557c95c01790 2024-04-26T20:26:10.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2057856940 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4dd4043070 con 0x557c95c01790 2024-04-26T20:26:10.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4de6bcf740 10 client.26612 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:10.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4de6bcf740 10 client.26612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:10.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4de6bcf740 20 client.26612 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:10.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.978+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c7daa0 0x557c95c7fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.979+0000 7f4de6bcf740 4 client.26612 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T20:26:10.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.979+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c95b6d650 con 0x557c95c7daa0 2024-04-26T20:26:10.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.979+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c7daa0 0x557c95c7fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.979+0000 7f4de0a00640 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c7daa0 0x557c95c7fec0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675eb905da10fe5d server_cookie=f44d636cb843513d in_seq=0 out_seq=0 2024-04-26T20:26:11.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.979+0000 7f4dc6a00640 10 client.26612 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:11.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4dc6a00640 1 -- 172.21.15.131:0/2057856940 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557c95b6d650 con 0x557c95c7daa0 2024-04-26T20:26:11.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4dc6a00640 10 client.26612 handle_command_reply: tid=0 2024-04-26T20:26:11.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 client.26612 shutdown 2024-04-26T20:26:11.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 msgr2=0x557c95c10240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c10240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009a60 tx=0x7f4dd4003bb0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 shutdown_connections 2024-04-26T20:26:11.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557c95c7daa0 0x557c95c7fec0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c95c072f0 0x557c95c11790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4dc6a00640 0 client.26612 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:11.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c95c02180 0x557c95c10780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 --2- 172.21.15.131:0/2057856940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c95c01790 0x557c95c10240 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 >> 172.21.15.131:0/2057856940 conn(0x557c95bfcdb0 msgr2=0x557c95bffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:11.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 shutdown_connections 2024-04-26T20:26:11.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 1 -- 172.21.15.131:0/2057856940 wait complete. 2024-04-26T20:26:11.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.980+0000 7f4de6bcf740 20 client.26612 trim_cache size 0 max 16384 2024-04-26T20:26:11.286 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:26:11.344 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-04-26T20:26:11.354 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-04-26T20:26:11.363 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:26:11.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4dc00a3c40 msgr2=0x7f4dc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4dc00a3c40 0x7f4dc00a6100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x557c95c029e0 tx=0x7f4dc80096c0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc078ed0 msgr2=0x7f4ddc1a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc078ed0 0x7f4ddc1a6780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e970 tx=0x7f4dcc00ed70 comp rx=0 tx=0).stop 2024-04-26T20:26:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 shutdown_connections 2024-04-26T20:26:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ddc078ed0 0x7f4ddc1a6780 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4dc00a3c40 0x7f4dc00a6100 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc078530 0x7f4ddc19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 --2- 172.21.15.131:0/2133747413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ddc0772f0 0x7f4ddc19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 >> 172.21.15.131:0/2133747413 conn(0x7f4ddc100120 msgr2=0x7f4ddc101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:11.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 shutdown_connections 2024-04-26T20:26:11.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:06.981+0000 7f4de3e00640 1 -- 172.21.15.131:0/2133747413 wait complete. 2024-04-26T20:26:11.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T20:26:11.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:11.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:07 smithi157 ceph-mon[32703]: pgmap v893: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 379 KiB/s wr, 637 op/s 2024-04-26T20:26:11.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:09 smithi157 ceph-mon[32703]: pgmap v894: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 40 KiB/s wr, 640 op/s 2024-04-26T20:26:11.727 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:07 smithi148 ceph-mon[32281]: pgmap v893: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 379 KiB/s wr, 637 op/s 2024-04-26T20:26:11.738 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:09 smithi148 ceph-mon[32281]: pgmap v894: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 40 KiB/s wr, 640 op/s 2024-04-26T20:26:11.750 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-04-26T20:26:11.762 DEBUG:teuthology.orchestra.run.smithi131:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-04-26T20:26:11.834 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:07 smithi131 ceph-mon[27919]: pgmap v893: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 379 KiB/s wr, 637 op/s 2024-04-26T20:26:11.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:09 smithi131 ceph-mon[27919]: pgmap v894: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 40 KiB/s wr, 640 op/s 2024-04-26T20:26:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 -- 172.21.15.131:0/4288389872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078ed0 msgr2=0x7fc308079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4288389872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078ed0 0x7fc308079770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc300009ca0 tx=0x7fc30002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 -- 172.21.15.131:0/4288389872 shutdown_connections 2024-04-26T20:26:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4288389872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078ed0 0x7fc308079770 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4288389872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308078530 0x7fc308078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4288389872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3080772f0 0x7fc308077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 -- 172.21.15.131:0/4288389872 >> 172.21.15.131:0/4288389872 conn(0x7fc308100150 msgr2=0x7fc308102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 -- 172.21.15.131:0/4288389872 shutdown_connections 2024-04-26T20:26:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.916+0000 7fc30ca00640 1 -- 172.21.15.131:0/4288389872 wait complete. 2024-04-26T20:26:12.097 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:11 smithi148 ceph-mon[32281]: pgmap v895: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 38 KiB/s wr, 721 op/s 2024-04-26T20:26:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.917+0000 7fc30ca00640 1 Processor -- start 2024-04-26T20:26:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.917+0000 7fc30ca00640 1 -- start start 2024-04-26T20:26:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc30ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3080772f0 0x7fc30819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc30ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078530 0x7fc30819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc30ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308078ed0 0x7fc3081a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc30ca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc308114b40 con 0x7fc308078530 2024-04-26T20:26:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc30ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081149c0 con 0x7fc3080772f0 2024-04-26T20:26:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc30ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc308114cc0 con 0x7fc308078ed0 2024-04-26T20:26:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc305400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078530 0x7fc30819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc305400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078530 0x7fc30819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57028/0 (socket says 172.21.15.131:57028) 2024-04-26T20:26:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc305400640 1 -- 172.21.15.131:0/4097519097 learned_addr learned my addr 172.21.15.131:0/4097519097 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc305e00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3080772f0 0x7fc30819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.918+0000 7fc306800640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308078ed0 0x7fc3081a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc305400640 1 -- 172.21.15.131:0/4097519097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308078ed0 msgr2=0x7fc3081a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc305400640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308078ed0 0x7fc3081a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc305400640 1 -- 172.21.15.131:0/4097519097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3080772f0 msgr2=0x7fc30819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc305400640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3080772f0 0x7fc30819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc305400640 1 -- 172.21.15.131:0/4097519097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3081a6c50 con 0x7fc308078530 2024-04-26T20:26:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc305400640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078530 0x7fc30819f570 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400e9f0 tx=0x7fc2f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc2fea00640 1 -- 172.21.15.131:0/4097519097 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f400cd80 con 0x7fc308078530 2024-04-26T20:26:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc2fea00640 1 -- 172.21.15.131:0/4097519097 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f40108e0 con 0x7fc308078530 2024-04-26T20:26:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3081a6f40 con 0x7fc308078530 2024-04-26T20:26:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc2fea00640 1 -- 172.21.15.131:0/4097519097 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4018980 con 0x7fc308078530 2024-04-26T20:26:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.919+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a7400 con 0x7fc308078530 2024-04-26T20:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.923+0000 7fc2fea00640 1 -- 172.21.15.131:0/4097519097 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc2f4010430 con 0x7fc308078530 2024-04-26T20:26:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.923+0000 7fc30f871740 1 Processor -- start 2024-04-26T20:26:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc30f871740 1 -- start start 2024-04-26T20:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc30f871740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b676bd0 0x556d1b676ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc30f871740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b67c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc30f871740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b67ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc30f871740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556d1b5db380 con 0x556d1b676bd0 2024-04-26T20:26:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc30f871740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556d1b577c30 con 0x556d1b67c730 2024-04-26T20:26:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc30f871740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556d1b5d7340 con 0x556d1b6775c0 2024-04-26T20:26:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc306800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b676bd0 0x556d1b676ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc305e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b67c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc305400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b67ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc306800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b676bd0 0x556d1b676ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57044/0 (socket says 172.21.15.131:57044) 2024-04-26T20:26:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.924+0000 7fc306800640 1 -- 172.21.15.131:0/1119589709 learned_addr learned my addr 172.21.15.131:0/1119589709 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc305e00640 1 -- 172.21.15.131:0/1119589709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 msgr2=0x556d1b67ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc305e00640 1 --2- 172.21.15.131:0/1119589709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b67ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc305e00640 1 -- 172.21.15.131:0/1119589709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b676bd0 msgr2=0x556d1b676ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc305e00640 1 --2- 172.21.15.131:0/1119589709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b676bd0 0x556d1b676ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc305e00640 1 -- 172.21.15.131:0/1119589709 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1b599940 con 0x556d1b6775c0 2024-04-26T20:26:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc2fea00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2ec0a3cf0 0x7fc2ec0a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc305e00640 1 --2- 172.21.15.131:0/1119589709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b67c0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009c20 tx=0x7fc2f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcdbf726a7123bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.925+0000 7fc2fea00640 1 -- 172.21.15.131:0/4097519097 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc2f40d0c90 con 0x7fc308078530 2024-04-26T20:26:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.926+0000 7fc306800640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2ec0a3cf0 0x7fc2ec0a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.926+0000 7fc2f3000640 1 -- 172.21.15.131:0/1119589709 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004050 con 0x556d1b6775c0 2024-04-26T20:26:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.926+0000 7fc2f3000640 1 -- 172.21.15.131:0/1119589709 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f80054d0 con 0x556d1b6775c0 2024-04-26T20:26:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.926+0000 7fc2f3000640 1 -- 172.21.15.131:0/1119589709 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004d00 con 0x556d1b6775c0 2024-04-26T20:26:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 -- 172.21.15.131:0/1119589709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 msgr2=0x556d1b67c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc306800640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2ec0a3cf0 0x7fc2ec0a61b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x556d1b677e20 tx=0x7fc300031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 --2- 172.21.15.131:0/1119589709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b67c0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009c20 tx=0x7fc2f8011810 comp rx=0 tx=0).stop 2024-04-26T20:26:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 -- 172.21.15.131:0/1119589709 shutdown_connections 2024-04-26T20:26:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 --2- 172.21.15.131:0/1119589709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b67ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 --2- 172.21.15.131:0/1119589709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b67c0c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 --2- 172.21.15.131:0/1119589709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b676bd0 0x556d1b676ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 -- 172.21.15.131:0/1119589709 >> 172.21.15.131:0/1119589709 conn(0x556d1b672240 msgr2=0x556d1b672640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 -- 172.21.15.131:0/1119589709 shutdown_connections 2024-04-26T20:26:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.927+0000 7fc30f871740 1 -- 172.21.15.131:0/1119589709 wait complete. 2024-04-26T20:26:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 -- start start 2024-04-26T20:26:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 Processor -- start 2024-04-26T20:26:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b676bd0 0x556d1b680ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b681410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b688450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556d1b67f6e0 con 0x556d1b67c730 2024-04-26T20:26:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556d1b562c60 con 0x556d1b676bd0 2024-04-26T20:26:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.928+0000 7fc30f871740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556d1b67f860 con 0x556d1b6775c0 2024-04-26T20:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc30f871740 10 client.0 _ll_register_callbacks cb 0x556d1b58ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b676bd0 0x556d1b680ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc306800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b688450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b676bd0 0x556d1b680ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:48442/0 (socket says 172.21.15.131:48442) 2024-04-26T20:26:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 -- 172.21.15.131:0/2661022745 learned_addr learned my addr 172.21.15.131:0/2661022745 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305400640 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b681410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 -- 172.21.15.131:0/2661022745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 msgr2=0x556d1b681410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b681410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 -- 172.21.15.131:0/2661022745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 msgr2=0x556d1b688450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b688450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.929+0000 7fc305e00640 1 -- 172.21.15.131:0/2661022745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1b571510 con 0x556d1b676bd0 2024-04-26T20:26:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.930+0000 7fc305e00640 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b676bd0 0x556d1b680ed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f8004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cbda8539bf76c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.930+0000 7fc2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:26:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.930+0000 7fc2e7400640 1 -- 172.21.15.131:0/2661022745 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8029070 con 0x556d1b676bd0 2024-04-26T20:26:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.930+0000 7fc2e7400640 1 -- 172.21.15.131:0/2661022745 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc2f8003800 con 0x556d1b676bd0 2024-04-26T20:26:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.930+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f80054d0 con 0x556d1b676bd0 2024-04-26T20:26:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc2e7400640 1 -- 172.21.15.131:0/2661022745 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2f8035050 con 0x556d1b676bd0 2024-04-26T20:26:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc30f871740 10 client.26624 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d1b2ed210 con 0x556d1b676bd0 2024-04-26T20:26:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc2e7400640 1 -- 172.21.15.131:0/2661022745 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f80054d0 con 0x556d1b676bd0 2024-04-26T20:26:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc2e7400640 1 -- 172.21.15.131:0/2661022745 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc2f8004fe0 con 0x556d1b676bd0 2024-04-26T20:26:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc30f871740 10 client.26624 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc30f871740 10 client.26624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.931+0000 7fc30f871740 20 client.26624 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.932+0000 7fc30f871740 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b68c6d0 0x556d1b68eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.932+0000 7fc30f871740 4 client.26624 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.932+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d1b5e8f90 con 0x556d1b68c6d0 2024-04-26T20:26:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.932+0000 7fc305400640 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b68c6d0 0x556d1b68eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.933+0000 7fc305400640 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b68c6d0 0x556d1b68eac0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed72e78eefb8e209 server_cookie=d3dc55297d48444d in_seq=0 out_seq=0 2024-04-26T20:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.933+0000 7fc2e7400640 10 client.26624 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.933+0000 7fc2e7400640 1 -- 172.21.15.131:0/2661022745 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d1b5e8f90 con 0x556d1b68c6d0 2024-04-26T20:26:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.933+0000 7fc2e7400640 10 client.26624 handle_command_reply: tid=0 2024-04-26T20:26:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.933+0000 7fc30f871740 1 client.26624 shutdown 2024-04-26T20:26:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b676bd0 msgr2=0x556d1b680ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b676bd0 0x556d1b680ed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f8004700 comp rx=0 tx=0).stop 2024-04-26T20:26:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 shutdown_connections 2024-04-26T20:26:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b68c6d0 0x556d1b68eac0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b688450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b6775c0 0x556d1b681410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 --2- 172.21.15.131:0/2661022745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b676bd0 0x556d1b680ed0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 >> 172.21.15.131:0/2661022745 conn(0x556d1b672240 msgr2=0x556d1b671440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc2e7400640 0 client.26624 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 shutdown_connections 2024-04-26T20:26:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 1 -- 172.21.15.131:0/2661022745 wait complete. 2024-04-26T20:26:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.934+0000 7fc30f871740 20 client.26624 trim_cache size 0 max 16384 2024-04-26T20:26:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 Processor -- start 2024-04-26T20:26:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 -- start start 2024-04-26T20:26:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b675920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b6775c0 0x556d1b688370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b6933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b44e0 con 0x556d1b6775c0 2024-04-26T20:26:14.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b1db0 con 0x556d1b67c730 2024-04-26T20:26:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.951+0000 7fc30f871740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081afc00 con 0x556d1b676bd0 2024-04-26T20:26:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b6933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc305400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b6775c0 0x556d1b688370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b6933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:48454/0 (socket says 172.21.15.131:48454) 2024-04-26T20:26:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 -- 172.21.15.131:0/3252045282 learned_addr learned my addr 172.21.15.131:0/3252045282 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc305e00640 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b675920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 -- 172.21.15.131:0/3252045282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 msgr2=0x556d1b675920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b675920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 -- 172.21.15.131:0/3252045282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b6775c0 msgr2=0x556d1b688370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b6775c0 0x556d1b688370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 -- 172.21.15.131:0/3252045282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1b3b9620 con 0x556d1b67c730 2024-04-26T20:26:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.952+0000 7fc306800640 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b6933c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc300035000 tx=0x7fc300041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204a696f6976eb81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.953+0000 7fc2f2600640 1 -- 172.21.15.131:0/3252045282 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300003af0 con 0x556d1b67c730 2024-04-26T20:26:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.953+0000 7fc2f2600640 1 -- 172.21.15.131:0/3252045282 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc300036050 con 0x556d1b67c730 2024-04-26T20:26:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.953+0000 7fc2f2600640 1 -- 172.21.15.131:0/3252045282 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30002f700 con 0x556d1b67c730 2024-04-26T20:26:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 -- 172.21.15.131:0/3252045282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 msgr2=0x556d1b6933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b6933c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc300035000 tx=0x7fc300041000 comp rx=0 tx=0).stop 2024-04-26T20:26:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 -- 172.21.15.131:0/3252045282 shutdown_connections 2024-04-26T20:26:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b67c730 0x556d1b6933c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b6775c0 0x556d1b688370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 --2- 172.21.15.131:0/3252045282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b675920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 -- 172.21.15.131:0/3252045282 >> 172.21.15.131:0/3252045282 conn(0x556d1b672240 msgr2=0x556d1b671440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 -- 172.21.15.131:0/3252045282 shutdown_connections 2024-04-26T20:26:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.954+0000 7fc30f871740 1 -- 172.21.15.131:0/3252045282 wait complete. 2024-04-26T20:26:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 -- start start 2024-04-26T20:26:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 Processor -- start 2024-04-26T20:26:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b685cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b6775c0 0x556d1b686210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b6871d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b0270 con 0x556d1b67c730 2024-04-26T20:26:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b16e0 con 0x556d1b6775c0 2024-04-26T20:26:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b0940 con 0x556d1b676bd0 2024-04-26T20:26:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.955+0000 7fc30f871740 10 client.0 _ll_register_callbacks cb 0x7fc2f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b6775c0 0x556d1b686210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc306800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b6871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b6775c0 0x556d1b686210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:48462/0 (socket says 172.21.15.131:48462) 2024-04-26T20:26:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 -- 172.21.15.131:0/3312266900 learned_addr learned my addr 172.21.15.131:0/3312266900 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305e00640 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b685cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 -- 172.21.15.131:0/3312266900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 msgr2=0x556d1b685cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b685cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 -- 172.21.15.131:0/3312266900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 msgr2=0x556d1b6871d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b6871d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 -- 172.21.15.131:0/3312266900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1b595cb0 con 0x556d1b6775c0 2024-04-26T20:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc305400640 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b6775c0 0x556d1b686210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4018460 tx=0x7fc2f40bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7753f6c975903564 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.956+0000 7fc2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:26:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc2e7400640 1 -- 172.21.15.131:0/3312266900 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f409ca70 con 0x556d1b6775c0 2024-04-26T20:26:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc2e7400640 1 -- 172.21.15.131:0/3312266900 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc2f4010a90 con 0x556d1b6775c0 2024-04-26T20:26:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc300036050 con 0x556d1b6775c0 2024-04-26T20:26:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc2e7400640 1 -- 172.21.15.131:0/3312266900 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2f40a0d70 con 0x556d1b6775c0 2024-04-26T20:26:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 10 client.26630 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d1b6f2c40 con 0x556d1b6775c0 2024-04-26T20:26:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc2e7400640 1 -- 172.21.15.131:0/3312266900 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc300036050 con 0x556d1b6775c0 2024-04-26T20:26:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc2e7400640 1 -- 172.21.15.131:0/3312266900 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc2f40bbb80 con 0x556d1b6775c0 2024-04-26T20:26:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 10 client.26630 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 10 client.26630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 20 client.26630 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b6f34c0 0x556d1b6f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 4 client.26630 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:26:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.957+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d1b5e2e00 con 0x556d1b6f34c0 2024-04-26T20:26:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.958+0000 7fc305e00640 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b6f34c0 0x556d1b6f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.958+0000 7fc305e00640 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b6f34c0 0x556d1b6f58e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e562db53e4d2de10 server_cookie=5352356527177a42 in_seq=0 out_seq=0 2024-04-26T20:26:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.958+0000 7fc2e7400640 10 client.26630 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:15.646 INFO:tasks.workunit.client.0.smithi131.stdout:test read from hole 2024-04-26T20:26:15.658 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test read from hole' 2024-04-26T20:26:15.667 INFO:tasks.workunit.client.0.smithi131.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-04-26T20:26:15.676 INFO:tasks.workunit.client.0.smithi131.stderr:0+0 records in 2024-04-26T20:26:15.735 INFO:tasks.workunit.client.0.smithi131.stderr:0+0 records out 2024-04-26T20:26:15.744 INFO:tasks.workunit.client.0.smithi131.stderr:0 bytes copied, 7.9109e-05 s, 0.0 kB/s 2024-04-26T20:26:15.755 INFO:tasks.workunit.client.0.smithi131.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-04-26T20:26:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.971+0000 7fc2e7400640 1 -- 172.21.15.131:0/3312266900 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d1b5e2e00 con 0x556d1b6f34c0 2024-04-26T20:26:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.971+0000 7fc2e7400640 10 client.26630 handle_command_reply: tid=0 2024-04-26T20:26:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.971+0000 7fc30f871740 1 client.26630 shutdown 2024-04-26T20:26:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.971+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b6775c0 msgr2=0x556d1b686210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.971+0000 7fc30f871740 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b6775c0 0x556d1b686210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4018460 tx=0x7fc2f40bd040 comp rx=0 tx=0).stop 2024-04-26T20:26:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.971+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 shutdown_connections 2024-04-26T20:26:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.971+0000 7fc30f871740 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x556d1b6f34c0 0x556d1b6f58e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc30f871740 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d1b67c730 0x556d1b6871d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc30f871740 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d1b6775c0 0x556d1b686210 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc2e7400640 0 client.26630 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc30f871740 1 --2- 172.21.15.131:0/3312266900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d1b676bd0 0x556d1b685cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 >> 172.21.15.131:0/3312266900 conn(0x556d1b672240 msgr2=0x556d1b680490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 shutdown_connections 2024-04-26T20:26:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc30f871740 1 -- 172.21.15.131:0/3312266900 wait complete. 2024-04-26T20:26:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.972+0000 7fc30f871740 20 client.26630 trim_cache size 0 max 16384 2024-04-26T20:26:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:26:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:26:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:26:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:26:16.156 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:12 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3337188632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:26:16.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:13 smithi148 ceph-mon[32281]: pgmap v896: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 KiB/s wr, 457 op/s 2024-04-26T20:26:16.273 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:15 smithi148 ceph-mon[32281]: pgmap v897: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 KiB/s wr, 459 op/s 2024-04-26T20:26:16.280 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:11 smithi157 ceph-mon[32703]: pgmap v895: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 38 KiB/s wr, 721 op/s 2024-04-26T20:26:16.287 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:12 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3337188632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:26:16.295 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:13 smithi157 ceph-mon[32703]: pgmap v896: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 KiB/s wr, 457 op/s 2024-04-26T20:26:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2ec0a3cf0 msgr2=0x7fc2ec0a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2ec0a3cf0 0x7fc2ec0a61b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x556d1b677e20 tx=0x7fc300031af0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078530 msgr2=0x7fc30819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078530 0x7fc30819f570 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400e9f0 tx=0x7fc2f400eeb0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 shutdown_connections 2024-04-26T20:26:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc2ec0a3cf0 0x7fc2ec0a61b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308078ed0 0x7fc3081a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308078530 0x7fc30819f570 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 --2- 172.21.15.131:0/4097519097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3080772f0 0x7fc30819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 >> 172.21.15.131:0/4097519097 conn(0x7fc308100150 msgr2=0x7fc308101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 shutdown_connections 2024-04-26T20:26:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:11.973+0000 7fc30ca00640 1 -- 172.21.15.131:0/4097519097 wait complete. 2024-04-26T20:26:16.528 INFO:tasks.workunit.client.0.smithi131.stderr:2+0 records in 2024-04-26T20:26:16.599 INFO:tasks.workunit.client.0.smithi131.stderr:2+0 records out 2024-04-26T20:26:16.608 INFO:tasks.workunit.client.0.smithi131.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00106613 s, 960 kB/s 2024-04-26T20:26:16.618 INFO:tasks.workunit.client.0.smithi131.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-04-26T20:26:16.628 INFO:tasks.workunit.client.0.smithi131.stderr:2+0 records in 2024-04-26T20:26:16.669 INFO:tasks.workunit.client.0.smithi131.stderr:2+0 records out 2024-04-26T20:26:16.682 INFO:tasks.workunit.client.0.smithi131.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 9.9321e-05 s, 10.3 MB/s 2024-04-26T20:26:16.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-04-26T20:26:16.701 INFO:tasks.workunit.client.0.smithi131.stdout:other thing 2024-04-26T20:26:16.714 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'other thing' 2024-04-26T20:26:16.725 INFO:tasks.workunit.client.0.smithi131.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-04-26T20:26:16.778 INFO:tasks.workunit.client.0.smithi131.stderr:1+0 records in 2024-04-26T20:26:16.790 INFO:tasks.workunit.client.0.smithi131.stderr:1+0 records out 2024-04-26T20:26:16.802 INFO:tasks.workunit.client.0.smithi131.stderr:500 bytes copied, 0.000109209 s, 4.6 MB/s 2024-04-26T20:26:16.813 INFO:tasks.workunit.client.0.smithi131.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-04-26T20:26:16.824 INFO:tasks.workunit.client.0.smithi131.stderr:0+1 records in 2024-04-26T20:26:16.835 INFO:tasks.workunit.client.0.smithi131.stderr:0+1 records out 2024-04-26T20:26:16.880 INFO:tasks.workunit.client.0.smithi131.stderr:500 bytes copied, 0.00606484 s, 82.4 kB/s 2024-04-26T20:26:16.958 INFO:tasks.workunit.client.0.smithi131.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-04-26T20:26:16.971 INFO:tasks.workunit.client.0.smithi131.stderr:0+1 records in 2024-04-26T20:26:16.985 INFO:tasks.workunit.client.0.smithi131.stderr:0+1 records out 2024-04-26T20:26:17.000 INFO:tasks.workunit.client.0.smithi131.stderr:500 bytes copied, 0.00656412 s, 76.2 kB/s 2024-04-26T20:26:17.061 INFO:tasks.workunit.client.0.smithi131.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-04-26T20:26:17.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:15 smithi157 ceph-mon[32703]: pgmap v897: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 KiB/s wr, 459 op/s 2024-04-26T20:26:17.082 INFO:tasks.workunit.client.0.smithi131.stdout:OK 2024-04-26T20:26:17.093 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo OK 2024-04-26T20:26:17.100 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T20:26:17.111 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T20:26:17.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:11 smithi131 ceph-mon[27919]: pgmap v895: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 38 KiB/s wr, 721 op/s 2024-04-26T20:26:17.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 -- 172.21.15.131:0/3904180717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5278109f80 msgr2=0x7f527810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 --2- 172.21.15.131:0/3904180717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5278109f80 0x7f527810a400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5264009900 tx=0x7f526402a410 comp rx=0 tx=0).stop 2024-04-26T20:26:17.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 -- 172.21.15.131:0/3904180717 shutdown_connections 2024-04-26T20:26:17.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 --2- 172.21.15.131:0/3904180717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f527810a940 0x7f52781111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:17.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 --2- 172.21.15.131:0/3904180717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5278109f80 0x7f527810a400 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 --2- 172.21.15.131:0/3904180717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278108d90 0x7f52781091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 -- 172.21.15.131:0/3904180717 >> 172.21.15.131:0/3904180717 conn(0x7f5278077d30 msgr2=0x7f527807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:17.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 -- 172.21.15.131:0/3904180717 shutdown_connections 2024-04-26T20:26:17.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.091+0000 7f527e400640 1 -- 172.21.15.131:0/3904180717 wait complete. 2024-04-26T20:26:17.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 Processor -- start 2024-04-26T20:26:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 -- start start 2024-04-26T20:26:17.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278108d90 0x7f527819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:17.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278109f80 0x7f527819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:17.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527810a940 0x7f52781a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:17.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5278114910 con 0x7f527810a940 2024-04-26T20:26:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5278114790 con 0x7f5278109f80 2024-04-26T20:26:17.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.092+0000 7f527e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5278114a90 con 0x7f5278108d90 2024-04-26T20:26:17.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278108d90 0x7f527819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:17.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527810a940 0x7f52781a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:17.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278108d90 0x7f527819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33770/0 (socket says 172.21.15.131:33770) 2024-04-26T20:26:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527d000640 1 -- 172.21.15.131:0/3337188632 learned_addr learned my addr 172.21.15.131:0/3337188632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:17.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f5277e00640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278109f80 0x7f527819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:17.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527da00640 1 -- 172.21.15.131:0/3337188632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278108d90 msgr2=0x7f527819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:17.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527da00640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278108d90 0x7f527819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:17.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527da00640 1 -- 172.21.15.131:0/3337188632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278109f80 msgr2=0x7f527819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:17.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527da00640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278109f80 0x7f527819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:17.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527da00640 1 -- 172.21.15.131:0/3337188632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52781a6bb0 con 0x7f527810a940 2024-04-26T20:26:17.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f527d000640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278108d90 0x7f527819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.093+0000 7f5277e00640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278109f80 0x7f527819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:17.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.094+0000 7f527da00640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527810a940 0x7f52781a6540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f527000e940 tx=0x7f527000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:17.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.094+0000 7f5275600640 1 -- 172.21.15.131:0/3337188632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270004d20 con 0x7f527810a940 2024-04-26T20:26:17.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.094+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52781a6ea0 con 0x7f527810a940 2024-04-26T20:26:17.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.094+0000 7f5275600640 1 -- 172.21.15.131:0/3337188632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5270005020 con 0x7f527810a940 2024-04-26T20:26:17.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.094+0000 7f5275600640 1 -- 172.21.15.131:0/3337188632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700116d0 con 0x7f527810a940 2024-04-26T20:26:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.094+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52781a73b0 con 0x7f527810a940 2024-04-26T20:26:17.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.095+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52781100b0 con 0x7f527810a940 2024-04-26T20:26:17.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.097+0000 7f5275600640 1 -- 172.21.15.131:0/3337188632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5270005460 con 0x7f527810a940 2024-04-26T20:26:17.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.098+0000 7f5275600640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52600a3c40 0x7f52600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:17.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.099+0000 7f5275600640 1 -- 172.21.15.131:0/3337188632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f52700d0d30 con 0x7f527810a940 2024-04-26T20:26:17.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.099+0000 7f527d000640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52600a3c40 0x7f52600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:17.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.100+0000 7f527d000640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52600a3c40 0x7f52600a6100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f526c0058c0 tx=0x7f526c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.104+0000 7f5275600640 1 -- 172.21.15.131:0/3337188632 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5270016030 con 0x7f527810a940 2024-04-26T20:26:18.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.340+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5278063530 con 0x7f527810a940 2024-04-26T20:26:18.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.342+0000 7f5275600640 1 -- 172.21.15.131:0/3337188632 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7f527009cb10 con 0x7f527810a940 2024-04-26T20:26:18.061 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:26:18.068 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:12:47.696526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24571,24557]},"id":1}]} 2024-04-26T20:26:18.078 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:17 smithi148 ceph-mon[32281]: pgmap v898: 129 pgs: 129 active+clean; 884 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 KiB/s wr, 456 op/s 2024-04-26T20:26:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-04-26T20:26:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52600a3c40 msgr2=0x7f52600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:18.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52600a3c40 0x7f52600a6100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f526c0058c0 tx=0x7f526c005f90 comp rx=0 tx=0).stop 2024-04-26T20:26:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527810a940 msgr2=0x7f52781a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:18.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527810a940 0x7f52781a6540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f527000e940 tx=0x7f527000ee00 comp rx=0 tx=0).stop 2024-04-26T20:26:18.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 shutdown_connections 2024-04-26T20:26:18.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527810a940 0x7f52781a6540 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278109f80 0x7f527819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f52600a3c40 0x7f52600a6100 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:18.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 --2- 172.21.15.131:0/3337188632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278108d90 0x7f527819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:18.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.344+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 >> 172.21.15.131:0/3337188632 conn(0x7f5278077d30 msgr2=0x7f527810d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:18.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.345+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 shutdown_connections 2024-04-26T20:26:18.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:12.345+0000 7f527e400640 1 -- 172.21.15.131:0/3337188632 wait complete. 2024-04-26T20:26:18.370 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T20:26:18.428 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:12 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3337188632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:26:18.437 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:13 smithi131 ceph-mon[27919]: pgmap v896: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 KiB/s wr, 457 op/s 2024-04-26T20:26:18.446 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:15 smithi131 ceph-mon[27919]: pgmap v897: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 KiB/s wr, 459 op/s 2024-04-26T20:26:18.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:17 smithi131 ceph-mon[27919]: pgmap v898: 129 pgs: 129 active+clean; 884 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 KiB/s wr, 456 op/s 2024-04-26T20:26:18.468 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:17 smithi157 ceph-mon[32703]: pgmap v898: 129 pgs: 129 active+clean; 884 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 KiB/s wr, 456 op/s 2024-04-26T20:26:18.529 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-04-26T20:26:18.537 DEBUG:teuthology.orchestra.run.smithi131:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-04-26T20:26:18.641 INFO:tasks.workunit.client.0.smithi131.stdout:writing first 3 bytes of 10k file 2024-04-26T20:26:18.647 INFO:tasks.workunit.client.0.smithi131.stdout:reading O_DIRECT 2024-04-26T20:26:18.659 INFO:tasks.workunit.client.0.smithi131.stdout:got 10000 2024-04-26T20:26:18.670 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T20:26:18.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T20:26:18.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 -- 172.21.15.131:0/1865289427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0104b90 msgr2=0x7fe3e0104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:18.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 --2- 172.21.15.131:0/1865289427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0104b90 0x7fe3e0104fb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d002a410 comp rx=0 tx=0).stop 2024-04-26T20:26:18.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 -- 172.21.15.131:0/1865289427 shutdown_connections 2024-04-26T20:26:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 --2- 172.21.15.131:0/1865289427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0106740 0x7fe3e010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 --2- 172.21.15.131:0/1865289427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e0105d80 0x7fe3e0106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:18.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 --2- 172.21.15.131:0/1865289427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0104b90 0x7fe3e0104fb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:18.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 -- 172.21.15.131:0/1865289427 >> 172.21.15.131:0/1865289427 conn(0x7fe3e0100120 msgr2=0x7fe3e0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:18.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 -- 172.21.15.131:0/1865289427 shutdown_connections 2024-04-26T20:26:18.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.669+0000 7fe3e5000640 1 -- 172.21.15.131:0/1865289427 wait complete. 2024-04-26T20:26:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.670+0000 7fe3e5000640 1 Processor -- start 2024-04-26T20:26:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.670+0000 7fe3e5000640 1 -- start start 2024-04-26T20:26:19.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3e5000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0104b90 0x7fe3e019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3e5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e0105d80 0x7fe3e019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3e5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0106740 0x7fe3e01a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3e5000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01106b0 con 0x7fe3e0106740 2024-04-26T20:26:19.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3e5000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0110530 con 0x7fe3e0104b90 2024-04-26T20:26:19.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3e5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0110830 con 0x7fe3e0105d80 2024-04-26T20:26:19.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3de800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0106740 0x7fe3e01a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0104b90 0x7fe3e019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0104b90 0x7fe3e019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55122/0 (socket says 172.21.15.131:55122) 2024-04-26T20:26:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3d5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e0105d80 0x7fe3e019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.671+0000 7fe3dde00640 1 -- 172.21.15.131:0/2022268128 learned_addr learned my addr 172.21.15.131:0/2022268128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:19.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3de800640 1 -- 172.21.15.131:0/2022268128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e0105d80 msgr2=0x7fe3e019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3de800640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e0105d80 0x7fe3e019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3de800640 1 -- 172.21.15.131:0/2022268128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0104b90 msgr2=0x7fe3e019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3de800640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0104b90 0x7fe3e019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3de800640 1 -- 172.21.15.131:0/2022268128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e01a2a40 con 0x7fe3e0106740 2024-04-26T20:26:19.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3dde00640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0104b90 0x7fe3e019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:19.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3de800640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0106740 0x7fe3e01a23d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009fd0 tx=0x7fe3d800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3d7400640 1 -- 172.21.15.131:0/2022268128 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800ce20 con 0x7fe3e0106740 2024-04-26T20:26:19.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01a2d30 con 0x7fe3e0106740 2024-04-26T20:26:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3d7400640 1 -- 172.21.15.131:0/2022268128 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d8005020 con 0x7fe3e0106740 2024-04-26T20:26:19.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3d7400640 1 -- 172.21.15.131:0/2022268128 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8010690 con 0x7fe3e0106740 2024-04-26T20:26:19.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.672+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01a31d0 con 0x7fe3e0106740 2024-04-26T20:26:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3d7400640 1 -- 172.21.15.131:0/2022268128 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe3d80054d0 con 0x7fe3e0106740 2024-04-26T20:26:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 Processor -- start 2024-04-26T20:26:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 -- start start 2024-04-26T20:26:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x5644321fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321fe890 0x564432203390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564432203a00 0x564432205e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564432161c70 con 0x5644321fdea0 2024-04-26T20:26:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56443211a980 con 0x5644321fe890 2024-04-26T20:26:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.676+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56443215dc30 con 0x564432203a00 2024-04-26T20:26:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321fe890 0x564432203390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3d5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x5644321fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321fe890 0x564432203390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55124/0 (socket says 172.21.15.131:55124) 2024-04-26T20:26:19.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 -- 172.21.15.131:0/1276014570 learned_addr learned my addr 172.21.15.131:0/1276014570 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:19.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 -- 172.21.15.131:0/1276014570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564432203a00 msgr2=0x564432205e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:19.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 --2- 172.21.15.131:0/1276014570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564432203a00 0x564432205e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 -- 172.21.15.131:0/1276014570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 msgr2=0x5644321fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 --2- 172.21.15.131:0/1276014570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x5644321fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 -- 172.21.15.131:0/1276014570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644320f27b0 con 0x5644321fe890 2024-04-26T20:26:19.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3dde00640 1 --2- 172.21.15.131:0/1276014570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321fe890 0x564432203390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005e20 tx=0x7fe3d0005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a397b92f6da14994 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.677+0000 7fe3d7400640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3c40a3c40 0x7fe3c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:19.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.678+0000 7fe3d7400640 1 -- 172.21.15.131:0/2022268128 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe3d80d10a0 con 0x7fe3e0106740 2024-04-26T20:26:19.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.678+0000 7fe3d5400640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3c40a3c40 0x7fe3c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:19.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.678+0000 7fe3cf400640 1 -- 172.21.15.131:0/1276014570 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0040070 con 0x5644321fe890 2024-04-26T20:26:19.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.678+0000 7fe3cf400640 1 -- 172.21.15.131:0/1276014570 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d0005040 con 0x5644321fe890 2024-04-26T20:26:19.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.678+0000 7fe3cf400640 1 -- 172.21.15.131:0/1276014570 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0035040 con 0x5644321fe890 2024-04-26T20:26:19.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1276014570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321fe890 msgr2=0x564432203390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3d5400640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3c40a3c40 0x7fe3c40a6100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009710 tx=0x7fe3c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:19.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1276014570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321fe890 0x564432203390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005e20 tx=0x7fe3d0005e50 comp rx=0 tx=0).stop 2024-04-26T20:26:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1276014570 shutdown_connections 2024-04-26T20:26:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1276014570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564432203a00 0x564432205e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1276014570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321fe890 0x564432203390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1276014570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x5644321fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1276014570 >> 172.21.15.131:0/1276014570 conn(0x5644321f9510 msgr2=0x5644321f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:20.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1276014570 shutdown_connections 2024-04-26T20:26:20.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1276014570 wait complete. 2024-04-26T20:26:20.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.679+0000 7fe3e7fdf740 1 -- start start 2024-04-26T20:26:20.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 1 Processor -- start 2024-04-26T20:26:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x564432207d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x564432208270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564432203a00 0x56443220f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564432160f80 con 0x5644321fdea0 2024-04-26T20:26:20.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56443210e260 con 0x564432203a00 2024-04-26T20:26:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5644321662a0 con 0x5644321fe890 2024-04-26T20:26:20.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3e7fdf740 10 client.0 _ll_register_callbacks cb 0x564432158960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:20.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x564432207d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3de800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564432203a00 0x56443220f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3de800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564432203a00 0x56443220f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55138/0 (socket says 172.21.15.131:55138) 2024-04-26T20:26:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.680+0000 7fe3de800640 1 -- 172.21.15.131:0/1346915405 learned_addr learned my addr 172.21.15.131:0/1346915405 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3d5400640 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x564432208270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3de800640 1 -- 172.21.15.131:0/1346915405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 msgr2=0x564432208270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3de800640 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x564432208270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3de800640 1 -- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 msgr2=0x564432207d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:20.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3de800640 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x564432207d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:20.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3de800640 1 -- 172.21.15.131:0/1346915405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644320f01c0 con 0x564432203a00 2024-04-26T20:26:20.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3dde00640 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x564432207d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:20.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3de800640 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564432203a00 0x56443220f2b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80a5510 tx=0x7fe3d809cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18040e4204783f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:20.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:26:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.681+0000 7fe3c3e00640 1 -- 172.21.15.131:0/1346915405 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80b6260 con 0x564432203a00 2024-04-26T20:26:20.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d0005040 con 0x564432203a00 2024-04-26T20:26:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3c3e00640 1 -- 172.21.15.131:0/1346915405 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe3d80051d0 con 0x564432203a00 2024-04-26T20:26:20.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3c3e00640 1 -- 172.21.15.131:0/1346915405 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3d80a55b0 con 0x564432203a00 2024-04-26T20:26:20.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 10 client.26648 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:20.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564431e733f0 con 0x564432203a00 2024-04-26T20:26:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3c3e00640 1 -- 172.21.15.131:0/1346915405 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d0005040 con 0x564432203a00 2024-04-26T20:26:20.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3c3e00640 1 -- 172.21.15.131:0/1346915405 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe3d809bc90 con 0x564432203a00 2024-04-26T20:26:20.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 10 client.26648 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:20.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 10 client.26648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-26T20:26:20.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 20 client.26648 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:20.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5644322137f0 0x564432215c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:20.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 4 client.26648 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.682+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56443216f880 con 0x5644322137f0 2024-04-26T20:26:21.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.683+0000 7fe3dde00640 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5644322137f0 0x564432215c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.683+0000 7fe3dde00640 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5644322137f0 0x564432215c90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61c2e69afe5aeea server_cookie=14ff648cc2478ef3 in_seq=0 out_seq=0 2024-04-26T20:26:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.683+0000 7fe3c3e00640 10 client.26648 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:26:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3c3e00640 1 -- 172.21.15.131:0/1346915405 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56443216f880 con 0x5644322137f0 2024-04-26T20:26:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3c3e00640 10 client.26648 handle_command_reply: tid=0 2024-04-26T20:26:21.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 client.26648 shutdown 2024-04-26T20:26:21.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564432203a00 msgr2=0x56443220f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564432203a00 0x56443220f2b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80a5510 tx=0x7fe3d809cf10 comp rx=0 tx=0).stop 2024-04-26T20:26:21.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 shutdown_connections 2024-04-26T20:26:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x5644322137f0 0x564432215c90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564432203a00 0x56443220f2b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x564432208270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3c3e00640 0 client.26648 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:26:21.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/1346915405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x564432207d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 >> 172.21.15.131:0/1346915405 conn(0x5644321f9510 msgr2=0x5644321f8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 shutdown_connections 2024-04-26T20:26:21.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.684+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/1346915405 wait complete. 2024-04-26T20:26:21.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.685+0000 7fe3e7fdf740 20 client.26648 trim_cache size 0 max 16384 2024-04-26T20:26:21.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 Processor -- start 2024-04-26T20:26:21.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 -- start start 2024-04-26T20:26:21.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x5644321fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x56443220f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x5644321fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:21.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01b0160 con 0x5644321fdea0 2024-04-26T20:26:21.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01ada30 con 0x5644321f9510 2024-04-26T20:26:21.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.699+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01ab880 con 0x5644321fe890 2024-04-26T20:26:21.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x56443220f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x56443220f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46818/0 (socket says 172.21.15.131:46818) 2024-04-26T20:26:21.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 -- 172.21.15.131:0/2476348508 learned_addr learned my addr 172.21.15.131:0/2476348508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3dde00640 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x5644321fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3de800640 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x5644321fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:21.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 -- 172.21.15.131:0/2476348508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 msgr2=0x5644321fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:21.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x5644321fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 -- 172.21.15.131:0/2476348508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 msgr2=0x5644321fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:21.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x5644321fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 -- 172.21.15.131:0/2476348508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564431f3d7d0 con 0x5644321fdea0 2024-04-26T20:26:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.700+0000 7fe3d5400640 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x56443220f270 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80058c0 tx=0x7fe3c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e202a6e8b140c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:21.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3cea00640 1 -- 172.21.15.131:0/2476348508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8004450 con 0x5644321fdea0 2024-04-26T20:26:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3cea00640 1 -- 172.21.15.131:0/2476348508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c802d070 con 0x5644321fdea0 2024-04-26T20:26:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3cea00640 1 -- 172.21.15.131:0/2476348508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8028400 con 0x5644321fdea0 2024-04-26T20:26:21.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/2476348508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 msgr2=0x56443220f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:21.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x56443220f270 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80058c0 tx=0x7fe3c8017000 comp rx=0 tx=0).stop 2024-04-26T20:26:21.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/2476348508 shutdown_connections 2024-04-26T20:26:21.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fe890 0x5644321fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fdea0 0x56443220f270 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/2476348508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x5644321fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:21.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/2476348508 >> 172.21.15.131:0/2476348508 conn(0x564432206f40 msgr2=0x5644321fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:21.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.701+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/2476348508 shutdown_connections 2024-04-26T20:26:21.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/2476348508 wait complete. 2024-04-26T20:26:21.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 -- start start 2024-04-26T20:26:21.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 Processor -- start 2024-04-26T20:26:22.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x56443220c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fdea0 0x56443220cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fe890 0x56443220dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01abef0 con 0x5644321fe890 2024-04-26T20:26:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01ad360 con 0x5644321f9510 2024-04-26T20:26:22.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.702+0000 7fe3e7fdf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01ac5c0 con 0x5644321fdea0 2024-04-26T20:26:22.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3e7fdf740 10 client.0 _ll_register_callbacks cb 0x7fe3d8099070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fe890 0x56443220dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x56443220c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fe890 0x56443220dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46828/0 (socket says 172.21.15.131:46828) 2024-04-26T20:26:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x56443220c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55144/0 (socket says 172.21.15.131:55144) 2024-04-26T20:26:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 -- 172.21.15.131:0/4268582997 learned_addr learned my addr 172.21.15.131:0/4268582997 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:22.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3d5400640 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fdea0 0x56443220cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 -- 172.21.15.131:0/4268582997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fdea0 msgr2=0x56443220cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fdea0 0x56443220cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 -- 172.21.15.131:0/4268582997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 msgr2=0x56443220c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x56443220c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 -- 172.21.15.131:0/4268582997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644321082a0 con 0x5644321fe890 2024-04-26T20:26:22.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3dde00640 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x56443220c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:22.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3de800640 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fe890 0x56443220dc70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809b080 tx=0x7fe3d8098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81dcb07f784925c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.703+0000 7fe3c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:26:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3c3e00640 1 -- 172.21.15.131:0/4268582997 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8098b20 con 0x5644321fe890 2024-04-26T20:26:22.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3c3e00640 1 -- 172.21.15.131:0/4268582997 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fe3d8098cc0 con 0x5644321fe890 2024-04-26T20:26:22.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c802d070 con 0x5644321fe890 2024-04-26T20:26:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3c3e00640 1 -- 172.21.15.131:0/4268582997 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3d8086120 con 0x5644321fe890 2024-04-26T20:26:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 10 client.16536 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:22.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56443220f400 con 0x5644321fe890 2024-04-26T20:26:22.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3c3e00640 1 -- 172.21.15.131:0/4268582997 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c802d070 con 0x5644321fe890 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3c3e00640 1 -- 172.21.15.131:0/4268582997 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe3d80b6060 con 0x5644321fe890 2024-04-26T20:26:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 10 client.16536 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 10 client.16536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-26T20:26:22.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 20 client.16536 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:22.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x564432203a00 0x56443227c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.704+0000 7fe3e7fdf740 4 client.16536 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:26:22.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.705+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644321696f0 con 0x564432203a00 2024-04-26T20:26:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.705+0000 7fe3dde00640 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x564432203a00 0x56443227c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.705+0000 7fe3dde00640 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x564432203a00 0x56443227c250 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4aa26a74afab005 server_cookie=250f7ad722563076 in_seq=0 out_seq=0 2024-04-26T20:26:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.705+0000 7fe3c3e00640 10 client.16536 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:26:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.705+0000 7fe3c3e00640 1 -- 172.21.15.131:0/4268582997 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644321696f0 con 0x564432203a00 2024-04-26T20:26:22.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3c3e00640 10 client.16536 handle_command_reply: tid=0 2024-04-26T20:26:22.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 client.16536 shutdown 2024-04-26T20:26:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fe890 msgr2=0x56443220dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fe890 0x56443220dc70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809b080 tx=0x7fe3d8098f50 comp rx=0 tx=0).stop 2024-04-26T20:26:22.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 shutdown_connections 2024-04-26T20:26:22.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x564432203a00 0x56443227c250 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644321fe890 0x56443220dc70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644321fdea0 0x56443220cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 --2- 172.21.15.131:0/4268582997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644321f9510 0x56443220c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3c3e00640 0 client.16536 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:26:22.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 >> 172.21.15.131:0/4268582997 conn(0x564432206f40 msgr2=0x5644321fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 shutdown_connections 2024-04-26T20:26:22.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 1 -- 172.21.15.131:0/4268582997 wait complete. 2024-04-26T20:26:22.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.706+0000 7fe3e7fdf740 20 client.16536 trim_cache size 0 max 16384 2024-04-26T20:26:23.001 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:26:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3c40a3c40 msgr2=0x7fe3c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3c40a3c40 0x7fe3c40a6100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009710 tx=0x7fe3c8005850 comp rx=0 tx=0).stop 2024-04-26T20:26:23.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0106740 msgr2=0x7fe3e01a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0106740 0x7fe3e01a23d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009fd0 tx=0x7fe3d800ef10 comp rx=0 tx=0).stop 2024-04-26T20:26:23.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 shutdown_connections 2024-04-26T20:26:23.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3e0106740 0x7fe3e01a23d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe3c40a3c40 0x7fe3c40a6100 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e0105d80 0x7fe3e019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 --2- 172.21.15.131:0/2022268128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3e0104b90 0x7fe3e019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 >> 172.21.15.131:0/2022268128 conn(0x7fe3e0100120 msgr2=0x7fe3e01091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 shutdown_connections 2024-04-26T20:26:23.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:18.707+0000 7fe3e5000640 1 -- 172.21.15.131:0/2022268128 wait complete. 2024-04-26T20:26:23.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T20:26:23.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:19 smithi131 ceph-mon[27919]: pgmap v899: 129 pgs: 129 active+clean; 891 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 6.2 KiB/s wr, 98 op/s 2024-04-26T20:26:23.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:21 smithi131 ceph-mon[27919]: pgmap v900: 129 pgs: 129 active+clean; 891 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 4.7 KiB/s wr, 95 op/s 2024-04-26T20:26:23.235 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:19 smithi157 ceph-mon[32703]: pgmap v899: 129 pgs: 129 active+clean; 891 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 6.2 KiB/s wr, 98 op/s 2024-04-26T20:26:23.267 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:21 smithi157 ceph-mon[32703]: pgmap v900: 129 pgs: 129 active+clean; 891 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 4.7 KiB/s wr, 95 op/s 2024-04-26T20:26:23.274 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:19 smithi148 ceph-mon[32281]: pgmap v899: 129 pgs: 129 active+clean; 891 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 6.2 KiB/s wr, 98 op/s 2024-04-26T20:26:23.281 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:21 smithi148 ceph-mon[32281]: pgmap v900: 129 pgs: 129 active+clean; 891 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 4.7 KiB/s wr, 95 op/s 2024-04-26T20:26:23.289 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-04-26T20:26:23.297 DEBUG:teuthology.orchestra.run.smithi131:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-04-26T20:26:23.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.523+0000 7fbbc7e00640 1 -- 172.21.15.131:0/4260379515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 msgr2=0x7fbbc810aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.523+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/4260379515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 0x7fbbc810aca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009860 tx=0x7fbbb802a340 comp rx=0 tx=0).stop 2024-04-26T20:26:23.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.524+0000 7fbbc7e00640 1 -- 172.21.15.131:0/4260379515 shutdown_connections 2024-04-26T20:26:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.524+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/4260379515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 0x7fbbc810aca0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.524+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/4260379515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc8103a50 0x7fbbc8103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.524+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/4260379515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 0x7fbbc8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.524+0000 7fbbc7e00640 1 -- 172.21.15.131:0/4260379515 >> 172.21.15.131:0/4260379515 conn(0x7fbbc80fddd0 msgr2=0x7fbbc8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.524+0000 7fbbc7e00640 1 -- 172.21.15.131:0/4260379515 shutdown_connections 2024-04-26T20:26:23.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.524+0000 7fbbc7e00640 1 -- 172.21.15.131:0/4260379515 wait complete. 2024-04-26T20:26:23.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 Processor -- start 2024-04-26T20:26:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 -- start start 2024-04-26T20:26:23.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 0x7fbbc819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc8103a50 0x7fbbc819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 0x7fbbc81a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc810e370 con 0x7fbbc8102860 2024-04-26T20:26:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc810e1f0 con 0x7fbbc8103a50 2024-04-26T20:26:23.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.525+0000 7fbbc7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc810e4f0 con 0x7fbbc8104410 2024-04-26T20:26:23.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 0x7fbbc819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 0x7fbbc819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46852/0 (socket says 172.21.15.131:46852) 2024-04-26T20:26:23.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 -- 172.21.15.131:0/2276014450 learned_addr learned my addr 172.21.15.131:0/2276014450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:23.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc7400640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 0x7fbbc81a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6000640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc8103a50 0x7fbbc819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 -- 172.21.15.131:0/2276014450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 msgr2=0x7fbbc81a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 0x7fbbc81a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 -- 172.21.15.131:0/2276014450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc8103a50 msgr2=0x7fbbc819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc8103a50 0x7fbbc819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.526+0000 7fbbc6a00640 1 -- 172.21.15.131:0/2276014450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc81a2830 con 0x7fbbc8102860 2024-04-26T20:26:24.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.527+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 0x7fbbc819ac20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c3a0 tx=0x7fbbc000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.527+0000 7fbbbf400640 1 -- 172.21.15.131:0/2276014450 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0003970 con 0x7fbbc8102860 2024-04-26T20:26:24.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.527+0000 7fbbbf400640 1 -- 172.21.15.131:0/2276014450 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc0004020 con 0x7fbbc8102860 2024-04-26T20:26:24.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.527+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc81a2b20 con 0x7fbbc8102860 2024-04-26T20:26:24.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.527+0000 7fbbbf400640 1 -- 172.21.15.131:0/2276014450 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000b920 con 0x7fbbc8102860 2024-04-26T20:26:24.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.527+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81a3070 con 0x7fbbc8102860 2024-04-26T20:26:24.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.530+0000 7fbbd0369740 1 Processor -- start 2024-04-26T20:26:24.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.530+0000 7fbbd0369740 1 -- start start 2024-04-26T20:26:24.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfe3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbfe8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe8bb0 0x5619fbfeb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbd0369740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5619fbf46c70 con 0x5619fbfe3050 2024-04-26T20:26:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbd0369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5619fbeff980 con 0x5619fbfe3a40 2024-04-26T20:26:24.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbd0369740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5619fbf42c30 con 0x5619fbfe8bb0 2024-04-26T20:26:24.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfe3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfe3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46862/0 (socket says 172.21.15.131:46862) 2024-04-26T20:26:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 -- 172.21.15.131:0/1309198091 learned_addr learned my addr 172.21.15.131:0/1309198091 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:24.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc6000640 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe8bb0 0x5619fbfeb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 -- 172.21.15.131:0/1309198091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe8bb0 msgr2=0x5619fbfeb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe8bb0 0x5619fbfeb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 -- 172.21.15.131:0/1309198091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 msgr2=0x5619fbfe8540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:26:24.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbfe8540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.531+0000 7fbbc7400640 1 -- 172.21.15.131:0/1309198091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fbed77b0 con 0x5619fbfe3050 2024-04-26T20:26:24.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.532+0000 7fbbc7400640 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfe3470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009830 tx=0x7fbbb8005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0c79ee8a01ff77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.532+0000 7fbbbf400640 1 -- 172.21.15.131:0/2276014450 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbbc001d020 con 0x7fbbc8102860 2024-04-26T20:26:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.532+0000 7fbbb7400640 1 -- 172.21.15.131:0/1309198091 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8037040 con 0x5619fbfe3050 2024-04-26T20:26:24.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.532+0000 7fbbb7400640 1 -- 172.21.15.131:0/1309198091 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004030 con 0x5619fbfe3050 2024-04-26T20:26:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.532+0000 7fbbb7400640 1 -- 172.21.15.131:0/1309198091 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80039b0 con 0x5619fbfe3050 2024-04-26T20:26:24.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 -- 172.21.15.131:0/1309198091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 msgr2=0x5619fbfe3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfe3470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009830 tx=0x7fbbb8005b20 comp rx=0 tx=0).stop 2024-04-26T20:26:24.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 -- 172.21.15.131:0/1309198091 shutdown_connections 2024-04-26T20:26:24.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe8bb0 0x5619fbfeb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbfe8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 --2- 172.21.15.131:0/1309198091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfe3470 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 -- 172.21.15.131:0/1309198091 >> 172.21.15.131:0/1309198091 conn(0x5619fbfde6c0 msgr2=0x5619fbfdeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 -- 172.21.15.131:0/1309198091 shutdown_connections 2024-04-26T20:26:24.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.533+0000 7fbbd0369740 1 -- 172.21.15.131:0/1309198091 wait complete. 2024-04-26T20:26:24.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.534+0000 7fbbbf400640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbac0a3b60 0x7fbbac0a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.534+0000 7fbbd0369740 1 -- start start 2024-04-26T20:26:24.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.534+0000 7fbbbf400640 1 -- 172.21.15.131:0/2276014450 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbbc00d0ab0 con 0x7fbbc8102860 2024-04-26T20:26:24.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.534+0000 7fbbc7400640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbac0a3b60 0x7fbbac0a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.534+0000 7fbbd0369740 1 Processor -- start 2024-04-26T20:26:24.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbfed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbd0369740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5619fbf45f80 con 0x5619fbfe3050 2024-04-26T20:26:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbd0369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5619fbef3260 con 0x5619fbfe8bb0 2024-04-26T20:26:24.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbd0369740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5619fbf4b2a0 con 0x5619fbfe3a40 2024-04-26T20:26:24.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbc7400640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbac0a3b60 0x7fbbac0a6020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8005940 tx=0x7fbbb802a340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbd0369740 10 client.0 _ll_register_callbacks cb 0x5619fbf3d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbc6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbfed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46868/0 (socket says 172.21.15.131:46868) 2024-04-26T20:26:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbc6a00640 1 -- 172.21.15.131:0/2218560204 learned_addr learned my addr 172.21.15.131:0/2218560204 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.535+0000 7fbbc7400640 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbc6000640 1 -- 172.21.15.131:0/2218560204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 msgr2=0x5619fbff44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbc6000640 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbc6000640 1 -- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 msgr2=0x5619fbfecf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbc6000640 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfecf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbc6000640 1 -- 172.21.15.131:0/2218560204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fbed51c0 con 0x5619fbfe3a40 2024-04-26T20:26:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbc6000640 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbfed490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00126d0 tx=0x7fbbb0012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27cf6e62a772fc1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbabe00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:26:25.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbabe00640 1 -- 172.21.15.131:0/2218560204 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000f8c0 con 0x5619fbfe3a40 2024-04-26T20:26:25.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbabe00640 1 -- 172.21.15.131:0/2218560204 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbbb000fa60 con 0x5619fbfe3a40 2024-04-26T20:26:25.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.536+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb8004030 con 0x5619fbfe3a40 2024-04-26T20:26:25.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.537+0000 7fbbabe00640 1 -- 172.21.15.131:0/2218560204 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbb000fc60 con 0x5619fbfe3a40 2024-04-26T20:26:25.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.537+0000 7fbbd0369740 10 client.26638 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.537+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5619fbc583f0 con 0x5619fbfe3a40 2024-04-26T20:26:25.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.537+0000 7fbbabe00640 1 -- 172.21.15.131:0/2218560204 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004030 con 0x5619fbfe3a40 2024-04-26T20:26:25.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.537+0000 7fbbabe00640 1 -- 172.21.15.131:0/2218560204 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbbb001c420 con 0x5619fbfe3a40 2024-04-26T20:26:25.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.538+0000 7fbbd0369740 10 client.26638 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:25.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.538+0000 7fbbd0369740 10 client.26638 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-26T20:26:25.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.538+0000 7fbbd0369740 20 client.26638 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:25.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.538+0000 7fbbd0369740 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fbff8a80 0x5619fbffae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.538+0000 7fbbd0369740 4 client.26638 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:25.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.538+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619fbf54880 con 0x5619fbff8a80 2024-04-26T20:26:25.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.538+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fbff8a80 0x5619fbffae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.539+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fbff8a80 0x5619fbffae90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4042ba1838855b73 server_cookie=326d64b00e25e1e6 in_seq=0 out_seq=0 2024-04-26T20:26:25.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.539+0000 7fbbabe00640 10 client.26638 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:26:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.539+0000 7fbbabe00640 1 -- 172.21.15.131:0/2218560204 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619fbf54880 con 0x5619fbff8a80 2024-04-26T20:26:25.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.539+0000 7fbbabe00640 10 client.26638 handle_command_reply: tid=0 2024-04-26T20:26:25.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.539+0000 7fbbd0369740 1 client.26638 shutdown 2024-04-26T20:26:25.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 msgr2=0x5619fbfed490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbfed490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00126d0 tx=0x7fbbb0012b90 comp rx=0 tx=0).stop 2024-04-26T20:26:25.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 shutdown_connections 2024-04-26T20:26:25.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fbff8a80 0x5619fbffae90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbfed490 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 --2- 172.21.15.131:0/2218560204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbfecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 >> 172.21.15.131:0/2218560204 conn(0x5619fbfde6c0 msgr2=0x5619fbfdd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbabe00640 0 client.26638 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:26:25.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 shutdown_connections 2024-04-26T20:26:25.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 1 -- 172.21.15.131:0/2218560204 wait complete. 2024-04-26T20:26:25.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.540+0000 7fbbd0369740 20 client.26638 trim_cache size 0 max 16384 2024-04-26T20:26:25.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 Processor -- start 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 -- start start 2024-04-26T20:26:25.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3050 0x5619fbfe1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbff4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe8bb0 0x5619fbfe1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81aff30 con 0x5619fbfe8bb0 2024-04-26T20:26:26.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ad800 con 0x5619fbfe3a40 2024-04-26T20:26:26.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.558+0000 7fbbd0369740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ab650 con 0x5619fbfe3050 2024-04-26T20:26:26.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3050 0x5619fbfe1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe8bb0 0x5619fbfe1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3050 0x5619fbfe1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33502/0 (socket says 172.21.15.131:33502) 2024-04-26T20:26:26.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 -- 172.21.15.131:0/994846259 learned_addr learned my addr 172.21.15.131:0/994846259 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:26.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6000640 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbff4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 -- 172.21.15.131:0/994846259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 msgr2=0x5619fbff4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbff4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 -- 172.21.15.131:0/994846259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe8bb0 msgr2=0x5619fbfe1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe8bb0 0x5619fbfe1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 -- 172.21.15.131:0/994846259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fbd227d0 con 0x5619fbfe3050 2024-04-26T20:26:26.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc7400640 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe8bb0 0x5619fbfe1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:26.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6000640 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbff4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:26.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.559+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3050 0x5619fbfe1ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0099120 tx=0x7fbbc0087040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b2c27887329b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbb6a00640 1 -- 172.21.15.131:0/994846259 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00a0b90 con 0x5619fbfe3050 2024-04-26T20:26:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbb6a00640 1 -- 172.21.15.131:0/994846259 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc00a5450 con 0x5619fbfe3050 2024-04-26T20:26:26.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbb6a00640 1 -- 172.21.15.131:0/994846259 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00a4fc0 con 0x5619fbfe3050 2024-04-26T20:26:26.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbd0369740 1 -- 172.21.15.131:0/994846259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3050 msgr2=0x5619fbfe1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbd0369740 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3050 0x5619fbfe1ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0099120 tx=0x7fbbc0087040 comp rx=0 tx=0).stop 2024-04-26T20:26:26.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbd0369740 1 -- 172.21.15.131:0/994846259 shutdown_connections 2024-04-26T20:26:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbd0369740 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe8bb0 0x5619fbfe1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.560+0000 7fbbd0369740 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe3a40 0x5619fbff4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.561+0000 7fbbd0369740 1 --2- 172.21.15.131:0/994846259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3050 0x5619fbfe1ab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.561+0000 7fbbd0369740 1 -- 172.21.15.131:0/994846259 >> 172.21.15.131:0/994846259 conn(0x5619fbfde6c0 msgr2=0x5619fbfe13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.561+0000 7fbbd0369740 1 -- 172.21.15.131:0/994846259 shutdown_connections 2024-04-26T20:26:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.561+0000 7fbbd0369740 1 -- 172.21.15.131:0/994846259 wait complete. 2024-04-26T20:26:26.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.561+0000 7fbbd0369740 1 -- start start 2024-04-26T20:26:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.561+0000 7fbbd0369740 1 Processor -- start 2024-04-26T20:26:26.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbff2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbff2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbd0369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbd0369740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81abcc0 con 0x5619fbfe3050 2024-04-26T20:26:26.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbd0369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ad130 con 0x5619fbfe8bb0 2024-04-26T20:26:26.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbd0369740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ac390 con 0x5619fbfe3a40 2024-04-26T20:26:26.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbd0369740 10 client.0 _ll_register_callbacks cb 0x7fbbb0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:26.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbff2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbc6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbff2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbc6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbff2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46882/0 (socket says 172.21.15.131:46882) 2024-04-26T20:26:26.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbc6a00640 1 -- 172.21.15.131:0/3339353889 learned_addr learned my addr 172.21.15.131:0/3339353889 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:26.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbc7400640 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbc6000640 1 -- 172.21.15.131:0/3339353889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 msgr2=0x5619fbff3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.562+0000 7fbbc6000640 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbc6000640 1 -- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 msgr2=0x5619fbff2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbc6000640 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbff2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbc6000640 1 -- 172.21.15.131:0/3339353889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fbeed2a0 con 0x5619fbfe3a40 2024-04-26T20:26:26.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbff2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:26.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbc6000640 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbff2570 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbb001c890 tx=0x7fbbb0009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64060e6f8324e993 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbabe00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:26:26.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbabe00640 1 -- 172.21.15.131:0/3339353889 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb002f060 con 0x5619fbfe3a40 2024-04-26T20:26:26.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc00a5450 con 0x5619fbfe3a40 2024-04-26T20:26:26.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbabe00640 1 -- 172.21.15.131:0/3339353889 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbbb000f790 con 0x5619fbfe3a40 2024-04-26T20:26:26.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbabe00640 1 -- 172.21.15.131:0/3339353889 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbb0036050 con 0x5619fbfe3a40 2024-04-26T20:26:26.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbd0369740 10 client.26644 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:26.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.563+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5619fc05f290 con 0x5619fbfe3a40 2024-04-26T20:26:26.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbabe00640 1 -- 172.21.15.131:0/3339353889 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc00a5450 con 0x5619fbfe3a40 2024-04-26T20:26:26.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbabe00640 1 -- 172.21.15.131:0/3339353889 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbbb0003680 con 0x5619fbfe3a40 2024-04-26T20:26:27.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbd0369740 10 client.26644 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbd0369740 10 client.26644 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-26T20:26:27.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbd0369740 20 client.26644 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:27.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbd0369740 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fc05fcc0 0x5619fc0620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbd0369740 4 client.26644 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:26:27.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619fbf4e6f0 con 0x5619fc05fcc0 2024-04-26T20:26:27.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.564+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fc05fcc0 0x5619fc0620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbc6a00640 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fc05fcc0 0x5619fc0620e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abae14fb35113ed8 server_cookie=3f02ae0a95d6b41d in_seq=0 out_seq=0 2024-04-26T20:26:27.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbabe00640 10 client.26644 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:26:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbabe00640 1 -- 172.21.15.131:0/3339353889 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619fbf4e6f0 con 0x5619fc05fcc0 2024-04-26T20:26:27.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbabe00640 10 client.26644 handle_command_reply: tid=0 2024-04-26T20:26:27.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 client.26644 shutdown 2024-04-26T20:26:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 msgr2=0x5619fbff2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbff2570 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbb001c890 tx=0x7fbbb0009e10 comp rx=0 tx=0).stop 2024-04-26T20:26:27.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 shutdown_connections 2024-04-26T20:26:27.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5619fc05fcc0 0x5619fc0620e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619fbfe8bb0 0x5619fbff3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5619fbfe3a40 0x5619fbff2570 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 --2- 172.21.15.131:0/3339353889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619fbfe3050 0x5619fbff2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbabe00640 0 client.26644 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:26:27.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.565+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 >> 172.21.15.131:0/3339353889 conn(0x5619fbfde6c0 msgr2=0x5619fbfe1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 shutdown_connections 2024-04-26T20:26:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbd0369740 1 -- 172.21.15.131:0/3339353889 wait complete. 2024-04-26T20:26:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbd0369740 20 client.26644 trim_cache size 0 max 16384 2024-04-26T20:26:27.368 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbac0a3b60 msgr2=0x7fbbac0a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbac0a3b60 0x7fbbac0a6020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8005940 tx=0x7fbbb802a340 comp rx=0 tx=0).stop 2024-04-26T20:26:27.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 msgr2=0x7fbbc819ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 0x7fbbc819ac20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c3a0 tx=0x7fbbc000c860 comp rx=0 tx=0).stop 2024-04-26T20:26:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.566+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 shutdown_connections 2024-04-26T20:26:27.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.567+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbbac0a3b60 0x7fbbac0a6020 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.567+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbc8104410 0x7fbbc81a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.567+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc8103a50 0x7fbbc819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.567+0000 7fbbc7e00640 1 --2- 172.21.15.131:0/2276014450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc8102860 0x7fbbc819ac20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.567+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 >> 172.21.15.131:0/2276014450 conn(0x7fbbc80fddd0 msgr2=0x7fbbc8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.567+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 shutdown_connections 2024-04-26T20:26:27.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:23.567+0000 7fbbc7e00640 1 -- 172.21.15.131:0/2276014450 wait complete. 2024-04-26T20:26:27.709 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T20:26:27.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:23 smithi131 ceph-mon[27919]: pgmap v901: 129 pgs: 129 active+clean; 895 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-04-26T20:26:27.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:23 smithi157 ceph-mon[32703]: pgmap v901: 129 pgs: 129 active+clean; 895 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-04-26T20:26:27.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:25 smithi157 ceph-mon[32703]: pgmap v902: 129 pgs: 129 active+clean; 824 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-04-26T20:26:27.746 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:23 smithi148 ceph-mon[32281]: pgmap v901: 129 pgs: 129 active+clean; 895 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-04-26T20:26:27.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:25 smithi148 ceph-mon[32281]: pgmap v902: 129 pgs: 129 active+clean; 824 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-04-26T20:26:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.104+0000 7f241a800640 1 -- 172.21.15.131:0/1012085727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141028a0 msgr2=0x7f2414102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.104+0000 7f241a800640 1 --2- 172.21.15.131:0/1012085727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141028a0 0x7f2414102d20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2400009900 tx=0x7f240002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.105+0000 7f241a800640 1 -- 172.21.15.131:0/1012085727 shutdown_connections 2024-04-26T20:26:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.105+0000 7f241a800640 1 --2- 172.21.15.131:0/1012085727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2414103260 0x7f2414109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.105+0000 7f241a800640 1 --2- 172.21.15.131:0/1012085727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141028a0 0x7f2414102d20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.105+0000 7f241a800640 1 --2- 172.21.15.131:0/1012085727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24141016b0 0x7f2414101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.105+0000 7f241a800640 1 -- 172.21.15.131:0/1012085727 >> 172.21.15.131:0/1012085727 conn(0x7f24140fcc20 msgr2=0x7f24140ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.105+0000 7f241a800640 1 -- 172.21.15.131:0/1012085727 shutdown_connections 2024-04-26T20:26:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.105+0000 7f241a800640 1 -- 172.21.15.131:0/1012085727 wait complete. 2024-04-26T20:26:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 Processor -- start 2024-04-26T20:26:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 -- start start 2024-04-26T20:26:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141016b0 0x7f2414105640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24141028a0 0x7f2414103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2414103260 0x7f24141041c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f241410d230 con 0x7f24141028a0 2024-04-26T20:26:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f241410d0b0 con 0x7f24141016b0 2024-04-26T20:26:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f241a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f241410d3b0 con 0x7f2414103260 2024-04-26T20:26:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f2418a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24141028a0 0x7f2414103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f2418a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24141028a0 0x7f2414103c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40232/0 (socket says 172.21.15.131:40232) 2024-04-26T20:26:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f2418a00640 1 -- 172.21.15.131:0/4020310152 learned_addr learned my addr 172.21.15.131:0/4020310152 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.106+0000 7f2419400640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141016b0 0x7f2414105640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2419e00640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2414103260 0x7f24141041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2418a00640 1 -- 172.21.15.131:0/4020310152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2414103260 msgr2=0x7f24141041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2418a00640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2414103260 0x7f24141041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2418a00640 1 -- 172.21.15.131:0/4020310152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141016b0 msgr2=0x7f2414105640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2418a00640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141016b0 0x7f2414105640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2418a00640 1 -- 172.21.15.131:0/4020310152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2414104a50 con 0x7f24141028a0 2024-04-26T20:26:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2419400640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141016b0 0x7f2414105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.107+0000 7f2418a00640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24141028a0 0x7f2414103c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2400005ab0 tx=0x7f2400003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.108+0000 7f2412000640 1 -- 172.21.15.131:0/4020310152 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240003f070 con 0x7f24141028a0 2024-04-26T20:26:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.108+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24141a2b40 con 0x7f24141028a0 2024-04-26T20:26:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.108+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24141a30e0 con 0x7f24141028a0 2024-04-26T20:26:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.108+0000 7f2412000640 1 -- 172.21.15.131:0/4020310152 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2400031c50 con 0x7f24141028a0 2024-04-26T20:26:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.108+0000 7f2412000640 1 -- 172.21.15.131:0/4020310152 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400049060 con 0x7f24141028a0 2024-04-26T20:26:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 Processor -- start 2024-04-26T20:26:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f2412000640 1 -- 172.21.15.131:0/4020310152 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2400042070 con 0x7f24141028a0 2024-04-26T20:26:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 -- start start 2024-04-26T20:26:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf053a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053a7d0 0x560bf053f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053f940 0x560bf0541dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560bf049e380 con 0x560bf0539de0 2024-04-26T20:26:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bf043ac30 con 0x560bf053a7d0 2024-04-26T20:26:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f241ec1f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560bf049a340 con 0x560bf053f940 2024-04-26T20:26:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.111+0000 7f2419e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf053a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf053a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40248/0 (socket says 172.21.15.131:40248) 2024-04-26T20:26:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 -- 172.21.15.131:0/3224021815 learned_addr learned my addr 172.21.15.131:0/3224021815 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419400640 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053a7d0 0x560bf053f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2418a00640 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053f940 0x560bf0541dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 -- 172.21.15.131:0/3224021815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053f940 msgr2=0x560bf0541dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053f940 0x560bf0541dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 -- 172.21.15.131:0/3224021815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053a7d0 msgr2=0x560bf053f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053a7d0 0x560bf053f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 -- 172.21.15.131:0/3224021815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf045c940 con 0x560bf0539de0 2024-04-26T20:26:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419400640 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053a7d0 0x560bf053f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2412000640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f23fc0a3cb0 0x7f23fc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.112+0000 7f2419e00640 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf053a200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f240c011440 tx=0x7f240c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2c30bad5b627a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f2412000640 1 -- 172.21.15.131:0/4020310152 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f24000f0c50 con 0x7f24141028a0 2024-04-26T20:26:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f2406600640 1 -- 172.21.15.131:0/3224021815 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c003c30 con 0x560bf0539de0 2024-04-26T20:26:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f2406600640 1 -- 172.21.15.131:0/3224021815 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c003dd0 con 0x560bf0539de0 2024-04-26T20:26:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f2406600640 1 -- 172.21.15.131:0/3224021815 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c00fae0 con 0x560bf0539de0 2024-04-26T20:26:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f2419400640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f23fc0a3cb0 0x7f23fc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f241ec1f740 1 -- 172.21.15.131:0/3224021815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 msgr2=0x560bf053a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f241ec1f740 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf053a200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f240c011440 tx=0x7f240c011900 comp rx=0 tx=0).stop 2024-04-26T20:26:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.113+0000 7f2419400640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f23fc0a3cb0 0x7f23fc0a6170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 -- 172.21.15.131:0/3224021815 shutdown_connections 2024-04-26T20:26:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053f940 0x560bf0541dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053a7d0 0x560bf053f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 --2- 172.21.15.131:0/3224021815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf053a200 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 -- 172.21.15.131:0/3224021815 >> 172.21.15.131:0/3224021815 conn(0x560bf0535450 msgr2=0x560bf0535850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 -- 172.21.15.131:0/3224021815 shutdown_connections 2024-04-26T20:26:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 -- 172.21.15.131:0/3224021815 wait complete. 2024-04-26T20:26:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 -- start start 2024-04-26T20:26:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.114+0000 7f241ec1f740 1 Processor -- start 2024-04-26T20:26:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0544020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf0544560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf054b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f241ec1f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560bf05428f0 con 0x560bf053f940 2024-04-26T20:26:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f241ec1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bf0425c60 con 0x560bf0539de0 2024-04-26T20:26:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f241ec1f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560bf0542a70 con 0x560bf053a7d0 2024-04-26T20:26:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f241ec1f740 10 client.0 _ll_register_callbacks cb 0x560bf0451e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf054b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2418a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf0544560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf054b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40252/0 (socket says 172.21.15.131:40252) 2024-04-26T20:26:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2418a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf0544560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53330/0 (socket says 172.21.15.131:53330) 2024-04-26T20:26:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 -- 172.21.15.131:0/2142444695 learned_addr learned my addr 172.21.15.131:0/2142444695 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419400640 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0544020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 -- 172.21.15.131:0/2142444695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 msgr2=0x560bf0544560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf0544560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 -- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 msgr2=0x560bf0544020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0544020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2419e00640 1 -- 172.21.15.131:0/2142444695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf0434510 con 0x560bf053f940 2024-04-26T20:26:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.115+0000 7f2418a00640 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf0544560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f2419e00640 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf054b5a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f240c006010 tx=0x7f240c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10605c4a0b3eaa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f23faa00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:26:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f23faa00640 1 -- 172.21.15.131:0/2142444695 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c028070 con 0x560bf053f940 2024-04-26T20:26:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c003dd0 con 0x560bf053f940 2024-04-26T20:26:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f23faa00640 1 -- 172.21.15.131:0/2142444695 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f240c00fa30 con 0x560bf053f940 2024-04-26T20:26:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f23faa00640 1 -- 172.21.15.131:0/2142444695 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f240c034050 con 0x560bf053f940 2024-04-26T20:26:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f241ec1f740 10 client.16566 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.116+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bf01b0210 con 0x560bf053f940 2024-04-26T20:26:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f23faa00640 1 -- 172.21.15.131:0/2142444695 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c003dd0 con 0x560bf053f940 2024-04-26T20:26:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f23faa00640 1 -- 172.21.15.131:0/2142444695 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f240c005040 con 0x560bf053f940 2024-04-26T20:26:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f241ec1f740 10 client.16566 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f241ec1f740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f241ec1f740 20 client.16566 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f241ec1f740 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf054f900 0x560bf0551d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f241ec1f740 4 client.16566 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bf04abf90 con 0x560bf054f900 2024-04-26T20:26:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.117+0000 7f2419400640 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf054f900 0x560bf0551d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.118+0000 7f23faa00640 10 client.16566 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.118+0000 7f2419400640 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf054f900 0x560bf0551d20 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6cdd6ccaca3e2d server_cookie=e826b0616774a0f3 in_seq=0 out_seq=0 2024-04-26T20:26:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f23faa00640 1 -- 172.21.15.131:0/2142444695 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bf04abf90 con 0x560bf054f900 2024-04-26T20:26:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f23faa00640 10 client.16566 handle_command_reply: tid=0 2024-04-26T20:26:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 client.16566 shutdown 2024-04-26T20:26:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 msgr2=0x560bf054b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf054b5a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f240c006010 tx=0x7f240c009890 comp rx=0 tx=0).stop 2024-04-26T20:26:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 shutdown_connections 2024-04-26T20:26:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf054f900 0x560bf0551d20 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf054b5a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf0544560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 --2- 172.21.15.131:0/2142444695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0544020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 >> 172.21.15.131:0/2142444695 conn(0x560bf0535450 msgr2=0x560bf0534650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f23faa00640 0 client.16566 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 shutdown_connections 2024-04-26T20:26:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 1 -- 172.21.15.131:0/2142444695 wait complete. 2024-04-26T20:26:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.119+0000 7f241ec1f740 20 client.16566 trim_cache size 0 max 16384 2024-04-26T20:26:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 Processor -- start 2024-04-26T20:26:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 -- start start 2024-04-26T20:26:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0538c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf054b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf05435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4012610 con 0x560bf053f940 2024-04-26T20:26:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23f400fee0 con 0x560bf0539de0 2024-04-26T20:26:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.135+0000 7f241ec1f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f23f400dd30 con 0x560bf053a7d0 2024-04-26T20:26:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf05435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf05435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40258/0 (socket says 172.21.15.131:40258) 2024-04-26T20:26:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 -- 172.21.15.131:0/1682980201 learned_addr learned my addr 172.21.15.131:0/1682980201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419400640 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0538c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2418a00640 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf054b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 -- 172.21.15.131:0/1682980201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 msgr2=0x560bf054b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf054b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 -- 172.21.15.131:0/1682980201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 msgr2=0x560bf0538c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0538c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 -- 172.21.15.131:0/1682980201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf027c620 con 0x560bf053f940 2024-04-26T20:26:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419400640 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0538c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.136+0000 7f2419e00640 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf05435b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f240c005d70 tx=0x7f240c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219632691136f31a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f2405c00640 1 -- 172.21.15.131:0/1682980201 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c028070 con 0x560bf053f940 2024-04-26T20:26:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f2405c00640 1 -- 172.21.15.131:0/1682980201 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c005760 con 0x560bf053f940 2024-04-26T20:26:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f2405c00640 1 -- 172.21.15.131:0/1682980201 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c004000 con 0x560bf053f940 2024-04-26T20:26:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 -- 172.21.15.131:0/1682980201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 msgr2=0x560bf05435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf05435b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f240c005d70 tx=0x7f240c009500 comp rx=0 tx=0).stop 2024-04-26T20:26:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 -- 172.21.15.131:0/1682980201 shutdown_connections 2024-04-26T20:26:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf053f940 0x560bf05435b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf054b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1682980201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf0539de0 0x560bf0538c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 -- 172.21.15.131:0/1682980201 >> 172.21.15.131:0/1682980201 conn(0x560bf0535450 msgr2=0x560bf0542ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 -- 172.21.15.131:0/1682980201 shutdown_connections 2024-04-26T20:26:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.137+0000 7f241ec1f740 1 -- 172.21.15.131:0/1682980201 wait complete. 2024-04-26T20:26:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 -- start start 2024-04-26T20:26:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 Processor -- start 2024-04-26T20:26:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf0548b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf05490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053f940 0x560bf054a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23f400e3a0 con 0x560bf0539de0 2024-04-26T20:26:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23f400f810 con 0x560bf053f940 2024-04-26T20:26:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.138+0000 7f241ec1f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f23f400ea70 con 0x560bf053a7d0 2024-04-26T20:26:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f241ec1f740 10 client.0 _ll_register_callbacks cb 0x7f2408009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf0548b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf0548b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40274/0 (socket says 172.21.15.131:40274) 2024-04-26T20:26:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 -- 172.21.15.131:0/1102209171 learned_addr learned my addr 172.21.15.131:0/1102209171 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:26:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2418a00640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf05490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419e00640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053f940 0x560bf054a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 -- 172.21.15.131:0/1102209171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 msgr2=0x560bf05490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf05490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 -- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053f940 msgr2=0x560bf054a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053f940 0x560bf054a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 -- 172.21.15.131:0/1102209171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf0458cb0 con 0x560bf0539de0 2024-04-26T20:26:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419e00640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053f940 0x560bf054a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f2419400640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf0548b60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x560bf053b240 tx=0x7f24080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9a49832e39518b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.139+0000 7f23faa00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.140+0000 7f23faa00640 1 -- 172.21.15.131:0/1102209171 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408004480 con 0x560bf0539de0 2024-04-26T20:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.140+0000 7f23faa00640 1 -- 172.21.15.131:0/1102209171 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f240800f990 con 0x560bf0539de0 2024-04-26T20:26:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.140+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c005760 con 0x560bf0539de0 2024-04-26T20:26:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.140+0000 7f23faa00640 1 -- 172.21.15.131:0/1102209171 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f240800fb90 con 0x560bf0539de0 2024-04-26T20:26:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.140+0000 7f241ec1f740 10 client.16572 fetch_fsmap learned FSMap version 27 2024-04-26T20:26:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.140+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bf054b8b0 con 0x560bf0539de0 2024-04-26T20:26:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.140+0000 7f23faa00640 1 -- 172.21.15.131:0/1102209171 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c005760 con 0x560bf0539de0 2024-04-26T20:26:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f23faa00640 1 -- 172.21.15.131:0/1102209171 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f24080038f0 con 0x560bf0539de0 2024-04-26T20:26:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f241ec1f740 10 client.16572 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:26:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f241ec1f740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:26:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f241ec1f740 20 client.16572 populate_metadata read hostname 'smithi131' 2024-04-26T20:26:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf05b6730 0x560bf05b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f241ec1f740 4 client.16572 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:26:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bf04a5e00 con 0x560bf05b6730 2024-04-26T20:26:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.141+0000 7f2418a00640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf05b6730 0x560bf05b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.142+0000 7f23faa00640 10 client.16572 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.142+0000 7f2418a00640 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf05b6730 0x560bf05b8b50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a448ef044f66d8dc server_cookie=705be067fe65d18f in_seq=0 out_seq=0 2024-04-26T20:26:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f23faa00640 1 -- 172.21.15.131:0/1102209171 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bf04a5e00 con 0x560bf05b6730 2024-04-26T20:26:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f23faa00640 10 client.16572 handle_command_reply: tid=0 2024-04-26T20:26:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 client.16572 shutdown 2024-04-26T20:26:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 msgr2=0x560bf0548b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf0548b60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x560bf053b240 tx=0x7f24080046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 shutdown_connections 2024-04-26T20:26:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560bf05b6730 0x560bf05b8b50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf053f940 0x560bf054a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560bf053a7d0 0x560bf05490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f23faa00640 0 client.16572 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:26:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 --2- 172.21.15.131:0/1102209171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560bf0539de0 0x560bf0548b60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.156+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 >> 172.21.15.131:0/1102209171 conn(0x560bf0535450 msgr2=0x560bf0534650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.157+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 shutdown_connections 2024-04-26T20:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.157+0000 7f241ec1f740 1 -- 172.21.15.131:0/1102209171 wait complete. 2024-04-26T20:26:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.157+0000 7f241ec1f740 20 client.16572 trim_cache size 0 max 16384 2024-04-26T20:26:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:26:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:26:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:26:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:26:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.157+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f23fc0a3cb0 msgr2=0x7f23fc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.157+0000 7f241a800640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f23fc0a3cb0 0x7f23fc0a6170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408005f90 comp rx=0 tx=0).stop 2024-04-26T20:26:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.157+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24141028a0 msgr2=0x7f2414103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.157+0000 7f241a800640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24141028a0 0x7f2414103c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2400005ab0 tx=0x7f2400003b90 comp rx=0 tx=0).stop 2024-04-26T20:26:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 shutdown_connections 2024-04-26T20:26:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f23fc0a3cb0 0x7f23fc0a6170 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2414103260 0x7f24141041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24141028a0 0x7f2414103c80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 --2- 172.21.15.131:0/4020310152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24141016b0 0x7f2414105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 >> 172.21.15.131:0/4020310152 conn(0x7f24140fcc20 msgr2=0x7f2414069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 shutdown_connections 2024-04-26T20:26:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:25.158+0000 7f241a800640 1 -- 172.21.15.131:0/4020310152 wait complete. 2024-04-26T20:26:32.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:25 smithi131 ceph-mon[27919]: pgmap v902: 129 pgs: 129 active+clean; 824 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-04-26T20:26:32.690 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:27 smithi131 ceph-mon[27919]: pgmap v903: 129 pgs: 129 active+clean; 824 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-04-26T20:26:32.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:27 smithi157 ceph-mon[32703]: pgmap v903: 129 pgs: 129 active+clean; 824 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-04-26T20:26:32.712 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:29 smithi157 ceph-mon[32703]: pgmap v904: 129 pgs: 129 active+clean; 866 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-04-26T20:26:32.757 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:31 smithi157 ceph-mon[32703]: pgmap v905: 129 pgs: 129 active+clean; 866 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-04-26T20:26:32.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:27 smithi148 ceph-mon[32281]: pgmap v903: 129 pgs: 129 active+clean; 824 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-04-26T20:26:32.775 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:29 smithi148 ceph-mon[32281]: pgmap v904: 129 pgs: 129 active+clean; 866 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-04-26T20:26:32.784 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:31 smithi148 ceph-mon[32281]: pgmap v905: 129 pgs: 129 active+clean; 866 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-04-26T20:26:32.804 INFO:tasks.workunit.client.0.smithi131.stdout:writing pattern 2024-04-26T20:26:32.813 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-04-26T20:26:32.871 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-26T20:26:32.948 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-04-26T20:26:32.957 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-26T20:26:32.969 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-04-26T20:26:32.979 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-26T20:26:33.044 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-04-26T20:26:33.052 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-26T20:26:33.060 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-04-26T20:26:33.066 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-26T20:26:33.073 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-04-26T20:26:33.081 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-26T20:26:33.162 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-04-26T20:26:33.169 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-26T20:26:33.177 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-04-26T20:26:33.184 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-26T20:26:33.190 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-04-26T20:26:33.199 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-26T20:26:33.256 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-04-26T20:26:33.338 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-26T20:26:33.345 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-04-26T20:26:33.353 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-26T20:26:33.361 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-04-26T20:26:33.417 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-26T20:26:33.425 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-04-26T20:26:33.432 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-26T20:26:33.438 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-04-26T20:26:33.445 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-26T20:26:33.452 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-04-26T20:26:33.503 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-26T20:26:33.511 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-04-26T20:26:33.519 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-26T20:26:33.525 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-04-26T20:26:33.534 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-26T20:26:33.541 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-04-26T20:26:33.579 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-26T20:26:33.633 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-04-26T20:26:33.639 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-26T20:26:33.646 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-04-26T20:26:33.653 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-26T20:26:33.707 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-04-26T20:26:33.716 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-26T20:26:33.727 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-04-26T20:26:33.740 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-26T20:26:33.750 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-04-26T20:26:33.757 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-26T20:26:33.809 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-04-26T20:26:33.817 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-26T20:26:33.827 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-04-26T20:26:33.834 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-26T20:26:33.844 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-04-26T20:26:33.855 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-26T20:26:33.900 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-04-26T20:26:33.958 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-26T20:26:33.969 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-04-26T20:26:33.977 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-26T20:26:33.986 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-04-26T20:26:34.034 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-26T20:26:34.043 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-04-26T20:26:34.053 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-26T20:26:34.060 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-04-26T20:26:34.069 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-26T20:26:34.077 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-04-26T20:26:34.135 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-26T20:26:34.143 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-04-26T20:26:34.149 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-26T20:26:34.158 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-04-26T20:26:34.168 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-26T20:26:34.177 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-04-26T20:26:34.216 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-26T20:26:34.280 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-04-26T20:26:34.286 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-26T20:26:34.294 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-04-26T20:26:34.301 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-26T20:26:34.414 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-04-26T20:26:34.421 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-26T20:26:34.429 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-04-26T20:26:34.436 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-26T20:26:34.443 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-04-26T20:26:34.450 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-26T20:26:34.548 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-04-26T20:26:34.555 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-26T20:26:34.562 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-04-26T20:26:34.572 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-26T20:26:34.578 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-04-26T20:26:34.585 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-26T20:26:34.613 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-04-26T20:26:34.668 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-26T20:26:34.675 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-04-26T20:26:34.682 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-26T20:26:34.690 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-04-26T20:26:34.734 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-26T20:26:34.740 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-04-26T20:26:34.748 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-26T20:26:34.754 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-04-26T20:26:34.760 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-26T20:26:34.767 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-04-26T20:26:34.806 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-26T20:26:34.813 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-04-26T20:26:34.820 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-26T20:26:34.827 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-04-26T20:26:34.836 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-26T20:26:34.846 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-04-26T20:26:34.871 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-26T20:26:34.925 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-04-26T20:26:34.937 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-26T20:26:34.945 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-04-26T20:26:34.958 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-26T20:26:35.008 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-04-26T20:26:35.015 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-26T20:26:35.025 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-04-26T20:26:35.032 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-26T20:26:35.043 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-04-26T20:26:35.052 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-26T20:26:35.109 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-04-26T20:26:35.119 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-26T20:26:35.141 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-04-26T20:26:35.151 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-26T20:26:35.160 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-04-26T20:26:35.168 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-26T20:26:35.201 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-04-26T20:26:35.266 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-26T20:26:35.276 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-04-26T20:26:35.282 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-26T20:26:35.289 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-04-26T20:26:35.352 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-26T20:26:35.359 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-04-26T20:26:35.366 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-26T20:26:35.374 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-04-26T20:26:35.385 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-26T20:26:35.394 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-04-26T20:26:35.445 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-26T20:26:35.455 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-04-26T20:26:35.467 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-26T20:26:35.492 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-04-26T20:26:35.507 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-26T20:26:35.519 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-04-26T20:26:35.573 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-26T20:26:35.651 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-04-26T20:26:35.659 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-26T20:26:35.666 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-04-26T20:26:35.674 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-26T20:26:35.704 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-04-26T20:26:35.711 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-26T20:26:35.718 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-04-26T20:26:35.725 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-26T20:26:35.733 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-04-26T20:26:35.741 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-26T20:26:35.800 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-04-26T20:26:35.808 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-26T20:26:35.815 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-04-26T20:26:35.821 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-26T20:26:35.828 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-04-26T20:26:35.835 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-26T20:26:35.888 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-04-26T20:26:35.956 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-26T20:26:35.964 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-04-26T20:26:35.970 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-26T20:26:35.978 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-04-26T20:26:36.021 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-26T20:26:36.029 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-04-26T20:26:36.037 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-26T20:26:36.045 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-04-26T20:26:36.054 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-26T20:26:36.063 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-04-26T20:26:36.125 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-26T20:26:36.134 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-04-26T20:26:36.151 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-26T20:26:36.161 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-04-26T20:26:36.170 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-26T20:26:36.179 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-04-26T20:26:36.236 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-26T20:26:36.285 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-04-26T20:26:36.295 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-26T20:26:36.304 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-04-26T20:26:36.314 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-26T20:26:36.362 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-04-26T20:26:36.369 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-26T20:26:36.377 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-04-26T20:26:36.385 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-26T20:26:36.394 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-04-26T20:26:36.403 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-26T20:26:36.450 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-04-26T20:26:36.459 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-26T20:26:36.468 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-04-26T20:26:36.476 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-26T20:26:36.485 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-04-26T20:26:36.495 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-26T20:26:36.530 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-04-26T20:26:36.576 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-26T20:26:36.583 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-04-26T20:26:36.591 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-26T20:26:36.600 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-04-26T20:26:36.659 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-26T20:26:36.668 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-04-26T20:26:36.677 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-26T20:26:36.686 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-04-26T20:26:36.694 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-26T20:26:36.703 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-04-26T20:26:36.759 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-26T20:26:36.766 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-04-26T20:26:36.773 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-26T20:26:36.781 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-04-26T20:26:36.788 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-26T20:26:36.795 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-04-26T20:26:36.830 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-26T20:26:36.838 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-04-26T20:26:36.846 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-26T20:26:36.891 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-04-26T20:26:36.910 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-26T20:26:36.918 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-04-26T20:26:36.924 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-26T20:26:36.931 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-04-26T20:26:36.966 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-26T20:26:36.973 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-04-26T20:26:36.979 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-26T20:26:37.035 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-04-26T20:26:37.043 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-26T20:26:37.050 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-04-26T20:26:37.057 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-26T20:26:37.065 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-04-26T20:26:37.072 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-26T20:26:37.138 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-04-26T20:26:37.185 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-26T20:26:37.192 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-04-26T20:26:37.200 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-26T20:26:37.207 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-04-26T20:26:37.259 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-26T20:26:37.268 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-04-26T20:26:37.278 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-26T20:26:37.287 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-04-26T20:26:37.293 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-26T20:26:37.300 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-04-26T20:26:37.347 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-26T20:26:37.354 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-04-26T20:26:37.360 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-26T20:26:37.367 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-04-26T20:26:37.377 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-26T20:26:37.384 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-04-26T20:26:37.390 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-26T20:26:37.434 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-04-26T20:26:37.442 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-26T20:26:37.450 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-04-26T20:26:37.456 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-26T20:26:37.530 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-04-26T20:26:37.539 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-26T20:26:37.548 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-04-26T20:26:37.556 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-26T20:26:37.565 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-04-26T20:26:37.574 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-26T20:26:37.614 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-04-26T20:26:37.635 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-26T20:26:37.644 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-04-26T20:26:37.652 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-26T20:26:37.660 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-04-26T20:26:37.671 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-26T20:26:37.713 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-04-26T20:26:37.759 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-26T20:26:37.769 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-04-26T20:26:37.776 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-26T20:26:37.785 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-04-26T20:26:37.848 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-26T20:26:37.857 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-04-26T20:26:37.865 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-26T20:26:37.875 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-04-26T20:26:37.885 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-26T20:26:37.893 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-04-26T20:26:37.939 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-26T20:26:37.948 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-04-26T20:26:37.956 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-26T20:26:37.965 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-04-26T20:26:37.973 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-26T20:26:37.981 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-04-26T20:26:38.013 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-26T20:26:38.057 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-04-26T20:26:38.093 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-26T20:26:38.102 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-04-26T20:26:38.109 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-26T20:26:38.191 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-04-26T20:26:38.198 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-26T20:26:38.206 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-04-26T20:26:38.214 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-26T20:26:38.223 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-04-26T20:26:38.232 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-26T20:26:38.286 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-04-26T20:26:38.294 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-26T20:26:38.302 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-04-26T20:26:38.310 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-26T20:26:38.317 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-04-26T20:26:38.325 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-26T20:26:38.397 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-04-26T20:26:38.441 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-26T20:26:38.452 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-04-26T20:26:38.459 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-26T20:26:38.467 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-04-26T20:26:38.521 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-26T20:26:38.529 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-04-26T20:26:38.536 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-26T20:26:38.544 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-04-26T20:26:38.551 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-26T20:26:38.558 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-04-26T20:26:38.607 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-26T20:26:38.615 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-04-26T20:26:38.622 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-26T20:26:38.630 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-04-26T20:26:38.637 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-26T20:26:38.644 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-04-26T20:26:38.688 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-26T20:26:38.734 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-04-26T20:26:38.741 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-26T20:26:38.750 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-04-26T20:26:38.757 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-26T20:26:38.790 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-04-26T20:26:38.797 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-26T20:26:38.804 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-04-26T20:26:38.812 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-26T20:26:38.819 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-04-26T20:26:38.826 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-26T20:26:38.872 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-04-26T20:26:38.879 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-26T20:26:38.885 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-04-26T20:26:38.891 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-26T20:26:38.897 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-04-26T20:26:38.904 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-26T20:26:38.946 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-04-26T20:26:39.009 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-26T20:26:39.018 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-04-26T20:26:39.027 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-26T20:26:39.036 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-04-26T20:26:39.082 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-26T20:26:39.089 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-04-26T20:26:39.097 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-26T20:26:39.105 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-04-26T20:26:39.114 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-26T20:26:39.123 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-04-26T20:26:39.179 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-26T20:26:39.188 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-04-26T20:26:39.197 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-26T20:26:39.206 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-04-26T20:26:39.214 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-26T20:26:39.223 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-04-26T20:26:39.271 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-26T20:26:39.334 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-04-26T20:26:39.340 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-26T20:26:39.349 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-04-26T20:26:39.359 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-26T20:26:39.407 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-04-26T20:26:39.413 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-26T20:26:39.421 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-04-26T20:26:39.430 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-26T20:26:39.439 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-04-26T20:26:39.447 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-26T20:26:39.503 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-04-26T20:26:39.512 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-26T20:26:39.520 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-04-26T20:26:39.530 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-26T20:26:39.538 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-04-26T20:26:39.547 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-26T20:26:39.555 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-04-26T20:26:39.600 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-26T20:26:39.630 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-04-26T20:26:39.637 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-26T20:26:39.646 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-04-26T20:26:39.703 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-26T20:26:39.713 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-04-26T20:26:39.721 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-26T20:26:39.728 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-04-26T20:26:39.735 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-26T20:26:39.742 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-04-26T20:26:39.770 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-26T20:26:39.797 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-04-26T20:26:39.803 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-26T20:26:39.809 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-04-26T20:26:39.816 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-26T20:26:39.824 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-04-26T20:26:39.862 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-26T20:26:39.920 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-04-26T20:26:39.928 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-26T20:26:39.936 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-04-26T20:26:39.943 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-26T20:26:40.031 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-04-26T20:26:40.038 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-26T20:26:40.046 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-04-26T20:26:40.053 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-26T20:26:40.060 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-04-26T20:26:40.067 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-26T20:26:40.126 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-04-26T20:26:40.132 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-26T20:26:40.137 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-04-26T20:26:40.144 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-26T20:26:40.153 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-04-26T20:26:40.162 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-26T20:26:40.204 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-04-26T20:26:40.298 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-26T20:26:40.306 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-04-26T20:26:40.315 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-26T20:26:40.324 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-04-26T20:26:40.379 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-26T20:26:40.391 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-04-26T20:26:40.400 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-26T20:26:40.409 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-04-26T20:26:40.418 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-26T20:26:40.429 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-04-26T20:26:40.476 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-26T20:26:40.485 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-04-26T20:26:40.494 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-26T20:26:40.503 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-04-26T20:26:40.513 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-26T20:26:40.523 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-04-26T20:26:40.576 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-26T20:26:40.625 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-04-26T20:26:40.634 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-26T20:26:40.643 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-04-26T20:26:40.652 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-26T20:26:40.752 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-04-26T20:26:40.762 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-26T20:26:40.770 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-04-26T20:26:40.779 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-26T20:26:40.788 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-04-26T20:26:40.796 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-26T20:26:40.855 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-04-26T20:26:40.864 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-26T20:26:40.872 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-04-26T20:26:40.881 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-26T20:26:40.890 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-04-26T20:26:40.899 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-26T20:26:40.962 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-04-26T20:26:41.025 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-26T20:26:41.033 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-04-26T20:26:41.041 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-26T20:26:41.048 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-04-26T20:26:41.128 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-26T20:26:41.137 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-04-26T20:26:41.144 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-26T20:26:41.150 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-04-26T20:26:41.157 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-26T20:26:41.163 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-04-26T20:26:41.227 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-26T20:26:41.234 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-04-26T20:26:41.243 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-26T20:26:41.252 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-04-26T20:26:41.260 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-26T20:26:41.267 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-04-26T20:26:41.311 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-26T20:26:41.375 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-04-26T20:26:41.382 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-26T20:26:41.434 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-04-26T20:26:41.440 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-26T20:26:41.499 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-04-26T20:26:41.505 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-26T20:26:41.511 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-04-26T20:26:41.518 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-26T20:26:41.525 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-04-26T20:26:41.531 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-26T20:26:41.579 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-04-26T20:26:41.587 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-26T20:26:41.594 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-04-26T20:26:41.600 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-26T20:26:41.607 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-04-26T20:26:41.615 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-26T20:26:41.646 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-04-26T20:26:41.708 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-26T20:26:41.715 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-04-26T20:26:41.722 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-26T20:26:41.729 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-04-26T20:26:41.792 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-26T20:26:41.798 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-04-26T20:26:41.806 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-26T20:26:41.815 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-04-26T20:26:41.823 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-26T20:26:41.829 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-04-26T20:26:41.881 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-26T20:26:41.890 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-04-26T20:26:41.899 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-26T20:26:41.908 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-04-26T20:26:41.917 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-26T20:26:41.926 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-04-26T20:26:41.952 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-26T20:26:42.017 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-04-26T20:26:42.022 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-26T20:26:42.032 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-04-26T20:26:42.041 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-26T20:26:42.140 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-04-26T20:26:42.149 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-26T20:26:42.158 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-04-26T20:26:42.168 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-26T20:26:42.177 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-04-26T20:26:42.187 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-26T20:26:42.243 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-04-26T20:26:42.252 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-26T20:26:42.260 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-04-26T20:26:42.269 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-26T20:26:42.278 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-04-26T20:26:42.286 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-26T20:26:42.295 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-04-26T20:26:42.358 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-26T20:26:42.367 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-04-26T20:26:42.378 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-26T20:26:42.386 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-04-26T20:26:42.464 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-26T20:26:42.473 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-04-26T20:26:42.483 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-26T20:26:42.490 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-04-26T20:26:42.496 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-26T20:26:42.503 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-04-26T20:26:42.558 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-26T20:26:42.566 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-04-26T20:26:42.574 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-26T20:26:42.582 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-04-26T20:26:42.589 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-26T20:26:42.596 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-04-26T20:26:42.630 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-26T20:26:42.693 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-04-26T20:26:42.699 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-26T20:26:42.706 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-04-26T20:26:42.712 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-26T20:26:42.756 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-04-26T20:26:42.764 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-26T20:26:42.771 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-04-26T20:26:42.778 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-26T20:26:42.785 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-04-26T20:26:42.793 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-26T20:26:42.854 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-04-26T20:26:42.861 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-26T20:26:42.868 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-04-26T20:26:42.874 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-26T20:26:42.880 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-04-26T20:26:42.885 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-26T20:26:42.912 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-04-26T20:26:42.958 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-26T20:26:42.964 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-04-26T20:26:42.971 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-26T20:26:42.978 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-04-26T20:26:43.037 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-26T20:26:43.044 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-04-26T20:26:43.052 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-26T20:26:43.058 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-04-26T20:26:43.064 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-26T20:26:43.072 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-04-26T20:26:43.124 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-26T20:26:43.131 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-04-26T20:26:43.138 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-26T20:26:43.145 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-04-26T20:26:43.152 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-26T20:26:43.158 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-04-26T20:26:43.182 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-26T20:26:43.249 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-04-26T20:26:43.255 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-26T20:26:43.264 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-04-26T20:26:43.273 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-26T20:26:43.356 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-04-26T20:26:43.365 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-26T20:26:43.374 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-04-26T20:26:43.383 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-26T20:26:43.393 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-04-26T20:26:43.402 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-26T20:26:43.459 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-04-26T20:26:43.470 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-26T20:26:43.479 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-04-26T20:26:43.488 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-26T20:26:43.496 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-04-26T20:26:43.505 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-26T20:26:43.539 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-04-26T20:26:43.633 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-26T20:26:43.642 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-04-26T20:26:43.652 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-26T20:26:43.660 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-04-26T20:26:43.714 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-26T20:26:43.723 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-04-26T20:26:43.732 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-26T20:26:43.740 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-04-26T20:26:43.748 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-26T20:26:43.757 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-04-26T20:26:43.812 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-26T20:26:43.821 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-04-26T20:26:43.828 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-26T20:26:43.836 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-04-26T20:26:43.845 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-26T20:26:43.853 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-04-26T20:26:43.885 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-26T20:26:43.943 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-04-26T20:26:43.952 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-26T20:26:43.961 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-04-26T20:26:43.969 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-26T20:26:44.045 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-04-26T20:26:44.052 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-26T20:26:44.060 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-04-26T20:26:44.067 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-26T20:26:44.074 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-04-26T20:26:44.081 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-26T20:26:44.120 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-04-26T20:26:44.128 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-26T20:26:44.135 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-04-26T20:26:44.142 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-26T20:26:44.150 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-04-26T20:26:44.157 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-26T20:26:44.185 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-04-26T20:26:44.241 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-26T20:26:44.248 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-04-26T20:26:44.256 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-26T20:26:44.264 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-04-26T20:26:44.306 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-26T20:26:44.313 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-04-26T20:26:44.321 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-26T20:26:44.327 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-04-26T20:26:44.333 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-26T20:26:44.339 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-04-26T20:26:44.397 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-26T20:26:44.405 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-04-26T20:26:44.412 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-26T20:26:44.419 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-04-26T20:26:44.427 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-26T20:26:44.435 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-04-26T20:26:44.475 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-26T20:26:44.543 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-04-26T20:26:44.551 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-26T20:26:44.558 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-04-26T20:26:44.567 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-26T20:26:44.605 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-04-26T20:26:44.612 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-26T20:26:44.619 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-04-26T20:26:44.626 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-26T20:26:44.634 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-04-26T20:26:44.640 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-26T20:26:44.687 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-04-26T20:26:44.694 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-26T20:26:44.700 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-04-26T20:26:44.707 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-26T20:26:44.714 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-04-26T20:26:44.721 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-26T20:26:44.809 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-04-26T20:26:44.872 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-26T20:26:44.880 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-04-26T20:26:44.888 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-26T20:26:44.899 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-04-26T20:26:44.990 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-26T20:26:44.999 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-04-26T20:26:45.008 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-26T20:26:45.017 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-04-26T20:26:45.026 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-26T20:26:45.035 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-04-26T20:26:45.106 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-26T20:26:45.114 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-04-26T20:26:45.123 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-26T20:26:45.132 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-04-26T20:26:45.140 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-26T20:26:45.148 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-04-26T20:26:45.220 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-26T20:26:45.259 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-04-26T20:26:45.289 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-26T20:26:45.298 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-04-26T20:26:45.307 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-26T20:26:45.355 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-04-26T20:26:45.362 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-26T20:26:45.370 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-04-26T20:26:45.379 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-26T20:26:45.386 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-04-26T20:26:45.395 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-26T20:26:45.467 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-04-26T20:26:45.475 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-26T20:26:45.482 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-04-26T20:26:45.491 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-26T20:26:45.500 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-04-26T20:26:45.508 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-26T20:26:45.546 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-04-26T20:26:45.598 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-26T20:26:45.605 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-04-26T20:26:45.612 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-26T20:26:45.620 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-04-26T20:26:45.676 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-26T20:26:45.683 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-04-26T20:26:45.691 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-26T20:26:45.697 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-04-26T20:26:45.703 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-26T20:26:45.709 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-04-26T20:26:45.773 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-26T20:26:45.781 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-04-26T20:26:45.789 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-26T20:26:45.796 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-04-26T20:26:45.804 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-26T20:26:45.811 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-04-26T20:26:45.868 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-26T20:26:45.913 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-04-26T20:26:45.921 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-26T20:26:45.928 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-04-26T20:26:45.936 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-26T20:26:45.997 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-04-26T20:26:46.003 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-26T20:26:46.010 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-04-26T20:26:46.018 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-26T20:26:46.025 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-04-26T20:26:46.034 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-26T20:26:46.091 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-04-26T20:26:46.098 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-26T20:26:46.105 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-04-26T20:26:46.112 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-26T20:26:46.119 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-04-26T20:26:46.126 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-26T20:26:46.189 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-04-26T20:26:46.243 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-26T20:26:46.250 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-04-26T20:26:46.258 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-26T20:26:46.265 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-04-26T20:26:46.315 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-26T20:26:46.325 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-04-26T20:26:46.334 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-26T20:26:46.343 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-04-26T20:26:46.354 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-26T20:26:46.360 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-04-26T20:26:46.424 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-26T20:26:46.431 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-04-26T20:26:46.439 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-26T20:26:46.448 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-04-26T20:26:46.458 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-26T20:26:46.466 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-04-26T20:26:46.502 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-26T20:26:46.556 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-04-26T20:26:46.563 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-26T20:26:46.571 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-04-26T20:26:46.580 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-26T20:26:46.620 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-04-26T20:26:46.634 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-26T20:26:46.643 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-04-26T20:26:46.649 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-26T20:26:46.658 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-04-26T20:26:46.666 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-26T20:26:46.712 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-04-26T20:26:46.718 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-26T20:26:46.726 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-04-26T20:26:46.735 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-26T20:26:46.743 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-04-26T20:26:46.752 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-26T20:26:46.786 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-04-26T20:26:46.848 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-26T20:26:46.856 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-04-26T20:26:46.865 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-26T20:26:46.874 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-04-26T20:26:46.912 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-26T20:26:46.920 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-04-26T20:26:46.930 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-26T20:26:46.938 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-04-26T20:26:46.947 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-26T20:26:46.956 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-04-26T20:26:47.015 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-26T20:26:47.024 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-04-26T20:26:47.033 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-26T20:26:47.040 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-04-26T20:26:47.047 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-26T20:26:47.054 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-04-26T20:26:47.111 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-26T20:26:47.156 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-04-26T20:26:47.163 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-26T20:26:47.170 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-04-26T20:26:47.178 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-26T20:26:47.213 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-04-26T20:26:47.220 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-26T20:26:47.226 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-04-26T20:26:47.233 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-26T20:26:47.240 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-04-26T20:26:47.247 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-26T20:26:47.304 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-04-26T20:26:47.310 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-26T20:26:47.318 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-04-26T20:26:47.326 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-26T20:26:47.334 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-04-26T20:26:47.341 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-26T20:26:47.377 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-04-26T20:26:47.438 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-26T20:26:47.446 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-04-26T20:26:47.454 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-26T20:26:47.461 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-04-26T20:26:47.513 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-26T20:26:47.519 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-04-26T20:26:47.526 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-26T20:26:47.532 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-04-26T20:26:47.540 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-26T20:26:47.549 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-04-26T20:26:47.594 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-26T20:26:47.601 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-04-26T20:26:47.608 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-26T20:26:47.615 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-04-26T20:26:47.621 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-26T20:26:47.628 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-04-26T20:26:47.669 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-26T20:26:47.714 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-04-26T20:26:47.721 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-26T20:26:47.728 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-04-26T20:26:47.735 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-26T20:26:47.788 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-04-26T20:26:47.797 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-26T20:26:47.806 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-04-26T20:26:47.815 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-26T20:26:47.824 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-04-26T20:26:47.833 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-26T20:26:47.869 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-04-26T20:26:47.901 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-26T20:26:47.911 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-04-26T20:26:47.920 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-26T20:26:47.929 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-04-26T20:26:47.936 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-26T20:26:47.985 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-04-26T20:26:48.031 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-26T20:26:48.037 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-04-26T20:26:48.046 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-26T20:26:48.054 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-04-26T20:26:48.104 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-26T20:26:48.112 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-04-26T20:26:48.122 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-26T20:26:48.130 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-04-26T20:26:48.139 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-26T20:26:48.147 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-04-26T20:26:48.203 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-26T20:26:48.212 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-04-26T20:26:48.218 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-26T20:26:48.227 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-04-26T20:26:48.236 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-26T20:26:48.244 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-04-26T20:26:48.285 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-26T20:26:48.330 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-04-26T20:26:48.337 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-26T20:26:48.351 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-04-26T20:26:48.360 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-26T20:26:48.412 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-04-26T20:26:48.421 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-26T20:26:48.429 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-04-26T20:26:48.437 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-26T20:26:48.444 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-04-26T20:26:48.453 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-26T20:26:48.509 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-04-26T20:26:48.518 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-26T20:26:48.526 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-04-26T20:26:48.534 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-26T20:26:48.542 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-04-26T20:26:48.549 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-26T20:26:48.593 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-04-26T20:26:48.639 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-26T20:26:48.645 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-04-26T20:26:48.652 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-26T20:26:48.660 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-04-26T20:26:48.721 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-26T20:26:48.729 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-04-26T20:26:48.737 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-26T20:26:48.745 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-04-26T20:26:48.753 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-26T20:26:48.760 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-04-26T20:26:48.802 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-26T20:26:48.825 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-04-26T20:26:48.832 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-26T20:26:48.839 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-04-26T20:26:48.849 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-26T20:26:48.860 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-04-26T20:26:48.907 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-26T20:26:48.986 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-04-26T20:26:48.993 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-26T20:26:49.001 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-04-26T20:26:49.009 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-26T20:26:49.057 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-04-26T20:26:49.068 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-26T20:26:49.100 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-04-26T20:26:49.107 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-26T20:26:49.114 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-04-26T20:26:49.121 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-26T20:26:49.169 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-04-26T20:26:49.177 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-26T20:26:49.184 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-04-26T20:26:49.190 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-26T20:26:49.198 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-04-26T20:26:49.207 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-26T20:26:49.261 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-04-26T20:26:49.322 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-26T20:26:49.329 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-04-26T20:26:49.337 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-26T20:26:49.344 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-04-26T20:26:49.387 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-26T20:26:49.396 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-04-26T20:26:49.405 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-26T20:26:49.414 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-04-26T20:26:49.425 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-26T20:26:49.434 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-04-26T20:26:49.504 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-26T20:26:49.511 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-04-26T20:26:49.520 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-26T20:26:49.528 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-04-26T20:26:49.537 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-26T20:26:49.546 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-04-26T20:26:49.585 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-26T20:26:49.636 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-04-26T20:26:49.646 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-26T20:26:49.654 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-04-26T20:26:49.664 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-26T20:26:49.744 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-04-26T20:26:49.754 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-26T20:26:49.763 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-04-26T20:26:49.772 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-26T20:26:49.781 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-04-26T20:26:49.790 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-26T20:26:49.827 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-04-26T20:26:49.850 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-26T20:26:49.860 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-04-26T20:26:49.870 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-26T20:26:49.877 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-04-26T20:26:49.886 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-26T20:26:49.937 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-04-26T20:26:49.980 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-26T20:26:49.989 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-04-26T20:26:49.997 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-26T20:26:50.006 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-04-26T20:26:50.046 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-26T20:26:50.054 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-04-26T20:26:50.063 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-26T20:26:50.071 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-04-26T20:26:50.079 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-26T20:26:50.088 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-04-26T20:26:50.135 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-26T20:26:50.144 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-04-26T20:26:50.153 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-26T20:26:50.161 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-04-26T20:26:50.168 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-26T20:26:50.176 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-04-26T20:26:50.218 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-26T20:26:50.263 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-04-26T20:26:50.272 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-26T20:26:50.279 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-04-26T20:26:50.286 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-26T20:26:50.386 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-04-26T20:26:50.392 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-26T20:26:50.400 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-04-26T20:26:50.406 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-26T20:26:50.413 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-04-26T20:26:50.419 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-26T20:26:50.480 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-04-26T20:26:50.487 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-26T20:26:50.494 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-04-26T20:26:50.502 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-26T20:26:50.509 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-04-26T20:26:50.515 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-26T20:26:50.559 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-04-26T20:26:50.604 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-26T20:26:50.611 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-04-26T20:26:50.619 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-26T20:26:50.626 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-04-26T20:26:50.662 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-26T20:26:50.670 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-04-26T20:26:50.678 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-26T20:26:50.685 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-04-26T20:26:50.692 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-26T20:26:50.699 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-04-26T20:26:50.754 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-26T20:26:50.761 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-04-26T20:26:50.771 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-26T20:26:50.780 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-04-26T20:26:50.831 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-26T20:26:50.847 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-04-26T20:26:50.919 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-26T20:26:50.985 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-04-26T20:26:50.993 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-26T20:26:51.002 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-04-26T20:26:51.011 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-26T20:26:51.061 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-04-26T20:26:51.069 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-26T20:26:51.077 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-04-26T20:26:51.086 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-26T20:26:51.094 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-04-26T20:26:51.103 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-26T20:26:51.158 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-04-26T20:26:51.164 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-26T20:26:51.173 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-04-26T20:26:51.182 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-26T20:26:51.191 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-04-26T20:26:51.204 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-26T20:26:51.270 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-04-26T20:26:51.343 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-26T20:26:51.355 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-04-26T20:26:51.364 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-26T20:26:51.372 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-04-26T20:26:51.415 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-26T20:26:51.424 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-04-26T20:26:51.433 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-26T20:26:51.443 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-04-26T20:26:51.452 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-26T20:26:51.461 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-04-26T20:26:51.515 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-26T20:26:51.526 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-04-26T20:26:51.535 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-26T20:26:51.544 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-04-26T20:26:51.553 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-26T20:26:51.562 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-04-26T20:26:51.593 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-26T20:26:51.629 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-04-26T20:26:51.638 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-26T20:26:51.647 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-04-26T20:26:51.656 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-26T20:26:51.702 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-04-26T20:26:51.711 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-26T20:26:51.720 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-04-26T20:26:51.736 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-26T20:26:51.744 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-04-26T20:26:51.753 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-26T20:26:51.807 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-04-26T20:26:51.816 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-26T20:26:51.824 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-04-26T20:26:51.833 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-26T20:26:51.841 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-04-26T20:26:51.848 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-26T20:26:51.895 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-04-26T20:26:51.945 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-26T20:26:51.952 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-04-26T20:26:51.961 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-26T20:26:51.968 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-04-26T20:26:51.998 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-26T20:26:52.006 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-04-26T20:26:52.015 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-26T20:26:52.022 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-04-26T20:26:52.028 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-26T20:26:52.036 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-04-26T20:26:52.084 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-26T20:26:52.091 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-04-26T20:26:52.098 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-26T20:26:52.105 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-04-26T20:26:52.112 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-26T20:26:52.119 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-04-26T20:26:52.167 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-26T20:26:52.215 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-04-26T20:26:52.222 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-26T20:26:52.229 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-04-26T20:26:52.239 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-26T20:26:52.281 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-04-26T20:26:52.289 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-26T20:26:52.296 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-04-26T20:26:52.303 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-26T20:26:52.310 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-04-26T20:26:52.318 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-26T20:26:52.368 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-04-26T20:26:52.377 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-26T20:26:52.384 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-04-26T20:26:52.390 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-26T20:26:52.396 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-04-26T20:26:52.403 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-26T20:26:52.446 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-04-26T20:26:52.496 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-26T20:26:52.506 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-04-26T20:26:52.515 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-26T20:26:52.524 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-04-26T20:26:52.555 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-26T20:26:52.562 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-04-26T20:26:52.569 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-26T20:26:52.578 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-04-26T20:26:52.587 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-26T20:26:52.595 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-04-26T20:26:52.635 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-26T20:26:52.644 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-04-26T20:26:52.653 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-26T20:26:52.663 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-04-26T20:26:52.672 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-26T20:26:52.680 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-04-26T20:26:52.727 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-26T20:26:52.772 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-04-26T20:26:52.778 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-26T20:26:52.787 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-04-26T20:26:52.796 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-26T20:26:52.832 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-04-26T20:26:52.842 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-26T20:26:52.851 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-04-26T20:26:52.860 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-26T20:26:52.868 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-04-26T20:26:52.877 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-26T20:26:52.916 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-04-26T20:26:52.939 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-26T20:26:52.945 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-04-26T20:26:52.954 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-26T20:26:52.963 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-04-26T20:26:52.971 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-26T20:26:53.015 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-04-26T20:26:53.053 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-26T20:26:53.060 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-04-26T20:26:53.069 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-26T20:26:53.078 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-04-26T20:26:53.125 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-26T20:26:53.134 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-04-26T20:26:53.145 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-26T20:26:53.154 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-04-26T20:26:53.163 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-26T20:26:53.172 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-04-26T20:26:53.231 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-26T20:26:53.240 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-04-26T20:26:53.248 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-26T20:26:53.254 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-04-26T20:26:53.260 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-26T20:26:53.267 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-04-26T20:26:53.326 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-26T20:26:53.363 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-04-26T20:26:53.370 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-26T20:26:53.378 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-04-26T20:26:53.386 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-26T20:26:53.440 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-04-26T20:26:53.447 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-26T20:26:53.455 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-04-26T20:26:53.464 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-26T20:26:53.471 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-04-26T20:26:53.479 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-26T20:26:53.549 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-04-26T20:26:53.558 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-26T20:26:53.564 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-04-26T20:26:53.571 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-26T20:26:53.578 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-04-26T20:26:53.584 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-26T20:26:53.632 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-04-26T20:26:53.680 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-26T20:26:53.687 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-04-26T20:26:53.695 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-26T20:26:53.702 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-04-26T20:26:53.781 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-26T20:26:53.788 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-04-26T20:26:53.796 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-26T20:26:53.803 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-04-26T20:26:53.810 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-26T20:26:53.817 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-04-26T20:26:53.868 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-26T20:26:53.875 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-04-26T20:26:53.883 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-26T20:26:53.889 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-04-26T20:26:53.897 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-26T20:26:53.904 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-04-26T20:26:53.956 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-26T20:26:54.004 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-04-26T20:26:54.013 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-26T20:26:54.021 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-04-26T20:26:54.030 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-26T20:26:54.085 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-04-26T20:26:54.094 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-26T20:26:54.102 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-04-26T20:26:54.109 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-26T20:26:54.119 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-04-26T20:26:54.128 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-26T20:26:54.176 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-04-26T20:26:54.185 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-26T20:26:54.194 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-04-26T20:26:54.203 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-26T20:26:54.211 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-04-26T20:26:54.221 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-26T20:26:54.259 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-04-26T20:26:54.317 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-26T20:26:54.326 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-04-26T20:26:54.335 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-26T20:26:54.341 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-04-26T20:26:54.423 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-26T20:26:54.429 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-04-26T20:26:54.438 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-26T20:26:54.448 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-04-26T20:26:54.456 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-26T20:26:54.463 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-04-26T20:26:54.521 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-26T20:26:54.529 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-04-26T20:26:54.537 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-26T20:26:54.546 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-04-26T20:26:54.555 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-26T20:26:54.564 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-04-26T20:26:54.627 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-26T20:26:54.680 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-04-26T20:26:54.686 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-26T20:26:54.691 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-04-26T20:26:54.699 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-26T20:26:54.751 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-04-26T20:26:54.758 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-26T20:26:54.765 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-04-26T20:26:54.772 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-26T20:26:54.780 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-04-26T20:26:54.787 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-26T20:26:54.857 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-04-26T20:26:54.864 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-26T20:26:54.871 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-04-26T20:26:54.880 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-26T20:26:54.889 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-04-26T20:26:54.896 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-26T20:26:54.943 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-04-26T20:26:54.996 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-26T20:26:55.003 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-04-26T20:26:55.011 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-26T20:26:55.018 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-04-26T20:26:55.066 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-26T20:26:55.072 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-04-26T20:26:55.079 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-26T20:26:55.087 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-04-26T20:26:55.094 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-26T20:26:55.101 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-04-26T20:26:55.239 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-26T20:26:55.246 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-04-26T20:26:55.254 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-26T20:26:55.261 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-04-26T20:26:55.269 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-26T20:26:55.276 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-04-26T20:26:55.378 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-26T20:26:55.433 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-04-26T20:26:55.441 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-26T20:26:55.450 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-04-26T20:26:55.460 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-26T20:26:55.537 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-04-26T20:26:55.546 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-26T20:26:55.555 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-04-26T20:26:55.564 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-26T20:26:55.571 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-04-26T20:26:55.579 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-26T20:26:55.645 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-04-26T20:26:55.652 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-26T20:26:55.661 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-04-26T20:26:55.670 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-26T20:26:55.678 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-04-26T20:26:55.691 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-26T20:26:55.758 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-04-26T20:26:55.796 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-26T20:26:55.804 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-04-26T20:26:55.813 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-26T20:26:55.822 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-04-26T20:26:55.894 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-26T20:26:55.903 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-04-26T20:26:55.914 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-26T20:26:55.923 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-04-26T20:26:55.932 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-26T20:26:55.941 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-04-26T20:26:56.000 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-26T20:26:56.009 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-04-26T20:26:56.017 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-26T20:26:56.024 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-04-26T20:26:56.033 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-26T20:26:56.041 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-04-26T20:26:56.091 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-26T20:26:56.129 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-04-26T20:26:56.138 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-26T20:26:56.146 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-04-26T20:26:56.152 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-26T20:26:56.200 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-04-26T20:26:56.206 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-26T20:26:56.216 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-04-26T20:26:56.225 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-26T20:26:56.231 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-04-26T20:26:56.238 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-26T20:26:56.299 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-04-26T20:26:56.307 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-26T20:26:56.314 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-04-26T20:26:56.322 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-26T20:26:56.329 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-04-26T20:26:56.337 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-26T20:26:56.393 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-04-26T20:26:56.439 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-26T20:26:56.447 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-04-26T20:26:56.455 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-26T20:26:56.462 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-04-26T20:26:56.500 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-26T20:26:56.506 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-04-26T20:26:56.514 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-26T20:26:56.522 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-04-26T20:26:56.530 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-26T20:26:56.538 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-04-26T20:26:56.594 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-26T20:26:56.602 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-04-26T20:26:56.610 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-26T20:26:56.617 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-04-26T20:26:56.626 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-26T20:26:56.633 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-04-26T20:26:56.666 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-26T20:26:56.709 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-04-26T20:26:56.717 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-26T20:26:56.725 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-04-26T20:26:56.734 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-26T20:26:56.802 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-04-26T20:26:56.810 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-26T20:26:56.817 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-04-26T20:26:56.825 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-26T20:26:56.832 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-04-26T20:26:56.847 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-26T20:26:56.904 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-04-26T20:26:56.911 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-26T20:26:56.917 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-04-26T20:26:56.928 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-26T20:26:56.937 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-04-26T20:26:56.946 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-26T20:26:57.000 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-04-26T20:26:57.047 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-26T20:26:57.056 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-04-26T20:26:57.064 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-26T20:26:57.073 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-04-26T20:26:57.135 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-26T20:26:57.144 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-04-26T20:26:57.153 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-26T20:26:57.162 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-04-26T20:26:57.170 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-26T20:26:57.180 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-04-26T20:26:57.244 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-26T20:26:57.252 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-04-26T20:26:57.261 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-26T20:26:57.272 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-04-26T20:26:57.281 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-26T20:26:57.290 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-04-26T20:26:57.349 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-26T20:26:57.393 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-04-26T20:26:57.401 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-26T20:26:57.407 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-04-26T20:26:57.413 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-26T20:26:57.458 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-04-26T20:26:57.468 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-26T20:26:57.476 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-04-26T20:26:57.485 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-26T20:26:57.494 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-04-26T20:26:57.502 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-26T20:26:57.559 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-04-26T20:26:57.567 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-26T20:26:57.575 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-04-26T20:26:57.585 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-26T20:26:57.593 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-04-26T20:26:57.602 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-26T20:26:57.638 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-04-26T20:26:57.693 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-26T20:26:57.701 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-04-26T20:26:57.710 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-26T20:26:57.719 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-04-26T20:26:57.836 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-26T20:26:57.845 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-04-26T20:26:57.853 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-26T20:26:57.861 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-04-26T20:26:57.868 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-26T20:26:57.875 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-04-26T20:26:57.936 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-26T20:26:57.943 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-04-26T20:26:57.949 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-26T20:26:57.959 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-04-26T20:26:57.965 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-26T20:26:57.973 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-04-26T20:26:58.007 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-26T20:26:58.056 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-04-26T20:26:58.063 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-26T20:26:58.069 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-04-26T20:26:58.077 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-26T20:26:58.125 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-04-26T20:26:58.132 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-26T20:26:58.139 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-04-26T20:26:58.146 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-26T20:26:58.154 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-04-26T20:26:58.162 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-26T20:26:58.228 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-04-26T20:26:58.236 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-26T20:26:58.244 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-04-26T20:26:58.250 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-26T20:26:58.257 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-04-26T20:26:58.263 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-26T20:26:58.308 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-04-26T20:26:58.368 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-26T20:26:58.374 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-04-26T20:26:58.382 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-26T20:26:58.388 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-04-26T20:26:58.489 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-26T20:26:58.496 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-04-26T20:26:58.503 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-26T20:26:58.510 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-04-26T20:26:58.520 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-26T20:26:58.530 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-04-26T20:26:58.583 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-26T20:26:58.592 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-04-26T20:26:58.600 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-26T20:26:58.608 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-04-26T20:26:58.616 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-26T20:26:58.626 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-04-26T20:26:58.700 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-26T20:26:58.785 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-04-26T20:26:58.793 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-26T20:26:58.802 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-04-26T20:26:58.811 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-26T20:26:58.866 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-04-26T20:26:58.875 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-26T20:26:58.884 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-04-26T20:26:58.892 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-26T20:26:58.901 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-04-26T20:26:58.907 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-26T20:26:58.995 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-04-26T20:26:59.005 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-26T20:26:59.013 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-04-26T20:26:59.021 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-26T20:26:59.029 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-04-26T20:26:59.036 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-26T20:26:59.096 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-04-26T20:26:59.143 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-26T20:26:59.149 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-04-26T20:26:59.157 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-26T20:26:59.164 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-04-26T20:26:59.220 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-26T20:26:59.228 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-04-26T20:26:59.235 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-26T20:26:59.243 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-04-26T20:26:59.250 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-26T20:26:59.258 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-04-26T20:26:59.329 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-26T20:26:59.338 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-04-26T20:26:59.346 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-26T20:26:59.353 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-04-26T20:26:59.360 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-26T20:26:59.367 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-04-26T20:26:59.407 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-26T20:26:59.455 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-04-26T20:26:59.463 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-26T20:26:59.469 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-04-26T20:26:59.476 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-26T20:26:59.557 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-04-26T20:26:59.565 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-26T20:26:59.573 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-04-26T20:26:59.580 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-26T20:26:59.588 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-04-26T20:26:59.595 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-26T20:26:59.662 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-04-26T20:26:59.669 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-26T20:26:59.678 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-04-26T20:26:59.684 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-26T20:26:59.691 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-04-26T20:26:59.700 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-26T20:26:59.740 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-04-26T20:26:59.795 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-26T20:26:59.802 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-04-26T20:26:59.811 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-26T20:26:59.819 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-04-26T20:26:59.873 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-26T20:26:59.882 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-04-26T20:26:59.893 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-26T20:26:59.903 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-04-26T20:26:59.911 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-26T20:26:59.920 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-04-26T20:26:59.976 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-26T20:26:59.983 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-04-26T20:26:59.990 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-26T20:26:59.997 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-04-26T20:27:00.004 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-26T20:27:00.012 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-04-26T20:27:00.065 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-26T20:27:00.143 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-04-26T20:27:00.148 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-26T20:27:00.157 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-04-26T20:27:00.167 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-26T20:27:00.214 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-04-26T20:27:00.223 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-26T20:27:00.233 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-04-26T20:27:00.241 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-26T20:27:00.251 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-04-26T20:27:00.260 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-26T20:27:00.348 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-04-26T20:27:00.355 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-26T20:27:00.364 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-04-26T20:27:00.372 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-26T20:27:00.379 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-04-26T20:27:00.386 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-26T20:27:00.436 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-04-26T20:27:00.518 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-26T20:27:00.526 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-04-26T20:27:00.533 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-26T20:27:00.540 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-04-26T20:27:00.600 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-26T20:27:00.608 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-04-26T20:27:00.615 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-26T20:27:00.623 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-04-26T20:27:00.630 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-26T20:27:00.638 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-04-26T20:27:00.711 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-26T20:27:00.718 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-04-26T20:27:00.725 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-26T20:27:00.732 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-04-26T20:27:00.740 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-26T20:27:00.747 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-04-26T20:27:00.798 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-26T20:27:00.847 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-04-26T20:27:00.854 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-26T20:27:00.861 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-04-26T20:27:00.869 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-26T20:27:00.970 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-04-26T20:27:00.976 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-26T20:27:00.984 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-04-26T20:27:00.992 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-26T20:27:00.999 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-04-26T20:27:01.006 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-26T20:27:01.063 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-04-26T20:27:01.070 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-26T20:27:01.077 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-04-26T20:27:01.083 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-26T20:27:01.091 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-04-26T20:27:01.101 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-26T20:27:01.173 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-04-26T20:27:01.263 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-26T20:27:01.274 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-04-26T20:27:01.283 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-26T20:27:01.291 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-04-26T20:27:01.340 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-26T20:27:01.348 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-04-26T20:27:01.357 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-26T20:27:01.366 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-04-26T20:27:01.376 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-26T20:27:01.385 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-04-26T20:27:01.443 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-26T20:27:01.451 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-04-26T20:27:01.458 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-26T20:27:01.465 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-04-26T20:27:01.472 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-26T20:27:01.480 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-04-26T20:27:01.531 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-26T20:27:01.595 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-04-26T20:27:01.604 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-26T20:27:01.615 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-04-26T20:27:01.624 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-26T20:27:01.673 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-04-26T20:27:01.680 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-26T20:27:01.689 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-04-26T20:27:01.699 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-26T20:27:01.708 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-04-26T20:27:01.717 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-26T20:27:01.763 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-04-26T20:27:01.779 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-26T20:27:01.788 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-04-26T20:27:01.797 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-26T20:27:01.806 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-04-26T20:27:01.815 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-26T20:27:01.848 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-04-26T20:27:01.901 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-26T20:27:01.910 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-04-26T20:27:01.919 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-26T20:27:01.929 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-04-26T20:27:01.981 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-26T20:27:01.988 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-04-26T20:27:01.996 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-26T20:27:02.003 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-04-26T20:27:02.011 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-26T20:27:02.018 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-04-26T20:27:02.055 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-26T20:27:02.061 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-04-26T20:27:02.068 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-26T20:27:02.075 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-04-26T20:27:02.082 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-26T20:27:02.090 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-04-26T20:27:02.145 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-26T20:27:02.186 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-04-26T20:27:02.193 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-26T20:27:02.201 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-04-26T20:27:02.208 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-26T20:27:02.260 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-04-26T20:27:02.268 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-26T20:27:02.276 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-04-26T20:27:02.283 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-26T20:27:02.291 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-04-26T20:27:02.298 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-26T20:27:02.375 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-04-26T20:27:02.382 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-26T20:27:02.388 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-04-26T20:27:02.395 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-26T20:27:02.401 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-04-26T20:27:02.411 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-26T20:27:02.490 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-04-26T20:27:02.527 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-26T20:27:02.546 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-04-26T20:27:02.554 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-26T20:27:02.562 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-04-26T20:27:02.606 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-26T20:27:02.614 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-04-26T20:27:02.621 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-26T20:27:02.628 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-04-26T20:27:02.635 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-26T20:27:02.642 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-04-26T20:27:02.710 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-26T20:27:02.717 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-04-26T20:27:02.724 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-26T20:27:02.731 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-04-26T20:27:02.739 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-26T20:27:02.747 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-04-26T20:27:02.789 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-26T20:27:02.843 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-04-26T20:27:02.852 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-26T20:27:02.863 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-04-26T20:27:02.872 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-26T20:27:02.933 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-04-26T20:27:02.942 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-26T20:27:02.951 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-04-26T20:27:02.960 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-26T20:27:02.969 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-04-26T20:27:02.978 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-26T20:27:03.021 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-04-26T20:27:03.029 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-26T20:27:03.038 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-04-26T20:27:03.047 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-26T20:27:03.054 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-04-26T20:27:03.062 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-26T20:27:03.123 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-04-26T20:27:03.163 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-26T20:27:03.172 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-04-26T20:27:03.180 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-26T20:27:03.189 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-04-26T20:27:03.240 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-26T20:27:03.249 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-04-26T20:27:03.258 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-26T20:27:03.268 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-04-26T20:27:03.278 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-26T20:27:03.287 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-04-26T20:27:03.350 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-26T20:27:03.359 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-04-26T20:27:03.368 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-26T20:27:03.377 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-04-26T20:27:03.386 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-26T20:27:03.394 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-04-26T20:27:03.456 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-26T20:27:03.496 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-04-26T20:27:03.504 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-26T20:27:03.525 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-04-26T20:27:03.531 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-26T20:27:03.589 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-04-26T20:27:03.596 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-26T20:27:03.605 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-04-26T20:27:03.613 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-26T20:27:03.621 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-04-26T20:27:03.628 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-26T20:27:03.679 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-04-26T20:27:03.687 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-26T20:27:03.694 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-04-26T20:27:03.702 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-26T20:27:03.709 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-04-26T20:27:03.716 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-26T20:27:03.778 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-04-26T20:27:03.825 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-26T20:27:03.831 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-04-26T20:27:03.838 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-26T20:27:03.847 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-04-26T20:27:03.900 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-26T20:27:03.908 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-04-26T20:27:03.915 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-26T20:27:03.921 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-04-26T20:27:03.927 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-26T20:27:03.933 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-04-26T20:27:04.000 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-26T20:27:04.009 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-04-26T20:27:04.015 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-26T20:27:04.021 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-04-26T20:27:04.028 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-26T20:27:04.034 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-04-26T20:27:04.098 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-26T20:27:04.146 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-04-26T20:27:04.151 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-26T20:27:04.160 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-04-26T20:27:04.166 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-26T20:27:04.214 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-04-26T20:27:04.220 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-26T20:27:04.226 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-04-26T20:27:04.235 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-26T20:27:04.242 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-04-26T20:27:04.250 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-26T20:27:04.301 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-04-26T20:27:04.309 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-26T20:27:04.318 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-04-26T20:27:04.326 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-26T20:27:04.335 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-04-26T20:27:04.344 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-26T20:27:04.400 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-04-26T20:27:04.479 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-26T20:27:04.489 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-04-26T20:27:04.498 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-26T20:27:04.507 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-04-26T20:27:04.558 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-26T20:27:04.566 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-04-26T20:27:04.575 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-26T20:27:04.583 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-04-26T20:27:04.592 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-26T20:27:04.601 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-04-26T20:27:04.666 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-26T20:27:04.675 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-04-26T20:27:04.684 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-26T20:27:04.691 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-04-26T20:27:04.697 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-26T20:27:04.705 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-04-26T20:27:04.764 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-26T20:27:04.811 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-04-26T20:27:04.820 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-26T20:27:04.829 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-04-26T20:27:04.838 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-26T20:27:04.884 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-04-26T20:27:04.890 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-26T20:27:04.898 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-04-26T20:27:04.907 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-26T20:27:04.915 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-04-26T20:27:04.924 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-26T20:27:04.987 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-04-26T20:27:04.999 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-26T20:27:05.008 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-04-26T20:27:05.015 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-26T20:27:05.024 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-04-26T20:27:05.031 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-26T20:27:05.096 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-04-26T20:27:05.141 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-26T20:27:05.148 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-04-26T20:27:05.155 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-26T20:27:05.162 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-04-26T20:27:05.229 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-26T20:27:05.236 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-04-26T20:27:05.241 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-26T20:27:05.248 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-04-26T20:27:05.254 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-26T20:27:05.262 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-04-26T20:27:05.321 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-26T20:27:05.328 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-04-26T20:27:05.336 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-26T20:27:05.343 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-04-26T20:27:05.350 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-26T20:27:05.356 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-04-26T20:27:05.415 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-26T20:27:05.452 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-04-26T20:27:05.479 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-26T20:27:05.484 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-04-26T20:27:05.491 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-26T20:27:05.556 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-04-26T20:27:05.565 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-26T20:27:05.573 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-04-26T20:27:05.579 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-26T20:27:05.585 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-04-26T20:27:05.592 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-26T20:27:05.653 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-04-26T20:27:05.660 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-26T20:27:05.668 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-04-26T20:27:05.675 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-26T20:27:05.683 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-04-26T20:27:05.692 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-26T20:27:05.772 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-04-26T20:27:05.816 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-26T20:27:05.835 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-04-26T20:27:05.844 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-26T20:27:05.852 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-04-26T20:27:05.906 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-26T20:27:05.915 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-04-26T20:27:05.924 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-26T20:27:05.933 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-04-26T20:27:05.942 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-26T20:27:05.951 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-04-26T20:27:06.008 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-26T20:27:06.023 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-04-26T20:27:06.030 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-26T20:27:06.039 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-04-26T20:27:06.047 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-26T20:27:06.056 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-04-26T20:27:06.105 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-26T20:27:06.143 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-04-26T20:27:06.159 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-26T20:27:06.170 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-04-26T20:27:06.176 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-26T20:27:06.243 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-04-26T20:27:06.252 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-26T20:27:06.261 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-04-26T20:27:06.268 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-26T20:27:06.276 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-04-26T20:27:06.284 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-26T20:27:06.346 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-04-26T20:27:06.352 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-26T20:27:06.360 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-04-26T20:27:06.368 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-26T20:27:06.377 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-04-26T20:27:06.384 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-26T20:27:06.430 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-04-26T20:27:06.468 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-26T20:27:06.491 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-04-26T20:27:06.500 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-26T20:27:06.510 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-04-26T20:27:06.572 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-26T20:27:06.580 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-04-26T20:27:06.586 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-26T20:27:06.592 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-04-26T20:27:06.598 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-26T20:27:06.604 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-04-26T20:27:06.641 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-26T20:27:06.648 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-04-26T20:27:06.654 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-26T20:27:06.661 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-04-26T20:27:06.668 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-26T20:27:06.675 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-04-26T20:27:06.755 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-26T20:27:06.809 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-04-26T20:27:06.816 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-26T20:27:06.822 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-04-26T20:27:06.829 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-26T20:27:06.876 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-04-26T20:27:06.882 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-26T20:27:06.889 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-04-26T20:27:06.897 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-26T20:27:06.905 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-04-26T20:27:06.912 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-26T20:27:06.963 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-04-26T20:27:06.972 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-26T20:27:06.978 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-04-26T20:27:06.985 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-26T20:27:06.994 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-04-26T20:27:07.001 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-26T20:27:07.055 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-04-26T20:27:07.109 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-26T20:27:07.116 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-04-26T20:27:07.123 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-26T20:27:07.130 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-04-26T20:27:07.194 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-26T20:27:07.201 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-04-26T20:27:07.208 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-26T20:27:07.215 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-04-26T20:27:07.222 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-26T20:27:07.229 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-04-26T20:27:07.295 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-26T20:27:07.301 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-04-26T20:27:07.308 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-26T20:27:07.315 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-04-26T20:27:07.324 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-26T20:27:07.332 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-04-26T20:27:07.380 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-26T20:27:07.435 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-04-26T20:27:07.442 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-26T20:27:07.448 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-04-26T20:27:07.458 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-26T20:27:07.505 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-04-26T20:27:07.512 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-26T20:27:07.521 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-04-26T20:27:07.529 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-26T20:27:07.538 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-04-26T20:27:07.547 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-26T20:27:07.613 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-04-26T20:27:07.621 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-26T20:27:07.630 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-04-26T20:27:07.639 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-26T20:27:07.646 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-04-26T20:27:07.657 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-26T20:27:07.723 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-04-26T20:27:07.776 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-26T20:27:07.782 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-04-26T20:27:07.791 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-26T20:27:07.799 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-04-26T20:27:07.856 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-26T20:27:07.863 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-04-26T20:27:07.871 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-26T20:27:07.880 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-04-26T20:27:07.889 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-26T20:27:07.898 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-04-26T20:27:07.955 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-26T20:27:07.961 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-04-26T20:27:07.968 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-26T20:27:07.977 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-04-26T20:27:07.986 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-26T20:27:07.995 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-04-26T20:27:08.046 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-26T20:27:08.101 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-04-26T20:27:08.108 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-26T20:27:08.116 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-04-26T20:27:08.123 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-26T20:27:08.172 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-04-26T20:27:08.179 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-26T20:27:08.187 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-04-26T20:27:08.194 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-26T20:27:08.200 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-04-26T20:27:08.206 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-26T20:27:08.249 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-04-26T20:27:08.257 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-26T20:27:08.264 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-04-26T20:27:08.270 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-26T20:27:08.277 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-04-26T20:27:08.284 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-26T20:27:08.335 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-04-26T20:27:08.430 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-26T20:27:08.438 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-04-26T20:27:08.445 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-26T20:27:08.452 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-04-26T20:27:08.551 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-26T20:27:08.557 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-04-26T20:27:08.564 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-26T20:27:08.571 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-04-26T20:27:08.578 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-26T20:27:08.585 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-04-26T20:27:08.633 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-26T20:27:08.640 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-04-26T20:27:08.647 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-26T20:27:08.654 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-04-26T20:27:08.659 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-26T20:27:08.667 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-04-26T20:27:08.726 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-26T20:27:08.784 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-04-26T20:27:08.791 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-26T20:27:08.798 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-04-26T20:27:08.805 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-26T20:27:08.871 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-04-26T20:27:08.880 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-26T20:27:08.889 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-04-26T20:27:08.898 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-26T20:27:08.907 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-04-26T20:27:08.916 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-26T20:27:08.972 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-04-26T20:27:08.980 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-26T20:27:08.989 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-04-26T20:27:08.999 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-26T20:27:09.008 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-04-26T20:27:09.016 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-26T20:27:09.061 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-04-26T20:27:09.127 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-26T20:27:09.137 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-04-26T20:27:09.146 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-26T20:27:09.155 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-04-26T20:27:09.204 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-26T20:27:09.213 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-04-26T20:27:09.223 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-26T20:27:09.232 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-04-26T20:27:09.240 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-26T20:27:09.248 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-04-26T20:27:09.311 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-26T20:27:09.318 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-04-26T20:27:09.326 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-26T20:27:09.334 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-04-26T20:27:09.343 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-26T20:27:09.350 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-04-26T20:27:09.388 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-26T20:27:09.442 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-04-26T20:27:09.450 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-26T20:27:09.458 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-04-26T20:27:09.467 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-26T20:27:09.566 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-04-26T20:27:09.573 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-26T20:27:09.581 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-04-26T20:27:09.590 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-26T20:27:09.598 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-04-26T20:27:09.604 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-26T20:27:09.653 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-04-26T20:27:09.669 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-26T20:27:09.678 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-04-26T20:27:09.686 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-26T20:27:09.693 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-04-26T20:27:09.701 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-26T20:27:09.763 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-04-26T20:27:09.817 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-26T20:27:09.824 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-04-26T20:27:09.831 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-26T20:27:09.839 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-04-26T20:27:09.887 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-26T20:27:09.896 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-04-26T20:27:09.905 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-26T20:27:09.913 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-04-26T20:27:09.920 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-26T20:27:09.926 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-04-26T20:27:09.989 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-26T20:27:09.997 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-04-26T20:27:10.004 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-26T20:27:10.010 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-04-26T20:27:10.017 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-26T20:27:10.023 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-04-26T20:27:10.088 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-26T20:27:10.142 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-04-26T20:27:10.150 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-26T20:27:10.157 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-04-26T20:27:10.163 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-26T20:27:10.224 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-04-26T20:27:10.230 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-26T20:27:10.238 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-04-26T20:27:10.245 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-26T20:27:10.251 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-04-26T20:27:10.258 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-26T20:27:10.299 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-04-26T20:27:10.305 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-26T20:27:10.311 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-04-26T20:27:10.322 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-26T20:27:10.329 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-04-26T20:27:10.336 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-26T20:27:10.367 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-04-26T20:27:10.491 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-26T20:27:10.501 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-04-26T20:27:10.510 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-26T20:27:10.520 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-04-26T20:27:10.588 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-26T20:27:10.594 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-04-26T20:27:10.603 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-26T20:27:10.612 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-04-26T20:27:10.621 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-26T20:27:10.630 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-04-26T20:27:10.693 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-26T20:27:10.702 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-04-26T20:27:10.712 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-26T20:27:10.720 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-04-26T20:27:10.729 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-26T20:27:10.737 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-04-26T20:27:10.796 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-26T20:27:10.850 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-04-26T20:27:10.860 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-26T20:27:10.866 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-04-26T20:27:10.874 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-26T20:27:10.954 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-04-26T20:27:10.963 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-26T20:27:10.972 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-04-26T20:27:10.981 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-26T20:27:10.990 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-04-26T20:27:10.998 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-26T20:27:11.062 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-04-26T20:27:11.071 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-26T20:27:11.079 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-04-26T20:27:11.088 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-26T20:27:11.097 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-04-26T20:27:11.106 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-26T20:27:11.174 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-04-26T20:27:11.232 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-26T20:27:11.242 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-04-26T20:27:11.249 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-26T20:27:11.256 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-04-26T20:27:11.312 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-26T20:27:11.318 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-04-26T20:27:11.325 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-26T20:27:11.333 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-04-26T20:27:11.342 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-26T20:27:11.351 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-04-26T20:27:11.388 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-26T20:27:11.395 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-04-26T20:27:11.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:29 smithi131 ceph-mon[27919]: pgmap v904: 129 pgs: 129 active+clean; 866 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-04-26T20:27:11.413 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-26T20:27:11.422 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-04-26T20:27:11.429 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-26T20:27:11.484 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-04-26T20:27:11.531 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-26T20:27:11.536 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-04-26T20:27:11.543 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-26T20:27:11.550 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-04-26T20:27:11.612 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-26T20:27:11.620 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-04-26T20:27:11.627 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-26T20:27:11.633 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-04-26T20:27:11.639 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-26T20:27:11.645 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-04-26T20:27:11.701 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-26T20:27:11.708 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-04-26T20:27:11.714 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-26T20:27:11.721 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-04-26T20:27:11.727 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-26T20:27:11.734 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-04-26T20:27:11.785 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-26T20:27:11.848 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-04-26T20:27:11.855 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-26T20:27:11.863 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-04-26T20:27:11.869 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-26T20:27:11.931 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-04-26T20:27:11.939 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-26T20:27:11.948 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-04-26T20:27:11.957 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-26T20:27:11.965 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-04-26T20:27:11.974 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-26T20:27:12.033 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-04-26T20:27:12.043 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-26T20:27:12.052 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-04-26T20:27:12.060 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-26T20:27:12.070 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-04-26T20:27:12.079 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-26T20:27:12.128 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-04-26T20:27:12.183 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-26T20:27:12.192 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-04-26T20:27:12.201 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-26T20:27:12.210 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-04-26T20:27:12.259 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-26T20:27:12.265 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-04-26T20:27:12.272 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-26T20:27:12.280 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-04-26T20:27:12.288 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-26T20:27:12.298 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-04-26T20:27:12.356 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-26T20:27:12.363 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-04-26T20:27:12.371 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-26T20:27:12.380 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-04-26T20:27:12.388 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-26T20:27:12.397 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-04-26T20:27:12.446 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-26T20:27:12.499 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-04-26T20:27:12.507 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-26T20:27:12.515 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-04-26T20:27:12.524 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-26T20:27:12.564 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-04-26T20:27:12.573 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-26T20:27:12.582 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-04-26T20:27:12.590 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-26T20:27:12.598 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-04-26T20:27:12.605 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-26T20:27:12.668 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-04-26T20:27:12.677 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-26T20:27:12.683 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-04-26T20:27:12.691 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-26T20:27:12.697 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-04-26T20:27:12.704 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-26T20:27:12.761 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-04-26T20:27:12.814 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-26T20:27:12.841 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-04-26T20:27:12.849 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-26T20:27:12.856 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-04-26T20:27:12.932 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-26T20:27:12.939 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-04-26T20:27:12.947 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-26T20:27:12.954 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-04-26T20:27:12.963 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-26T20:27:12.971 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-04-26T20:27:13.032 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-26T20:27:13.039 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-04-26T20:27:13.050 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-26T20:27:13.057 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-04-26T20:27:13.064 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-26T20:27:13.070 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-04-26T20:27:13.132 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-26T20:27:13.189 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-04-26T20:27:13.196 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-26T20:27:13.203 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-04-26T20:27:13.211 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-26T20:27:13.262 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-04-26T20:27:13.269 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-26T20:27:13.277 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-04-26T20:27:13.284 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-26T20:27:13.291 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-04-26T20:27:13.299 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-26T20:27:13.354 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-04-26T20:27:13.361 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-26T20:27:13.369 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-04-26T20:27:13.376 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-26T20:27:13.385 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-04-26T20:27:13.390 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-26T20:27:13.451 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-04-26T20:27:13.514 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-26T20:27:13.523 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-04-26T20:27:13.532 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-26T20:27:13.541 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-04-26T20:27:13.591 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-26T20:27:13.599 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-04-26T20:27:13.605 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-26T20:27:13.614 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-04-26T20:27:13.623 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-26T20:27:13.632 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-04-26T20:27:13.714 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-26T20:27:13.723 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-04-26T20:27:13.733 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-26T20:27:13.742 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-04-26T20:27:13.751 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-26T20:27:13.760 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-04-26T20:27:13.811 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-26T20:27:13.874 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-04-26T20:27:13.881 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-26T20:27:13.889 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-04-26T20:27:13.898 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-26T20:27:13.936 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-04-26T20:27:13.944 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-26T20:27:13.954 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-04-26T20:27:13.962 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-26T20:27:13.970 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-04-26T20:27:13.979 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-26T20:27:14.057 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-04-26T20:27:14.066 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-26T20:27:14.075 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-04-26T20:27:14.084 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-26T20:27:14.093 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-04-26T20:27:14.104 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-26T20:27:14.178 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-04-26T20:27:14.230 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-26T20:27:14.237 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-04-26T20:27:14.245 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-26T20:27:14.252 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-04-26T20:27:14.306 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-26T20:27:14.314 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-04-26T20:27:14.321 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-26T20:27:14.329 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-04-26T20:27:14.336 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-26T20:27:14.342 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-04-26T20:27:14.399 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-26T20:27:14.407 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-04-26T20:27:14.414 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-26T20:27:14.421 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-04-26T20:27:14.429 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-26T20:27:14.436 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-04-26T20:27:14.482 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-26T20:27:14.538 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-04-26T20:27:14.546 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-26T20:27:14.554 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-04-26T20:27:14.561 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-26T20:27:14.628 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-04-26T20:27:14.635 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-26T20:27:14.641 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-04-26T20:27:14.647 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-26T20:27:14.657 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-04-26T20:27:14.665 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-26T20:27:14.703 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-04-26T20:27:14.720 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-26T20:27:14.727 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-04-26T20:27:14.734 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-26T20:27:14.741 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-04-26T20:27:14.749 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-26T20:27:14.811 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-04-26T20:27:14.873 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-26T20:27:14.883 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-04-26T20:27:14.894 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-26T20:27:14.903 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-04-26T20:27:14.991 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-26T20:27:15.000 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-04-26T20:27:15.009 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-26T20:27:15.018 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-04-26T20:27:15.027 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-26T20:27:15.036 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-04-26T20:27:15.093 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-26T20:27:15.101 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-04-26T20:27:15.110 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-26T20:27:15.119 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-04-26T20:27:15.128 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-26T20:27:15.138 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-04-26T20:27:15.212 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-26T20:27:15.257 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-04-26T20:27:15.307 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-26T20:27:15.314 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-04-26T20:27:15.322 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-26T20:27:15.392 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-04-26T20:27:15.402 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-26T20:27:15.409 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-04-26T20:27:15.417 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-26T20:27:15.424 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-04-26T20:27:15.431 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-26T20:27:15.497 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-04-26T20:27:15.504 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-26T20:27:15.511 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-04-26T20:27:15.518 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-26T20:27:15.524 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-04-26T20:27:15.531 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-26T20:27:15.609 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-04-26T20:27:15.663 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-26T20:27:15.670 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-04-26T20:27:15.678 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-26T20:27:15.685 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-04-26T20:27:15.727 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-26T20:27:15.733 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-04-26T20:27:15.743 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-26T20:27:15.751 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-04-26T20:27:15.758 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-26T20:27:15.765 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-04-26T20:27:15.830 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-26T20:27:15.838 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-04-26T20:27:15.845 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-26T20:27:15.853 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-04-26T20:27:15.861 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-26T20:27:15.868 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-04-26T20:27:15.940 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-26T20:27:16.005 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-04-26T20:27:16.013 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-26T20:27:16.019 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-04-26T20:27:16.028 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-26T20:27:16.098 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-04-26T20:27:16.108 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-26T20:27:16.117 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-04-26T20:27:16.125 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-26T20:27:16.131 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-04-26T20:27:16.139 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-26T20:27:16.178 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-04-26T20:27:16.204 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-26T20:27:16.214 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-04-26T20:27:16.223 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-26T20:27:16.232 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-04-26T20:27:16.242 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-26T20:27:16.311 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-04-26T20:27:16.363 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-26T20:27:16.372 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-04-26T20:27:16.381 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-26T20:27:16.389 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-04-26T20:27:16.435 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-26T20:27:16.443 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-04-26T20:27:16.450 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-26T20:27:16.458 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-04-26T20:27:16.467 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-26T20:27:16.475 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-04-26T20:27:16.593 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-26T20:27:16.601 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-04-26T20:27:16.609 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-26T20:27:16.618 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-04-26T20:27:16.626 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-26T20:27:16.635 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-04-26T20:27:16.727 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-26T20:27:16.796 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-04-26T20:27:16.805 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-26T20:27:16.814 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-04-26T20:27:16.823 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-26T20:27:16.860 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-04-26T20:27:16.867 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-26T20:27:16.874 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-04-26T20:27:16.881 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-26T20:27:16.888 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-04-26T20:27:16.894 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-26T20:27:16.946 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-04-26T20:27:16.954 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-26T20:27:16.961 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-04-26T20:27:16.968 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-26T20:27:16.977 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-04-26T20:27:16.984 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-26T20:27:17.067 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-04-26T20:27:17.148 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-26T20:27:17.156 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-04-26T20:27:17.163 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-26T20:27:17.170 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-04-26T20:27:17.250 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-26T20:27:17.257 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-04-26T20:27:17.265 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-26T20:27:17.271 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-04-26T20:27:17.276 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-26T20:27:17.282 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-04-26T20:27:17.347 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-26T20:27:17.354 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-04-26T20:27:17.361 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-26T20:27:17.368 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-04-26T20:27:17.375 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-26T20:27:17.381 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-04-26T20:27:17.449 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-26T20:27:17.520 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-04-26T20:27:17.527 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-26T20:27:17.534 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-04-26T20:27:17.540 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-26T20:27:17.586 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-04-26T20:27:17.593 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-26T20:27:17.601 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-04-26T20:27:17.607 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-26T20:27:17.613 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-04-26T20:27:17.620 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-26T20:27:17.674 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-04-26T20:27:17.684 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-26T20:27:17.695 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-04-26T20:27:17.706 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-26T20:27:17.718 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-04-26T20:27:17.727 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-26T20:27:17.773 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-04-26T20:27:17.807 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-26T20:27:17.817 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-04-26T20:27:17.827 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-26T20:27:17.835 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-04-26T20:27:17.871 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-26T20:27:17.880 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-04-26T20:27:17.891 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-26T20:27:17.903 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-04-26T20:27:17.911 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-26T20:27:17.923 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-04-26T20:27:17.979 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-26T20:27:17.989 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-04-26T20:27:17.998 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-26T20:27:18.007 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-04-26T20:27:18.016 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-26T20:27:18.025 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-04-26T20:27:18.065 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-26T20:27:18.123 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-04-26T20:27:18.131 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-26T20:27:18.139 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-04-26T20:27:18.148 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-26T20:27:18.179 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-04-26T20:27:18.188 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-26T20:27:18.196 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-04-26T20:27:18.205 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-26T20:27:18.214 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-04-26T20:27:18.222 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-26T20:27:18.260 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-04-26T20:27:18.269 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-26T20:27:18.278 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-04-26T20:27:18.286 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-26T20:27:18.297 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-04-26T20:27:18.306 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-26T20:27:18.356 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-04-26T20:27:18.389 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-26T20:27:18.399 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-04-26T20:27:18.408 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-26T20:27:18.418 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-04-26T20:27:18.454 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-26T20:27:18.461 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-04-26T20:27:18.470 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-26T20:27:18.476 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-04-26T20:27:18.481 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-26T20:27:18.487 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-04-26T20:27:18.558 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-26T20:27:18.566 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-04-26T20:27:18.574 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-26T20:27:18.581 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-04-26T20:27:18.588 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-26T20:27:18.598 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-04-26T20:27:18.641 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-26T20:27:18.698 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-04-26T20:27:18.705 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-26T20:27:18.713 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-04-26T20:27:18.719 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-26T20:27:18.780 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-04-26T20:27:18.786 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-26T20:27:18.794 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-04-26T20:27:18.801 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-26T20:27:18.807 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-04-26T20:27:18.815 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-26T20:27:18.864 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-04-26T20:27:18.870 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-26T20:27:18.878 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-04-26T20:27:18.885 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-26T20:27:18.892 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-04-26T20:27:18.899 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-26T20:27:18.906 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-04-26T20:27:18.956 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-26T20:27:18.989 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-04-26T20:27:18.998 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-26T20:27:19.005 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-04-26T20:27:19.037 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-26T20:27:19.045 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-04-26T20:27:19.052 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-26T20:27:19.060 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-04-26T20:27:19.067 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-26T20:27:19.078 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-04-26T20:27:19.119 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-26T20:27:19.126 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-04-26T20:27:19.133 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-26T20:27:19.141 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-04-26T20:27:19.148 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-26T20:27:19.158 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-04-26T20:27:19.198 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-26T20:27:19.256 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-04-26T20:27:19.264 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-26T20:27:19.273 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-04-26T20:27:19.282 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-26T20:27:19.321 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-04-26T20:27:19.329 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-26T20:27:19.338 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-04-26T20:27:19.347 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-26T20:27:19.358 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-04-26T20:27:19.368 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-26T20:27:19.413 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-04-26T20:27:19.419 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-26T20:27:19.427 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-04-26T20:27:19.436 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-26T20:27:19.445 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-04-26T20:27:19.454 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-26T20:27:19.506 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-04-26T20:27:19.547 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-26T20:27:19.554 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-04-26T20:27:19.562 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-26T20:27:19.571 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-04-26T20:27:19.612 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-26T20:27:19.627 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-04-26T20:27:19.634 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-26T20:27:19.642 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-04-26T20:27:19.651 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-26T20:27:19.660 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-04-26T20:27:19.704 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-26T20:27:19.725 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-04-26T20:27:19.734 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-26T20:27:19.743 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-04-26T20:27:19.752 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-26T20:27:19.763 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-04-26T20:27:19.805 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-26T20:27:19.864 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-04-26T20:27:19.871 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-26T20:27:19.880 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-04-26T20:27:19.890 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-26T20:27:19.929 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-04-26T20:27:19.938 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-26T20:27:19.947 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-04-26T20:27:19.955 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-26T20:27:19.964 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-04-26T20:27:19.972 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-26T20:27:20.021 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-04-26T20:27:20.030 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-26T20:27:20.037 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-04-26T20:27:20.044 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-26T20:27:20.052 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-04-26T20:27:20.058 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-26T20:27:20.097 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-04-26T20:27:20.146 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-26T20:27:20.154 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-04-26T20:27:20.161 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-26T20:27:20.171 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-04-26T20:27:20.201 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-26T20:27:20.209 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-04-26T20:27:20.216 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-26T20:27:20.224 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-04-26T20:27:20.231 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-26T20:27:20.237 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-04-26T20:27:20.293 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-26T20:27:20.303 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-04-26T20:27:20.310 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-26T20:27:20.316 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-04-26T20:27:20.324 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-26T20:27:20.331 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-04-26T20:27:20.372 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-26T20:27:20.405 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-04-26T20:27:20.438 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-26T20:27:20.445 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-04-26T20:27:20.452 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-26T20:27:20.488 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-04-26T20:27:20.496 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-26T20:27:20.504 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-04-26T20:27:20.511 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-26T20:27:20.519 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-04-26T20:27:20.526 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-26T20:27:20.571 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-04-26T20:27:20.590 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-26T20:27:20.598 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-04-26T20:27:20.608 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-26T20:27:20.614 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-04-26T20:27:20.622 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-26T20:27:20.664 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-04-26T20:27:20.705 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-26T20:27:20.714 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-04-26T20:27:20.721 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-26T20:27:20.729 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-04-26T20:27:20.759 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-26T20:27:20.765 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-04-26T20:27:20.774 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-26T20:27:20.783 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-04-26T20:27:20.792 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-26T20:27:20.801 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-04-26T20:27:20.843 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-26T20:27:20.851 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-04-26T20:27:20.860 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-26T20:27:20.869 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-04-26T20:27:20.878 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-26T20:27:20.893 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-04-26T20:27:20.939 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-26T20:27:20.982 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-04-26T20:27:20.991 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-26T20:27:21.000 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-04-26T20:27:21.009 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-26T20:27:21.047 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-04-26T20:27:21.053 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-26T20:27:21.062 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-04-26T20:27:21.070 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-26T20:27:21.078 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-04-26T20:27:21.088 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-26T20:27:21.143 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-04-26T20:27:21.152 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-26T20:27:21.161 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-04-26T20:27:21.170 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-26T20:27:21.180 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-04-26T20:27:21.191 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-26T20:27:21.230 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-04-26T20:27:21.271 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-26T20:27:21.278 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-04-26T20:27:21.286 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-26T20:27:21.294 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-04-26T20:27:21.373 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-26T20:27:21.382 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-04-26T20:27:21.390 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-26T20:27:21.399 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-04-26T20:27:21.408 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-26T20:27:21.418 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-04-26T20:27:21.470 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-26T20:27:21.485 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-04-26T20:27:21.494 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-26T20:27:21.502 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-04-26T20:27:21.509 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-26T20:27:21.562 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-04-26T20:27:21.572 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-26T20:27:21.616 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-04-26T20:27:21.689 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-26T20:27:21.695 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-04-26T20:27:21.702 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-26T20:27:21.742 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-04-26T20:27:21.750 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-26T20:27:21.757 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-04-26T20:27:21.764 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-26T20:27:21.771 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-04-26T20:27:21.778 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-26T20:27:21.828 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-04-26T20:27:21.835 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-26T20:27:21.842 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-04-26T20:27:21.849 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-26T20:27:21.855 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-04-26T20:27:21.862 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-26T20:27:21.913 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-04-26T20:27:21.938 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-26T20:27:21.946 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-04-26T20:27:21.955 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-26T20:27:21.963 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-04-26T20:27:22.000 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-26T20:27:22.007 INFO:tasks.workunit.client.0.smithi131.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-04-26T20:27:22.014 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-26T20:27:22.022 INFO:tasks.workunit.client.0.smithi131.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-04-26T20:27:22.029 INFO:tasks.workunit.client.0.smithi131.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-26T20:27:22.036 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-04-26T20:27:22.068 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-26T20:27:22.075 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-04-26T20:27:22.082 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-26T20:27:22.089 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-04-26T20:27:22.096 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-26T20:27:22.102 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-04-26T20:27:22.140 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-26T20:27:22.172 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-04-26T20:27:22.178 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-26T20:27:22.185 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-04-26T20:27:22.192 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-26T20:27:22.225 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-04-26T20:27:22.231 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-26T20:27:22.240 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-04-26T20:27:22.249 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-26T20:27:22.259 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-04-26T20:27:22.267 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-26T20:27:22.301 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-04-26T20:27:22.329 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-26T20:27:22.338 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-04-26T20:27:22.348 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:31 smithi131 ceph-mon[27919]: pgmap v905: 129 pgs: 129 active+clean; 866 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-04-26T20:27:22.363 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:33 smithi157 ceph-mon[32703]: pgmap v906: 129 pgs: 129 active+clean; 17 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T20:27:22.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:35 smithi157 ceph-mon[32703]: pgmap v907: 129 pgs: 129 active+clean; 85 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 733 KiB/s wr, 169 op/s 2024-04-26T20:27:22.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:36 smithi157 ceph-mon[32703]: pgmap v908: 129 pgs: 129 active+clean; 133 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 739 KiB/s wr, 172 op/s 2024-04-26T20:27:22.457 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:39 smithi157 ceph-mon[32703]: pgmap v909: 129 pgs: 129 active+clean; 155 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 898 KiB/s wr, 212 op/s 2024-04-26T20:27:22.463 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:41 smithi157 ceph-mon[32703]: pgmap v910: 129 pgs: 129 active+clean; 163 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 920 KiB/s wr, 220 op/s 2024-04-26T20:27:22.469 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:43 smithi157 ceph-mon[32703]: pgmap v911: 129 pgs: 129 active+clean; 187 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 942 KiB/s wr, 237 op/s 2024-04-26T20:27:22.476 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:45 smithi157 ceph-mon[32703]: pgmap v912: 129 pgs: 129 active+clean; 200 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-04-26T20:27:22.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:47 smithi157 ceph-mon[32703]: pgmap v913: 129 pgs: 129 active+clean; 180 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 400 KiB/s wr, 140 op/s 2024-04-26T20:27:22.549 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:49 smithi157 ceph-mon[32703]: pgmap v914: 129 pgs: 129 active+clean; 209 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 497 KiB/s wr, 171 op/s 2024-04-26T20:27:22.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:51 smithi157 ceph-mon[32703]: pgmap v915: 129 pgs: 129 active+clean; 217 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 356 KiB/s wr, 141 op/s 2024-04-26T20:27:22.563 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:53 smithi157 ceph-mon[32703]: pgmap v916: 129 pgs: 129 active+clean; 193 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 349 KiB/s wr, 145 op/s 2024-04-26T20:27:22.569 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:55 smithi157 ceph-mon[32703]: pgmap v917: 129 pgs: 129 active+clean; 146 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 497 KiB/s wr, 182 op/s 2024-04-26T20:27:22.577 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:57 smithi157 ceph-mon[32703]: pgmap v918: 129 pgs: 129 active+clean; 158 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 352 KiB/s wr, 151 op/s 2024-04-26T20:27:22.632 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:26:59 smithi157 ceph-mon[32703]: pgmap v919: 129 pgs: 129 active+clean; 124 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 508 KiB/s wr, 188 op/s 2024-04-26T20:27:22.638 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:00 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:27:22.644 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: pgmap v920: 129 pgs: 129 active+clean; 108 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 419 KiB/s wr, 164 op/s 2024-04-26T20:27:22.649 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.656 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.746 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.807 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:27:22.866 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:22.873 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:27:22.882 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:27:22.915 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:27:22.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:27:22.932 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:27:22.985 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:27:22.994 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:03 smithi157 ceph-mon[32703]: pgmap v921: 129 pgs: 129 active+clean; 108 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 428 KiB/s wr, 169 op/s 2024-04-26T20:27:23.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:05 smithi157 ceph-mon[32703]: pgmap v922: 129 pgs: 129 active+clean; 125 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 575 KiB/s wr, 198 op/s 2024-04-26T20:27:23.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:07 smithi157 ceph-mon[32703]: pgmap v923: 129 pgs: 129 active+clean; 118 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 474 KiB/s wr, 173 op/s 2024-04-26T20:27:23.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:09 smithi157 ceph-mon[32703]: pgmap v924: 129 pgs: 129 active+clean; 94 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 499 KiB/s wr, 209 op/s 2024-04-26T20:27:23.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:11 smithi157 ceph-mon[32703]: pgmap v925: 129 pgs: 129 active+clean; 114 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 345 KiB/s wr, 192 op/s 2024-04-26T20:27:23.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:13 smithi157 ceph-mon[32703]: pgmap v926: 129 pgs: 129 active+clean; 139 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 478 KiB/s wr, 217 op/s 2024-04-26T20:27:23.113 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:15 smithi157 ceph-mon[32703]: pgmap v927: 129 pgs: 129 active+clean; 143 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 471 KiB/s wr, 219 op/s 2024-04-26T20:27:23.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:17 smithi157 ceph-mon[32703]: pgmap v928: 129 pgs: 129 active+clean; 120 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 458 KiB/s wr, 211 op/s 2024-04-26T20:27:23.131 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:19 smithi157 ceph-mon[32703]: pgmap v929: 129 pgs: 129 active+clean; 148 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 437 KiB/s wr, 205 op/s 2024-04-26T20:27:23.140 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:21 smithi157 ceph-mon[32703]: pgmap v930: 129 pgs: 129 active+clean; 148 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 407 KiB/s wr, 167 op/s 2024-04-26T20:27:23.192 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:33 smithi148 ceph-mon[32281]: pgmap v906: 129 pgs: 129 active+clean; 17 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T20:27:23.201 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:35 smithi148 ceph-mon[32281]: pgmap v907: 129 pgs: 129 active+clean; 85 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 733 KiB/s wr, 169 op/s 2024-04-26T20:27:23.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:36 smithi148 ceph-mon[32281]: pgmap v908: 129 pgs: 129 active+clean; 133 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 739 KiB/s wr, 172 op/s 2024-04-26T20:27:23.218 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:39 smithi148 ceph-mon[32281]: pgmap v909: 129 pgs: 129 active+clean; 155 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 898 KiB/s wr, 212 op/s 2024-04-26T20:27:23.229 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:41 smithi148 ceph-mon[32281]: pgmap v910: 129 pgs: 129 active+clean; 163 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 920 KiB/s wr, 220 op/s 2024-04-26T20:27:23.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:43 smithi148 ceph-mon[32281]: pgmap v911: 129 pgs: 129 active+clean; 187 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 942 KiB/s wr, 237 op/s 2024-04-26T20:27:23.279 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:45 smithi148 ceph-mon[32281]: pgmap v912: 129 pgs: 129 active+clean; 200 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-04-26T20:27:23.302 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:47 smithi148 ceph-mon[32281]: pgmap v913: 129 pgs: 129 active+clean; 180 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 400 KiB/s wr, 140 op/s 2024-04-26T20:27:23.311 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:49 smithi148 ceph-mon[32281]: pgmap v914: 129 pgs: 129 active+clean; 209 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 497 KiB/s wr, 171 op/s 2024-04-26T20:27:23.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:51 smithi148 ceph-mon[32281]: pgmap v915: 129 pgs: 129 active+clean; 217 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 356 KiB/s wr, 141 op/s 2024-04-26T20:27:23.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:53 smithi148 ceph-mon[32281]: pgmap v916: 129 pgs: 129 active+clean; 193 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 349 KiB/s wr, 145 op/s 2024-04-26T20:27:23.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:55 smithi148 ceph-mon[32281]: pgmap v917: 129 pgs: 129 active+clean; 146 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 497 KiB/s wr, 182 op/s 2024-04-26T20:27:23.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:57 smithi148 ceph-mon[32281]: pgmap v918: 129 pgs: 129 active+clean; 158 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 352 KiB/s wr, 151 op/s 2024-04-26T20:27:23.421 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:26:59 smithi148 ceph-mon[32281]: pgmap v919: 129 pgs: 129 active+clean; 124 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 508 KiB/s wr, 188 op/s 2024-04-26T20:27:23.429 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:00 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:27:23.437 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: pgmap v920: 129 pgs: 129 active+clean; 108 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 419 KiB/s wr, 164 op/s 2024-04-26T20:27:23.443 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.533 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.539 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.553 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.561 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:27:23.633 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:27:23.640 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:27:23.647 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:27:23.656 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:27:23.665 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:27:23.713 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:27:23.745 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:27:23.752 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:03 smithi148 ceph-mon[32281]: pgmap v921: 129 pgs: 129 active+clean; 108 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 428 KiB/s wr, 169 op/s 2024-04-26T20:27:23.761 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:05 smithi148 ceph-mon[32281]: pgmap v922: 129 pgs: 129 active+clean; 125 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 575 KiB/s wr, 198 op/s 2024-04-26T20:27:23.768 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:07 smithi148 ceph-mon[32281]: pgmap v923: 129 pgs: 129 active+clean; 118 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 474 KiB/s wr, 173 op/s 2024-04-26T20:27:23.808 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:09 smithi148 ceph-mon[32281]: pgmap v924: 129 pgs: 129 active+clean; 94 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 499 KiB/s wr, 209 op/s 2024-04-26T20:27:23.815 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:11 smithi148 ceph-mon[32281]: pgmap v925: 129 pgs: 129 active+clean; 114 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 345 KiB/s wr, 192 op/s 2024-04-26T20:27:23.822 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:13 smithi148 ceph-mon[32281]: pgmap v926: 129 pgs: 129 active+clean; 139 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 478 KiB/s wr, 217 op/s 2024-04-26T20:27:23.830 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:15 smithi148 ceph-mon[32281]: pgmap v927: 129 pgs: 129 active+clean; 143 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 471 KiB/s wr, 219 op/s 2024-04-26T20:27:23.837 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:17 smithi148 ceph-mon[32281]: pgmap v928: 129 pgs: 129 active+clean; 120 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 458 KiB/s wr, 211 op/s 2024-04-26T20:27:23.844 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:19 smithi148 ceph-mon[32281]: pgmap v929: 129 pgs: 129 active+clean; 148 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 437 KiB/s wr, 205 op/s 2024-04-26T20:27:23.878 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:21 smithi148 ceph-mon[32281]: pgmap v930: 129 pgs: 129 active+clean; 148 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 407 KiB/s wr, 167 op/s 2024-04-26T20:27:23.890 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:23 smithi148 ceph-mon[32281]: pgmap v931: 129 pgs: 129 active+clean; 81 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 533 KiB/s wr, 174 op/s 2024-04-26T20:27:23.897 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:23 smithi157 ceph-mon[32703]: pgmap v931: 129 pgs: 129 active+clean; 81 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 533 KiB/s wr, 174 op/s 2024-04-26T20:27:23.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 -- 172.21.15.131:0/759768461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 msgr2=0x7f1610109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 --2- 172.21.15.131:0/759768461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f1610109290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1604009900 tx=0x7f160402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 -- 172.21.15.131:0/759768461 shutdown_connections 2024-04-26T20:27:23.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 --2- 172.21.15.131:0/759768461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f161010a980 0x7f16101112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 --2- 172.21.15.131:0/759768461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610109fc0 0x7f161010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 --2- 172.21.15.131:0/759768461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f1610109290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 -- 172.21.15.131:0/759768461 >> 172.21.15.131:0/759768461 conn(0x7f1610077e00 msgr2=0x7f161007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 -- 172.21.15.131:0/759768461 shutdown_connections 2024-04-26T20:27:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.903+0000 7f1617000640 1 -- 172.21.15.131:0/759768461 wait complete. 2024-04-26T20:27:24.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 Processor -- start 2024-04-26T20:27:24.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 -- start start 2024-04-26T20:27:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f161019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1610109fc0 0x7f161019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f161010a980 0x7f16101a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16101149a0 con 0x7f1610109fc0 2024-04-26T20:27:24.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1610114820 con 0x7f161010a980 2024-04-26T20:27:24.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1617000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1610114b20 con 0x7f1610108e70 2024-04-26T20:27:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1615c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f161019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1615c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f161019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53366/0 (socket says 172.21.15.131:53366) 2024-04-26T20:27:24.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.904+0000 7f1615c00640 1 -- 172.21.15.131:0/807516655 learned_addr learned my addr 172.21.15.131:0/807516655 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:24.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1615200640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1610109fc0 0x7f161019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1616600640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f161010a980 0x7f16101a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1615c00640 1 -- 172.21.15.131:0/807516655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f161010a980 msgr2=0x7f16101a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1615c00640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f161010a980 0x7f16101a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1615c00640 1 -- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1610109fc0 msgr2=0x7f161019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1615c00640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1610109fc0 0x7f161019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1615c00640 1 -- 172.21.15.131:0/807516655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16101a6c90 con 0x7f1610108e70 2024-04-26T20:27:24.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1615c00640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f161019f070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1604005940 tx=0x7f16040042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:24.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f160e000640 1 -- 172.21.15.131:0/807516655 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160403f030 con 0x7f1610108e70 2024-04-26T20:27:24.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f160e000640 1 -- 172.21.15.131:0/807516655 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1604003cb0 con 0x7f1610108e70 2024-04-26T20:27:24.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16101a6f20 con 0x7f1610108e70 2024-04-26T20:27:24.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.905+0000 7f160e000640 1 -- 172.21.15.131:0/807516655 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604035070 con 0x7f1610108e70 2024-04-26T20:27:24.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.906+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16101a7380 con 0x7f1610108e70 2024-04-26T20:27:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f160e000640 1 -- 172.21.15.131:0/807516655 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1604042030 con 0x7f1610108e70 2024-04-26T20:27:24.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 Processor -- start 2024-04-26T20:27:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 -- start start 2024-04-26T20:27:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e0f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e14540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e17040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5573f7d72c70 con 0x5573f7e14bb0 2024-04-26T20:27:24.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5573f7d2b980 con 0x5573f7e0f050 2024-04-26T20:27:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.909+0000 7f161b40f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573f7d6ec30 con 0x5573f7e0fa40 2024-04-26T20:27:24.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1615200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e17040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1615c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e14540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e0f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e0f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:60818/0 (socket says 172.21.15.131:60818) 2024-04-26T20:27:24.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1615c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e14540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53374/0 (socket says 172.21.15.131:53374) 2024-04-26T20:27:24.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 -- 172.21.15.131:0/223528588 learned_addr learned my addr 172.21.15.131:0/223528588 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 -- 172.21.15.131:0/223528588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 msgr2=0x5573f7e14540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:24.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 --2- 172.21.15.131:0/223528588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e14540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 -- 172.21.15.131:0/223528588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 msgr2=0x5573f7e17040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:24.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 --2- 172.21.15.131:0/223528588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e17040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 -- 172.21.15.131:0/223528588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f7d037b0 con 0x5573f7e0f050 2024-04-26T20:27:24.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.910+0000 7f1616600640 1 --2- 172.21.15.131:0/223528588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e0f470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1608009fd0 tx=0x7f16080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88b38e57e5c08534 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f1602a00640 1 -- 172.21.15.131:0/223528588 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608004d20 con 0x5573f7e0f050 2024-04-26T20:27:24.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f1602a00640 1 -- 172.21.15.131:0/223528588 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1608004050 con 0x5573f7e0f050 2024-04-26T20:27:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f1602a00640 1 -- 172.21.15.131:0/223528588 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608005280 con 0x5573f7e0f050 2024-04-26T20:27:24.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f160e000640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15f80a3cc0 0x7f15f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f161b40f740 1 -- 172.21.15.131:0/223528588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 msgr2=0x5573f7e0f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f161b40f740 1 --2- 172.21.15.131:0/223528588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e0f470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1608009fd0 tx=0x7f16080118d0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f160e000640 1 -- 172.21.15.131:0/807516655 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f16040f0d10 con 0x7f1610108e70 2024-04-26T20:27:25.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f161b40f740 1 -- 172.21.15.131:0/223528588 shutdown_connections 2024-04-26T20:27:25.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f161b40f740 1 --2- 172.21.15.131:0/223528588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e17040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f161b40f740 1 --2- 172.21.15.131:0/223528588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f161b40f740 1 --2- 172.21.15.131:0/223528588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e0f470 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f161b40f740 1 -- 172.21.15.131:0/223528588 >> 172.21.15.131:0/223528588 conn(0x5573f7e0a6c0 msgr2=0x5573f7e0aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:25.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.911+0000 7f1615c00640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15f80a3cc0 0x7f15f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.912+0000 7f161b40f740 1 -- 172.21.15.131:0/223528588 shutdown_connections 2024-04-26T20:27:25.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.912+0000 7f161b40f740 1 -- 172.21.15.131:0/223528588 wait complete. 2024-04-26T20:27:25.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.912+0000 7f161b40f740 1 -- start start 2024-04-26T20:27:25.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.912+0000 7f1615c00640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15f80a3cc0 0x7f15f80a6180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5573f7e0fe40 tx=0x7f16040f47c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.912+0000 7f161b40f740 1 Processor -- start 2024-04-26T20:27:25.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f161b40f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e18ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f161b40f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e19420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f161b40f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e20460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f161b40f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5573f7d71f80 con 0x5573f7e14bb0 2024-04-26T20:27:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f161b40f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5573f7d1f260 con 0x5573f7e0f050 2024-04-26T20:27:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f161b40f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5573f7d772a0 con 0x5573f7e0fa40 2024-04-26T20:27:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f161b40f740 10 client.0 _ll_register_callbacks cb 0x5573f7d69960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:27:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f1616600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e18ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f1615200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e19420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f1616600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e20460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f1615200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e19420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53378/0 (socket says 172.21.15.131:53378) 2024-04-26T20:27:25.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.913+0000 7f1615200640 1 -- 172.21.15.131:0/4150380778 learned_addr learned my addr 172.21.15.131:0/4150380778 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:25.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f1615200640 1 -- 172.21.15.131:0/4150380778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 msgr2=0x5573f7e18ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:25.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f1615200640 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e18ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f1615200640 1 -- 172.21.15.131:0/4150380778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 msgr2=0x5573f7e20460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:25.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f1615200640 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e20460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f1615200640 1 -- 172.21.15.131:0/4150380778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f7d011c0 con 0x5573f7e0fa40 2024-04-26T20:27:25.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f1615200640 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e19420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f15fc012700 tx=0x7f15fc012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef52cd0e0ce137ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f15f7400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:27:25.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f15f7400640 1 -- 172.21.15.131:0/4150380778 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00f8c0 con 0x5573f7e0fa40 2024-04-26T20:27:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f15f7400640 1 -- 172.21.15.131:0/4150380778 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f15fc00fa60 con 0x5573f7e0fa40 2024-04-26T20:27:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1608004050 con 0x5573f7e0fa40 2024-04-26T20:27:25.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f15f7400640 1 -- 172.21.15.131:0/4150380778 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15fc00fc60 con 0x5573f7e0fa40 2024-04-26T20:27:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f161b40f740 10 client.26674 fetch_fsmap learned FSMap version 27 2024-04-26T20:27:25.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.914+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573f7a843f0 con 0x5573f7e0fa40 2024-04-26T20:27:25.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f15f7400640 1 -- 172.21.15.131:0/4150380778 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1608004050 con 0x5573f7e0fa40 2024-04-26T20:27:25.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f15f7400640 1 -- 172.21.15.131:0/4150380778 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f15fc01c420 con 0x5573f7e0fa40 2024-04-26T20:27:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f161b40f740 10 client.26674 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:27:25.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f161b40f740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:27:25.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f161b40f740 20 client.26674 populate_metadata read hostname 'smithi131' 2024-04-26T20:27:25.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f161b40f740 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e24ba0 0x5573f7e26fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f161b40f740 4 client.26674 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:27:25.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573f7d80880 con 0x5573f7e24ba0 2024-04-26T20:27:25.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.915+0000 7f1616600640 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e24ba0 0x5573f7e26fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.916+0000 7f1616600640 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e24ba0 0x5573f7e26fc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d7e96ceadf0b4d server_cookie=e9b06ba4d4180ce4 in_seq=0 out_seq=0 2024-04-26T20:27:25.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.916+0000 7f15f7400640 10 client.26674 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f15f7400640 1 -- 172.21.15.131:0/4150380778 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573f7d80880 con 0x5573f7e24ba0 2024-04-26T20:27:25.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f15f7400640 10 client.26674 handle_command_reply: tid=0 2024-04-26T20:27:25.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 client.26674 shutdown 2024-04-26T20:27:25.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 msgr2=0x5573f7e19420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e19420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f15fc012700 tx=0x7f15fc012bc0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 shutdown_connections 2024-04-26T20:27:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e24ba0 0x5573f7e26fc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e14bb0 0x5573f7e20460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0fa40 0x5573f7e19420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 --2- 172.21.15.131:0/4150380778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0f050 0x5573f7e18ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f15f7400640 0 client.26674 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 >> 172.21.15.131:0/4150380778 conn(0x5573f7e0a6c0 msgr2=0x5573f7e097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:26.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 shutdown_connections 2024-04-26T20:27:26.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 1 -- 172.21.15.131:0/4150380778 wait complete. 2024-04-26T20:27:26.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.917+0000 7f161b40f740 20 client.26674 trim_cache size 0 max 16384 2024-04-26T20:27:26.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.934+0000 7f161b40f740 1 Processor -- start 2024-04-26T20:27:26.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.934+0000 7f161b40f740 1 -- start start 2024-04-26T20:27:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f161b40f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0a6c0 0x5573f7e05640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f161b40f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0f050 0x5573f7e20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f161b40f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e0d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f161b40f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16101b42d0 con 0x5573f7e0f050 2024-04-26T20:27:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f161b40f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16101b1ba0 con 0x5573f7e0fa40 2024-04-26T20:27:26.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f161b40f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16101af9f0 con 0x5573f7e0a6c0 2024-04-26T20:27:26.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f1615200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0f050 0x5573f7e20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f1615200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0f050 0x5573f7e20420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40298/0 (socket says 172.21.15.131:40298) 2024-04-26T20:27:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f1615200640 1 -- 172.21.15.131:0/4086827865 learned_addr learned my addr 172.21.15.131:0/4086827865 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.935+0000 7f1616600640 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0a6c0 0x5573f7e05640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1616600640 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e0d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1615200640 1 -- 172.21.15.131:0/4086827865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0a6c0 msgr2=0x5573f7e05640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:26.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1615200640 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0a6c0 0x5573f7e05640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1615200640 1 -- 172.21.15.131:0/4086827865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 msgr2=0x5573f7e0d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:26.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1615200640 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e0d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1615200640 1 -- 172.21.15.131:0/4086827865 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f7b4e7d0 con 0x5573f7e0f050 2024-04-26T20:27:26.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1616600640 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e0d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:26.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1615200640 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0f050 0x5573f7e20420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0097e0 tx=0x7f15fc00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02ec00239c83eab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1602000640 1 -- 172.21.15.131:0/4086827865 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc01cc90 con 0x5573f7e0f050 2024-04-26T20:27:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1602000640 1 -- 172.21.15.131:0/4086827865 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15fc01c420 con 0x5573f7e0f050 2024-04-26T20:27:26.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.936+0000 7f1602000640 1 -- 172.21.15.131:0/4086827865 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0035f0 con 0x5573f7e0f050 2024-04-26T20:27:26.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 -- 172.21.15.131:0/4086827865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0f050 msgr2=0x5573f7e20420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0f050 0x5573f7e20420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0097e0 tx=0x7f15fc00ff90 comp rx=0 tx=0).stop 2024-04-26T20:27:26.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 -- 172.21.15.131:0/4086827865 shutdown_connections 2024-04-26T20:27:26.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e0d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0f050 0x5573f7e20420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 --2- 172.21.15.131:0/4086827865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0a6c0 0x5573f7e05640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 -- 172.21.15.131:0/4086827865 >> 172.21.15.131:0/4086827865 conn(0x5573f7e180f0 msgr2=0x5573f7e097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.937+0000 7f161b40f740 1 -- 172.21.15.131:0/4086827865 shutdown_connections 2024-04-26T20:27:26.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.938+0000 7f161b40f740 1 -- 172.21.15.131:0/4086827865 wait complete. 2024-04-26T20:27:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.938+0000 7f161b40f740 1 -- start start 2024-04-26T20:27:26.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.938+0000 7f161b40f740 1 Processor -- start 2024-04-26T20:27:26.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f161b40f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0a6c0 0x5573f7e1de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f161b40f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0f050 0x5573f7e1e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f161b40f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e1f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f161b40f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16101b0060 con 0x5573f7e0a6c0 2024-04-26T20:27:26.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f161b40f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16101b14d0 con 0x5573f7e0fa40 2024-04-26T20:27:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f161b40f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16101b0730 con 0x5573f7e0f050 2024-04-26T20:27:26.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f161b40f740 10 client.0 _ll_register_callbacks cb 0x7f15fc009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:27:26.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0f050 0x5573f7e1e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0f050 0x5573f7e1e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53404/0 (socket says 172.21.15.131:53404) 2024-04-26T20:27:26.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 -- 172.21.15.131:0/3434426800 learned_addr learned my addr 172.21.15.131:0/3434426800 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:26.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1616600640 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0a6c0 0x5573f7e1de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1616600640 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e1f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 -- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 msgr2=0x5573f7e1f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:26.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e1f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 -- 172.21.15.131:0/3434426800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0a6c0 msgr2=0x5573f7e1de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:26.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0a6c0 0x5573f7e1de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.939+0000 7f1615200640 1 -- 172.21.15.131:0/3434426800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f7d192a0 con 0x5573f7e0f050 2024-04-26T20:27:27.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.940+0000 7f1615200640 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0f050 0x5573f7e1e370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15fc012c00 tx=0x7f15fc00fe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7708d9176fa789a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:27.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.940+0000 7f15f7400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:27:27.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.940+0000 7f15f7400640 1 -- 172.21.15.131:0/3434426800 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc039060 con 0x5573f7e0f050 2024-04-26T20:27:27.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.940+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc01c420 con 0x5573f7e0f050 2024-04-26T20:27:27.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.940+0000 7f15f7400640 1 -- 172.21.15.131:0/3434426800 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f15fc018070 con 0x5573f7e0f050 2024-04-26T20:27:27.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f15f7400640 1 -- 172.21.15.131:0/3434426800 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15fc0050b0 con 0x5573f7e0f050 2024-04-26T20:27:27.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 10 client.26680 fetch_fsmap learned FSMap version 27 2024-04-26T20:27:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573f7e8b200 con 0x5573f7e0f050 2024-04-26T20:27:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f15f7400640 1 -- 172.21.15.131:0/3434426800 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15fc01c420 con 0x5573f7e0f050 2024-04-26T20:27:27.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f15f7400640 1 -- 172.21.15.131:0/3434426800 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f15fc02a430 con 0x5573f7e0f050 2024-04-26T20:27:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 10 client.26680 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:27:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:27:27.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 20 client.26680 populate_metadata read hostname 'smithi131' 2024-04-26T20:27:27.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e14bb0 0x5573f7e8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 4 client.26680 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:27:27.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573f7d7a6f0 con 0x5573f7e14bb0 2024-04-26T20:27:27.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.941+0000 7f1616600640 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e14bb0 0x5573f7e8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f1616600640 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e14bb0 0x5573f7e8da00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78c090daf1cbb469 server_cookie=c897b0119af537ce in_seq=0 out_seq=0 2024-04-26T20:27:27.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f15f7400640 10 client.26680 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:27.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f15f7400640 1 -- 172.21.15.131:0/3434426800 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573f7d7a6f0 con 0x5573f7e14bb0 2024-04-26T20:27:27.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f15f7400640 10 client.26680 handle_command_reply: tid=0 2024-04-26T20:27:27.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 client.26680 shutdown 2024-04-26T20:27:27.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0f050 msgr2=0x5573f7e1e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0f050 0x5573f7e1e370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15fc012c00 tx=0x7f15fc00fe70 comp rx=0 tx=0).stop 2024-04-26T20:27:27.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 shutdown_connections 2024-04-26T20:27:27.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5573f7e14bb0 0x5573f7e8da00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5573f7e0fa40 0x5573f7e1f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573f7e0f050 0x5573f7e1e370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f15f7400640 0 client.26680 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:27.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 --2- 172.21.15.131:0/3434426800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573f7e0a6c0 0x5573f7e1de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 >> 172.21.15.131:0/3434426800 conn(0x5573f7e180f0 msgr2=0x5573f7e104e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 shutdown_connections 2024-04-26T20:27:27.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.942+0000 7f161b40f740 1 -- 172.21.15.131:0/3434426800 wait complete. 2024-04-26T20:27:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f161b40f740 20 client.26680 trim_cache size 0 max 16384 2024-04-26T20:27:27.636 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:27:27.646 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:25 smithi148 ceph-mon[32281]: pgmap v932: 129 pgs: 129 active+clean; 49 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 411 KiB/s wr, 157 op/s 2024-04-26T20:27:27.653 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:25 smithi157 ceph-mon[32703]: pgmap v932: 129 pgs: 129 active+clean; 49 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 411 KiB/s wr, 157 op/s 2024-04-26T20:27:27.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15f80a3cc0 msgr2=0x7f15f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15f80a3cc0 0x7f15f80a6180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5573f7e0fe40 tx=0x7f16040f47c0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 msgr2=0x7f161019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f161019f070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1604005940 tx=0x7f16040042f0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 shutdown_connections 2024-04-26T20:27:27.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f161010a980 0x7f16101a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1610109fc0 0x7f161019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f15f80a3cc0 0x7f15f80a6180 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 --2- 172.21.15.131:0/807516655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1610108e70 0x7f161019f070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.943+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 >> 172.21.15.131:0/807516655 conn(0x7f1610077e00 msgr2=0x7f161010d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.944+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 shutdown_connections 2024-04-26T20:27:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:26:32.944+0000 7f1617000640 1 -- 172.21.15.131:0/807516655 wait complete. 2024-04-26T20:27:27.842 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:33 smithi131 ceph-mon[27919]: pgmap v906: 129 pgs: 129 active+clean; 17 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T20:27:27.902 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-26T20:27:27.944 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-04-26T20:27:27.951 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-26T20:27:27.958 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-04-26T20:27:27.965 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-26T20:27:28.028 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-04-26T20:27:28.036 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-26T20:27:28.044 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-04-26T20:27:28.050 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-26T20:27:28.057 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-04-26T20:27:28.064 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-26T20:27:28.113 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-04-26T20:27:28.121 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-26T20:27:28.130 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-04-26T20:27:28.139 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-26T20:27:28.147 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-04-26T20:27:28.155 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-26T20:27:28.214 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-04-26T20:27:28.260 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-26T20:27:28.268 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-04-26T20:27:28.273 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-26T20:27:28.280 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-04-26T20:27:28.323 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-26T20:27:28.330 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-04-26T20:27:28.338 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-26T20:27:28.344 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-04-26T20:27:28.351 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-26T20:27:28.357 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-04-26T20:27:28.402 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-26T20:27:28.412 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-04-26T20:27:28.421 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-26T20:27:28.430 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-04-26T20:27:28.439 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-26T20:27:28.447 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-04-26T20:27:28.499 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-26T20:27:28.527 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-04-26T20:27:28.536 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-26T20:27:28.546 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-04-26T20:27:28.556 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-26T20:27:28.598 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-04-26T20:27:28.605 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-26T20:27:28.612 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-04-26T20:27:28.620 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-26T20:27:28.626 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-04-26T20:27:28.633 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-26T20:27:28.666 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-04-26T20:27:28.675 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-26T20:27:28.684 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-04-26T20:27:28.691 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-26T20:27:28.698 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-04-26T20:27:28.705 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-26T20:27:28.782 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-04-26T20:27:28.852 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-26T20:27:28.860 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-04-26T20:27:28.869 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-26T20:27:28.877 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-04-26T20:27:28.973 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-26T20:27:28.983 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-04-26T20:27:28.992 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-26T20:27:29.001 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-04-26T20:27:29.008 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-26T20:27:29.016 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-04-26T20:27:29.084 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-26T20:27:29.097 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-04-26T20:27:29.106 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-26T20:27:29.114 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-04-26T20:27:29.121 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-26T20:27:29.129 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-04-26T20:27:29.193 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-26T20:27:29.252 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-04-26T20:27:29.260 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-26T20:27:29.266 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-04-26T20:27:29.272 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-26T20:27:29.373 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-04-26T20:27:29.381 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-26T20:27:29.389 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-04-26T20:27:29.396 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-26T20:27:29.402 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-04-26T20:27:29.410 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-26T20:27:29.480 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-04-26T20:27:29.487 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-26T20:27:29.496 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-04-26T20:27:29.506 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-26T20:27:29.514 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-04-26T20:27:29.522 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-26T20:27:29.591 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-04-26T20:27:29.645 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-26T20:27:29.653 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-04-26T20:27:29.660 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-26T20:27:29.667 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-04-26T20:27:29.715 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-26T20:27:29.722 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-04-26T20:27:29.730 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-26T20:27:29.737 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-04-26T20:27:29.745 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-26T20:27:29.752 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-04-26T20:27:29.817 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-26T20:27:29.825 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-04-26T20:27:29.833 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-26T20:27:29.839 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-04-26T20:27:29.846 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-26T20:27:29.853 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-04-26T20:27:29.916 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-26T20:27:29.943 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-04-26T20:27:29.953 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-26T20:27:29.962 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-04-26T20:27:29.972 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-26T20:27:30.015 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-04-26T20:27:30.024 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-26T20:27:30.031 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-04-26T20:27:30.040 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-26T20:27:30.049 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-04-26T20:27:30.057 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-26T20:27:30.092 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-04-26T20:27:30.115 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-26T20:27:30.123 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-04-26T20:27:30.133 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-26T20:27:30.143 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-04-26T20:27:30.152 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-26T20:27:30.203 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-04-26T20:27:30.243 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-26T20:27:30.252 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-04-26T20:27:30.261 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-26T20:27:30.271 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-04-26T20:27:30.315 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-26T20:27:30.321 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-04-26T20:27:30.327 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-26T20:27:30.333 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-04-26T20:27:30.341 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-26T20:27:30.349 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-04-26T20:27:30.393 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-26T20:27:30.402 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-04-26T20:27:30.412 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-26T20:27:30.421 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-04-26T20:27:30.429 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-26T20:27:30.438 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-04-26T20:27:30.504 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-26T20:27:30.552 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-04-26T20:27:30.560 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-26T20:27:30.570 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-04-26T20:27:30.579 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-26T20:27:30.623 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-04-26T20:27:30.629 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-26T20:27:30.638 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-04-26T20:27:30.647 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-26T20:27:30.654 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-04-26T20:27:30.661 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-26T20:27:30.723 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-04-26T20:27:30.730 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-26T20:27:30.737 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-04-26T20:27:30.744 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-26T20:27:30.751 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-04-26T20:27:30.758 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-26T20:27:30.808 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-04-26T20:27:30.855 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-26T20:27:30.862 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-04-26T20:27:30.869 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-26T20:27:30.877 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-04-26T20:27:30.922 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-26T20:27:30.929 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-04-26T20:27:30.937 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-26T20:27:30.943 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-04-26T20:27:30.950 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-26T20:27:30.958 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-04-26T20:27:31.001 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-26T20:27:31.009 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-04-26T20:27:31.017 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-26T20:27:31.028 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-04-26T20:27:31.037 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-26T20:27:31.046 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-04-26T20:27:31.108 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-26T20:27:31.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:35 smithi131 ceph-mon[27919]: pgmap v907: 129 pgs: 129 active+clean; 85 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 733 KiB/s wr, 169 op/s 2024-04-26T20:27:31.152 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-04-26T20:27:31.160 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-26T20:27:31.167 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-04-26T20:27:31.248 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-26T20:27:31.255 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-04-26T20:27:31.261 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-26T20:27:31.270 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-04-26T20:27:31.278 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-26T20:27:31.286 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-04-26T20:27:31.346 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-26T20:27:31.354 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-04-26T20:27:31.361 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-26T20:27:31.368 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-04-26T20:27:31.375 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-26T20:27:31.381 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-04-26T20:27:31.430 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-26T20:27:31.490 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-04-26T20:27:31.499 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-26T20:27:31.509 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-04-26T20:27:31.518 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-26T20:27:31.575 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-04-26T20:27:31.582 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-26T20:27:31.591 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-04-26T20:27:31.600 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-26T20:27:31.609 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-04-26T20:27:31.618 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-26T20:27:31.674 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-04-26T20:27:31.684 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-26T20:27:31.692 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-04-26T20:27:31.707 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-26T20:27:31.716 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-04-26T20:27:31.726 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-26T20:27:31.811 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-04-26T20:27:31.844 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-26T20:27:31.854 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-04-26T20:27:31.865 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-26T20:27:31.873 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-04-26T20:27:31.931 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-26T20:27:31.939 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-04-26T20:27:31.948 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-26T20:27:31.957 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-04-26T20:27:31.965 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-26T20:27:31.974 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-04-26T20:27:32.030 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-26T20:27:32.041 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-04-26T20:27:32.049 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-26T20:27:32.058 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-04-26T20:27:32.066 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-26T20:27:32.075 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-04-26T20:27:32.150 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-26T20:27:32.181 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-04-26T20:27:32.187 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-26T20:27:32.194 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-04-26T20:27:32.201 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-26T20:27:32.266 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-04-26T20:27:32.274 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-26T20:27:32.280 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-04-26T20:27:32.286 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-26T20:27:32.294 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-04-26T20:27:32.301 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-26T20:27:32.332 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-04-26T20:27:32.361 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-26T20:27:32.369 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-04-26T20:27:32.376 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-26T20:27:32.383 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-04-26T20:27:32.391 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-26T20:27:32.444 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-04-26T20:27:32.468 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-26T20:27:32.474 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-04-26T20:27:32.481 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-26T20:27:32.489 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-04-26T20:27:32.548 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-26T20:27:32.555 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-04-26T20:27:32.563 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-26T20:27:32.569 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-04-26T20:27:32.578 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-26T20:27:32.588 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-04-26T20:27:32.640 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-26T20:27:32.650 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-04-26T20:27:32.658 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-26T20:27:32.667 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-04-26T20:27:32.677 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-26T20:27:32.686 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-04-26T20:27:32.737 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-26T20:27:32.762 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-04-26T20:27:32.771 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-26T20:27:32.780 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:36 smithi131 ceph-mon[27919]: pgmap v908: 129 pgs: 129 active+clean; 133 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 739 KiB/s wr, 172 op/s 2024-04-26T20:27:32.790 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T20:27:32.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:32.857 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T20:27:32.867 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:27 smithi148 ceph-mon[32281]: pgmap v933: 129 pgs: 129 active+clean; 21 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 390 KiB/s wr, 143 op/s 2024-04-26T20:27:32.876 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:29 smithi148 ceph-mon[32281]: pgmap v934: 129 pgs: 129 active+clean; 5.0 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 236 KiB/s wr, 108 op/s 2024-04-26T20:27:32.886 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:31 smithi148 ceph-mon[32281]: pgmap v935: 129 pgs: 129 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 188 KiB/s wr, 84 op/s 2024-04-26T20:27:32.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:27 smithi157 ceph-mon[32703]: pgmap v933: 129 pgs: 129 active+clean; 21 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 390 KiB/s wr, 143 op/s 2024-04-26T20:27:32.924 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:29 smithi157 ceph-mon[32703]: pgmap v934: 129 pgs: 129 active+clean; 5.0 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 236 KiB/s wr, 108 op/s 2024-04-26T20:27:32.956 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:31 smithi157 ceph-mon[32703]: pgmap v935: 129 pgs: 129 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 188 KiB/s wr, 84 op/s 2024-04-26T20:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.337+0000 7f0a82e00640 1 -- 172.21.15.131:0/2274780389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c100bc0 msgr2=0x7f0a7c100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.337+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2274780389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c100fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0074e0 tx=0x7f0a6c02b060 comp rx=0 tx=0).stop 2024-04-26T20:27:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.337+0000 7f0a82e00640 1 -- 172.21.15.131:0/2274780389 shutdown_connections 2024-04-26T20:27:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.337+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2274780389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c102770 0x7f0a7c109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.337+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2274780389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101db0 0x7f0a7c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.337+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2274780389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c100fe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.337+0000 7f0a82e00640 1 -- 172.21.15.131:0/2274780389 >> 172.21.15.131:0/2274780389 conn(0x7f0a7c0fc150 msgr2=0x7f0a7c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.338+0000 7f0a82e00640 1 -- 172.21.15.131:0/2274780389 shutdown_connections 2024-04-26T20:27:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.339+0000 7f0a82e00640 1 -- 172.21.15.131:0/2274780389 wait complete. 2024-04-26T20:27:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.340+0000 7f0a82e00640 1 Processor -- start 2024-04-26T20:27:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.341+0000 7f0a82e00640 1 -- start start 2024-04-26T20:27:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a82e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a82e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101db0 0x7f0a7c106e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a82e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c102770 0x7f0a7c1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a82e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c10e4e0 con 0x7f0a7c101db0 2024-04-26T20:27:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a82e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c10e360 con 0x7f0a7c100bc0 2024-04-26T20:27:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a82e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c10e660 con 0x7f0a7c102770 2024-04-26T20:27:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55558/0 (socket says 172.21.15.131:55558) 2024-04-26T20:27:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.342+0000 7f0a7be00640 1 -- 172.21.15.131:0/2619191512 learned_addr learned my addr 172.21.15.131:0/2619191512 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a80e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c102770 0x7f0a7c1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a7be00640 1 -- 172.21.15.131:0/2619191512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c102770 msgr2=0x7f0a7c1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c102770 0x7f0a7c1034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a7be00640 1 -- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101db0 msgr2=0x7f0a7c106e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101db0 0x7f0a7c106e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101db0 0x7f0a7c106e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a7be00640 1 -- 172.21.15.131:0/2619191512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c103d70 con 0x7f0a7c100bc0 2024-04-26T20:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.343+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101db0 0x7f0a7c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.344+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c106940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c005ab0 tx=0x7f0a6c004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.344+0000 7f0a78c00640 1 -- 172.21.15.131:0/2619191512 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c005d50 con 0x7f0a7c100bc0 2024-04-26T20:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.345+0000 7f0a78c00640 1 -- 172.21.15.131:0/2619191512 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a6c041070 con 0x7f0a7c100bc0 2024-04-26T20:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.345+0000 7f0a78c00640 1 -- 172.21.15.131:0/2619191512 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00a4d0 con 0x7f0a7c100bc0 2024-04-26T20:27:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.345+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c1a7150 con 0x7f0a7c100bc0 2024-04-26T20:27:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.345+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c1a7500 con 0x7f0a7c100bc0 2024-04-26T20:27:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.353+0000 7f0a78c00640 1 -- 172.21.15.131:0/2619191512 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0a6c0053e0 con 0x7f0a7c100bc0 2024-04-26T20:27:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.355+0000 7f0a85bc8740 1 Processor -- start 2024-04-26T20:27:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.356+0000 7f0a85bc8740 1 -- start start 2024-04-26T20:27:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.356+0000 7f0a78c00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0a600a3eb0 0x7f0a600a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.356+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a5abd0 0x557607a5aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.357+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5b5c0 0x557607a600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.357+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a60730 0x557607a62bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.357+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5576079bf380 con 0x557607a60730 2024-04-26T20:27:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.357+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55760795bc30 con 0x557607a5b5c0 2024-04-26T20:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.357+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5576079bb340 con 0x557607a5abd0 2024-04-26T20:27:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a78c00640 1 -- 172.21.15.131:0/2619191512 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0a6c0f0540 con 0x7f0a7c100bc0 2024-04-26T20:27:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a5abd0 0x557607a5aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5b5c0 0x557607a600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a5abd0 0x557607a5aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34988/0 (socket says 172.21.15.131:34988) 2024-04-26T20:27:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7be00640 1 -- 172.21.15.131:0/1854139874 learned_addr learned my addr 172.21.15.131:0/1854139874 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a80e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0a600a3eb0 0x7f0a600a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7be00640 1 -- 172.21.15.131:0/1854139874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5b5c0 msgr2=0x557607a600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7be00640 1 --2- 172.21.15.131:0/1854139874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5b5c0 0x557607a600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7be00640 1 -- 172.21.15.131:0/1854139874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a60730 msgr2=0x557607a62bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.358+0000 7f0a7be00640 1 --2- 172.21.15.131:0/1854139874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a60730 0x557607a62bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.359+0000 7f0a7be00640 1 -- 172.21.15.131:0/1854139874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55760797d940 con 0x557607a5abd0 2024-04-26T20:27:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.359+0000 7f0a7be00640 1 --2- 172.21.15.131:0/1854139874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a5abd0 0x557607a5aff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c02b620 tx=0x7f0a6c00ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997a26415ee4fc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.359+0000 7f0a80e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0a600a3eb0 0x7f0a600a6370 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005fd0 tx=0x7f0a74009af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.360+0000 7f0a71600640 1 -- 172.21.15.131:0/1854139874 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0bc380 con 0x557607a5abd0 2024-04-26T20:27:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.360+0000 7f0a71600640 1 -- 172.21.15.131:0/1854139874 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a6c0c0730 con 0x557607a5abd0 2024-04-26T20:27:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.360+0000 7f0a71600640 1 -- 172.21.15.131:0/1854139874 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c07d500 con 0x557607a5abd0 2024-04-26T20:27:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.361+0000 7f0a85bc8740 1 -- 172.21.15.131:0/1854139874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a5abd0 msgr2=0x557607a5aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.361+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/1854139874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a5abd0 0x557607a5aff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c02b620 tx=0x7f0a6c00ac00 comp rx=0 tx=0).stop 2024-04-26T20:27:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.362+0000 7f0a85bc8740 1 -- 172.21.15.131:0/1854139874 shutdown_connections 2024-04-26T20:27:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.362+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/1854139874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a60730 0x557607a62bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.362+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/1854139874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5b5c0 0x557607a600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.362+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/1854139874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a5abd0 0x557607a5aff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.362+0000 7f0a85bc8740 1 -- 172.21.15.131:0/1854139874 >> 172.21.15.131:0/1854139874 conn(0x557607a56240 msgr2=0x557607a56640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.362+0000 7f0a85bc8740 1 -- 172.21.15.131:0/1854139874 shutdown_connections 2024-04-26T20:27:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.362+0000 7f0a85bc8740 1 -- 172.21.15.131:0/1854139874 wait complete. 2024-04-26T20:27:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.363+0000 7f0a85bc8740 1 -- start start 2024-04-26T20:27:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.363+0000 7f0a85bc8740 1 Processor -- start 2024-04-26T20:27:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a64a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a64fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a6c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557607a634a0 con 0x557607a5b5c0 2024-04-26T20:27:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557607946c60 con 0x557607a5abd0 2024-04-26T20:27:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557607a63620 con 0x557607a60730 2024-04-26T20:27:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a85bc8740 10 client.0 _ll_register_callbacks cb 0x557607972e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:27:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a64a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a64a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:55576/0 (socket says 172.21.15.131:55576) 2024-04-26T20:27:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.364+0000 7f0a7be00640 1 -- 172.21.15.131:0/2813104380 learned_addr learned my addr 172.21.15.131:0/2813104380 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a80e00640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a6c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7be00640 1 -- 172.21.15.131:0/2813104380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 msgr2=0x557607a6c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a64fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a6c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7be00640 1 -- 172.21.15.131:0/2813104380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 msgr2=0x557607a64fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a64fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7be00640 1 -- 172.21.15.131:0/2813104380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557607955510 con 0x557607a5abd0 2024-04-26T20:27:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a80e00640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a64fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a64a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c004f10 tx=0x7f0a6c0c5070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47417ab71ae4ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.365+0000 7f0a66000640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:27:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.366+0000 7f0a66000640 1 -- 172.21.15.131:0/2813104380 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0d4020 con 0x557607a5abd0 2024-04-26T20:27:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.366+0000 7f0a66000640 1 -- 172.21.15.131:0/2813104380 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0a6c0bc5e0 con 0x557607a5abd0 2024-04-26T20:27:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.366+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a6c0c0730 con 0x557607a5abd0 2024-04-26T20:27:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.366+0000 7f0a66000640 1 -- 172.21.15.131:0/2813104380 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a6c0da050 con 0x557607a5abd0 2024-04-26T20:27:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.366+0000 7f0a85bc8740 10 client.26714 fetch_fsmap learned FSMap version 27 2024-04-26T20:27:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.366+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576076d1210 con 0x557607a5abd0 2024-04-26T20:27:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.367+0000 7f0a66000640 1 -- 172.21.15.131:0/2813104380 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a6c0c0730 con 0x557607a5abd0 2024-04-26T20:27:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.367+0000 7f0a66000640 1 -- 172.21.15.131:0/2813104380 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a6c0c4b60 con 0x557607a5abd0 2024-04-26T20:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.367+0000 7f0a85bc8740 10 client.26714 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:27:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.367+0000 7f0a85bc8740 10 client.26714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:27:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.367+0000 7f0a85bc8740 20 client.26714 populate_metadata read hostname 'smithi131' 2024-04-26T20:27:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.369+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607a702a0 0x557607a72730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.369+0000 7f0a85bc8740 4 client.26714 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:27:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.369+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576079ccf90 con 0x557607a702a0 2024-04-26T20:27:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.370+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607a702a0 0x557607a72730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.371+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607a702a0 0x557607a72730 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d729b0498745fef0 server_cookie=da8a7cdb89efda9b in_seq=0 out_seq=0 2024-04-26T20:27:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.371+0000 7f0a66000640 10 client.26714 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.372+0000 7f0a66000640 1 -- 172.21.15.131:0/2813104380 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576079ccf90 con 0x557607a702a0 2024-04-26T20:27:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.372+0000 7f0a66000640 10 client.26714 handle_command_reply: tid=0 2024-04-26T20:27:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.372+0000 7f0a85bc8740 1 client.26714 shutdown 2024-04-26T20:27:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.373+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 msgr2=0x557607a64a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.373+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a64a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c004f10 tx=0x7f0a6c0c5070 comp rx=0 tx=0).stop 2024-04-26T20:27:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 shutdown_connections 2024-04-26T20:27:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607a702a0 0x557607a72730 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a64fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2813104380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a64a90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 >> 172.21.15.131:0/2813104380 conn(0x557607a56240 msgr2=0x557607a55440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a66000640 0 client.26714 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 shutdown_connections 2024-04-26T20:27:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2813104380 wait complete. 2024-04-26T20:27:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.374+0000 7f0a85bc8740 20 client.26714 trim_cache size 0 max 16384 2024-04-26T20:27:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.405+0000 7f0a85bc8740 1 Processor -- start 2024-04-26T20:27:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.405+0000 7f0a85bc8740 1 -- start start 2024-04-26T20:27:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.406+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a59900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.406+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.406+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.406+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b44f0 con 0x557607a5b5c0 2024-04-26T20:27:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.406+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b1dc0 con 0x557607a5abd0 2024-04-26T20:27:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.406+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1afbd0 con 0x557607a60730 2024-04-26T20:27:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.406+0000 7f0a80e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.407+0000 7f0a7b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.407+0000 7f0a7b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a6bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60546/0 (socket says 172.21.15.131:60546) 2024-04-26T20:27:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.407+0000 7f0a7b400640 1 -- 172.21.15.131:0/2328042696 learned_addr learned my addr 172.21.15.131:0/2328042696 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.407+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a59900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.408+0000 7f0a7b400640 1 -- 172.21.15.131:0/2328042696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 msgr2=0x557607a646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.408+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.408+0000 7f0a7b400640 1 -- 172.21.15.131:0/2328042696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 msgr2=0x557607a59900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.408+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a59900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.408+0000 7f0a7b400640 1 -- 172.21.15.131:0/2328042696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55760779d620 con 0x557607a5b5c0 2024-04-26T20:27:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.408+0000 7f0a7be00640 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a59900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.408+0000 7f0a7b400640 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a6bfa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x557607a5b9c0 tx=0x7f0a68014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa83a977c9f4813 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.409+0000 7f0a70c00640 1 -- 172.21.15.131:0/2328042696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68024070 con 0x557607a5b5c0 2024-04-26T20:27:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.409+0000 7f0a70c00640 1 -- 172.21.15.131:0/2328042696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a68009b30 con 0x557607a5b5c0 2024-04-26T20:27:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.409+0000 7f0a70c00640 1 -- 172.21.15.131:0/2328042696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68003e20 con 0x557607a5b5c0 2024-04-26T20:27:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.411+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2328042696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 msgr2=0x557607a6bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.411+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a6bfa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x557607a5b9c0 tx=0x7f0a68014840 comp rx=0 tx=0).stop 2024-04-26T20:27:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.411+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2328042696 shutdown_connections 2024-04-26T20:27:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.411+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a60730 0x557607a646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.411+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5b5c0 0x557607a6bfa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.411+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/2328042696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a5abd0 0x557607a59900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.411+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2328042696 >> 172.21.15.131:0/2328042696 conn(0x557607a56240 msgr2=0x557607a591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.412+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2328042696 shutdown_connections 2024-04-26T20:27:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.412+0000 7f0a85bc8740 1 -- 172.21.15.131:0/2328042696 wait complete. 2024-04-26T20:27:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.412+0000 7f0a85bc8740 1 -- start start 2024-04-26T20:27:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.413+0000 7f0a85bc8740 1 Processor -- start 2024-04-26T20:27:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.414+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5abd0 0x557607a69070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.414+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a60730 0x557607a695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.414+0000 7f0a85bc8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a6a5c0 0x557607a6aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.414+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b0280 con 0x557607a5abd0 2024-04-26T20:27:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.414+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b16f0 con 0x557607a60730 2024-04-26T20:27:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.414+0000 7f0a85bc8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b0950 con 0x557607a6a5c0 2024-04-26T20:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.415+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5abd0 0x557607a69070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.415+0000 7f0a7be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5abd0 0x557607a69070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60550/0 (socket says 172.21.15.131:60550) 2024-04-26T20:27:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.415+0000 7f0a7be00640 1 -- 172.21.15.131:0/3900687763 learned_addr learned my addr 172.21.15.131:0/3900687763 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.415+0000 7f0a85bc8740 10 client.0 _ll_register_callbacks cb 0x7f0a680095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:27:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.415+0000 7f0a7b400640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a60730 0x557607a695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.415+0000 7f0a80e00640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a6a5c0 0x557607a6aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.416+0000 7f0a7be00640 1 -- 172.21.15.131:0/3900687763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a6a5c0 msgr2=0x557607a6aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.416+0000 7f0a7be00640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a6a5c0 0x557607a6aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.416+0000 7f0a7be00640 1 -- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a60730 msgr2=0x557607a695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.416+0000 7f0a7be00640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a60730 0x557607a695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.416+0000 7f0a7be00640 1 -- 172.21.15.131:0/3900687763 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557607979cb0 con 0x557607a5abd0 2024-04-26T20:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.416+0000 7f0a80e00640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a6a5c0 0x557607a6aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.417+0000 7f0a7be00640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5abd0 0x557607a69070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0bc080 tx=0x7f0a6c00ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd86797059071fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.417+0000 7f0a66000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:27:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.418+0000 7f0a66000640 1 -- 172.21.15.131:0/3900687763 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0da050 con 0x557607a5abd0 2024-04-26T20:27:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.418+0000 7f0a66000640 1 -- 172.21.15.131:0/3900687763 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0a6c0ba680 con 0x557607a5abd0 2024-04-26T20:27:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.418+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a68009b30 con 0x557607a5abd0 2024-04-26T20:27:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.419+0000 7f0a66000640 1 -- 172.21.15.131:0/3900687763 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a6c005020 con 0x557607a5abd0 2024-04-26T20:27:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.420+0000 7f0a85bc8740 10 client.16599 fetch_fsmap learned FSMap version 27 2024-04-26T20:27:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.420+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557607979cb0 con 0x557607a5abd0 2024-04-26T20:27:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a66000640 1 -- 172.21.15.131:0/3900687763 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a68009b30 con 0x557607a5abd0 2024-04-26T20:27:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a66000640 1 -- 172.21.15.131:0/3900687763 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a6c0bc5e0 con 0x557607a5abd0 2024-04-26T20:27:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a85bc8740 10 client.16599 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:27:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a85bc8740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:27:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a85bc8740 20 client.16599 populate_metadata read hostname 'smithi131' 2024-04-26T20:27:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607ad6ae0 0x557607ad8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a85bc8740 4 client.16599 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:27:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.422+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576079c6e00 con 0x557607ad6ae0 2024-04-26T20:27:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.423+0000 7f0a7b400640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607ad6ae0 0x557607ad8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.423+0000 7f0a7b400640 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607ad6ae0 0x557607ad8ed0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f244c68f39bfb5 server_cookie=a4e7a70c357423cf in_seq=0 out_seq=0 2024-04-26T20:27:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.423+0000 7f0a66000640 10 client.16599 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.453+0000 7f0a66000640 1 -- 172.21.15.131:0/3900687763 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576079c6e00 con 0x557607ad6ae0 2024-04-26T20:27:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.453+0000 7f0a66000640 10 client.16599 handle_command_reply: tid=0 2024-04-26T20:27:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.453+0000 7f0a85bc8740 1 client.16599 shutdown 2024-04-26T20:27:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5abd0 msgr2=0x557607a69070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5abd0 0x557607a69070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0bc080 tx=0x7f0a6c00ac00 comp rx=0 tx=0).stop 2024-04-26T20:27:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 shutdown_connections 2024-04-26T20:27:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x557607ad6ae0 0x557607ad8ed0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557607a6a5c0 0x557607a6aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557607a60730 0x557607a695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 --2- 172.21.15.131:0/3900687763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557607a5abd0 0x557607a69070 secure :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0bc080 tx=0x7f0a6c00ac00 comp rx=0 tx=0).stop 2024-04-26T20:27:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a66000640 0 client.16599 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.454+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 >> 172.21.15.131:0/3900687763 conn(0x557607a56240 msgr2=0x557607a646a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.455+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 shutdown_connections 2024-04-26T20:27:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.455+0000 7f0a85bc8740 1 -- 172.21.15.131:0/3900687763 wait complete. 2024-04-26T20:27:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.455+0000 7f0a85bc8740 20 client.16599 trim_cache size 0 max 16384 2024-04-26T20:27:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:27:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:27:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:27:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:27:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0a600a3eb0 msgr2=0x7f0a600a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0a600a3eb0 0x7f0a600a6370 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005fd0 tx=0x7f0a74009af0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c100bc0 msgr2=0x7f0a7c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c106940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c005ab0 tx=0x7f0a6c004c50 comp rx=0 tx=0).stop 2024-04-26T20:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 shutdown_connections 2024-04-26T20:27:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0a600a3eb0 0x7f0a600a6370 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c102770 0x7f0a7c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101db0 0x7f0a7c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 --2- 172.21.15.131:0/2619191512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a7c100bc0 0x7f0a7c106940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.456+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 >> 172.21.15.131:0/2619191512 conn(0x7f0a7c0fc150 msgr2=0x7f0a7c10aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.457+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 shutdown_connections 2024-04-26T20:27:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:38.457+0000 7f0a82e00640 1 -- 172.21.15.131:0/2619191512 wait complete. 2024-04-26T20:27:37.044 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-04-26T20:27:37.050 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-26T20:27:37.059 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-04-26T20:27:37.097 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-26T20:27:37.108 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-04-26T20:27:37.117 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-26T20:27:37.126 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-04-26T20:27:37.135 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-26T20:27:37.144 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-04-26T20:27:37.210 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-26T20:27:37.218 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-04-26T20:27:37.228 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-26T20:27:37.236 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-04-26T20:27:37.244 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-26T20:27:37.257 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-04-26T20:27:37.310 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-26T20:27:37.342 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-04-26T20:27:37.351 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-26T20:27:37.361 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-04-26T20:27:37.370 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-26T20:27:37.378 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-04-26T20:27:37.469 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-26T20:27:37.476 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-04-26T20:27:37.482 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-26T20:27:37.489 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-04-26T20:27:37.496 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-26T20:27:37.554 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-04-26T20:27:37.562 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-26T20:27:37.570 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-04-26T20:27:37.577 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-26T20:27:37.589 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-04-26T20:27:37.597 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-26T20:27:37.659 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-04-26T20:27:37.667 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-26T20:27:37.675 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-04-26T20:27:37.709 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-26T20:27:37.717 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-04-26T20:27:37.754 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-26T20:27:37.760 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-04-26T20:27:37.767 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-26T20:27:37.775 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-04-26T20:27:37.782 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-26T20:27:37.790 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-04-26T20:27:37.860 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-26T20:27:37.868 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-04-26T20:27:37.875 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-26T20:27:37.883 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-04-26T20:27:37.890 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-26T20:27:37.898 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-04-26T20:27:37.948 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-26T20:27:37.981 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-04-26T20:27:37.989 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-26T20:27:37.996 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-04-26T20:27:38.003 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-26T20:27:38.056 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-04-26T20:27:38.063 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-26T20:27:38.071 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-04-26T20:27:38.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-26T20:27:38.086 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-04-26T20:27:38.092 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-26T20:27:38.147 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-04-26T20:27:38.154 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-26T20:27:38.163 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-04-26T20:27:38.170 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-26T20:27:38.177 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-04-26T20:27:38.185 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-26T20:27:38.227 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-04-26T20:27:38.266 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-26T20:27:38.273 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-04-26T20:27:38.281 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-26T20:27:38.289 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-04-26T20:27:38.320 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-26T20:27:38.329 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-04-26T20:27:38.337 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-26T20:27:38.344 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-04-26T20:27:38.353 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-26T20:27:38.362 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-04-26T20:27:38.413 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-26T20:27:38.422 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-04-26T20:27:38.431 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-26T20:27:38.440 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-04-26T20:27:38.449 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-26T20:27:38.458 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-04-26T20:27:38.506 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-26T20:27:38.542 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:39 smithi131 ceph-mon[27919]: pgmap v909: 129 pgs: 129 active+clean; 155 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 898 KiB/s wr, 212 op/s 2024-04-26T20:27:38.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:41 smithi131 ceph-mon[27919]: pgmap v910: 129 pgs: 129 active+clean; 163 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 920 KiB/s wr, 220 op/s 2024-04-26T20:27:38.561 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:33 smithi148 ceph-mon[32281]: pgmap v936: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 180 KiB/s wr, 72 op/s 2024-04-26T20:27:38.570 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:35 smithi148 ceph-mon[32281]: pgmap v937: 129 pgs: 129 active+clean; 5.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 36 KiB/s wr, 35 op/s 2024-04-26T20:27:38.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:37 smithi148 ceph-mon[32281]: pgmap v938: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.7 KiB/s wr, 17 op/s 2024-04-26T20:27:38.621 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:33 smithi157 ceph-mon[32703]: pgmap v936: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 180 KiB/s wr, 72 op/s 2024-04-26T20:27:38.630 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:35 smithi157 ceph-mon[32703]: pgmap v937: 129 pgs: 129 active+clean; 5.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 36 KiB/s wr, 35 op/s 2024-04-26T20:27:38.639 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:37 smithi157 ceph-mon[32703]: pgmap v938: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.7 KiB/s wr, 17 op/s 2024-04-26T20:27:38.648 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-04-26T20:27:38.658 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-26T20:27:38.667 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-04-26T20:27:38.718 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-26T20:27:38.726 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-04-26T20:27:38.735 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-26T20:27:38.743 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-04-26T20:27:38.752 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-26T20:27:38.786 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-04-26T20:27:38.816 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-26T20:27:38.825 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-04-26T20:27:38.834 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-26T20:27:38.845 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-04-26T20:27:38.943 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-26T20:27:38.952 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-04-26T20:27:38.960 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-26T20:27:38.967 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-04-26T20:27:38.974 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-26T20:27:38.981 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-04-26T20:27:39.038 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-26T20:27:39.047 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-04-26T20:27:39.056 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-26T20:27:39.063 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-04-26T20:27:39.071 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-26T20:27:39.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-04-26T20:27:39.130 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-26T20:27:39.175 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-04-26T20:27:39.182 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-26T20:27:39.190 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-04-26T20:27:39.199 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-26T20:27:39.248 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-04-26T20:27:39.255 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-26T20:27:39.261 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-04-26T20:27:39.267 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-26T20:27:39.274 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-04-26T20:27:39.281 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-26T20:27:39.319 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-04-26T20:27:39.326 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-26T20:27:39.333 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-04-26T20:27:39.340 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-26T20:27:39.347 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-04-26T20:27:39.354 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-26T20:27:39.398 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-04-26T20:27:39.425 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-26T20:27:39.431 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-04-26T20:27:39.439 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-26T20:27:39.446 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-04-26T20:27:39.528 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-26T20:27:39.536 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-04-26T20:27:39.544 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-26T20:27:39.558 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-04-26T20:27:39.565 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-26T20:27:39.573 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-04-26T20:27:39.616 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-26T20:27:39.625 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-04-26T20:27:39.632 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-26T20:27:39.640 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-04-26T20:27:39.646 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-26T20:27:39.653 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-04-26T20:27:39.710 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-26T20:27:39.750 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-04-26T20:27:39.756 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-26T20:27:39.763 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-04-26T20:27:39.772 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-26T20:27:39.811 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-04-26T20:27:39.819 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-26T20:27:39.828 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-04-26T20:27:39.837 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-26T20:27:39.846 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-04-26T20:27:39.855 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-26T20:27:39.912 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-04-26T20:27:39.922 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-26T20:27:39.931 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-04-26T20:27:39.941 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-26T20:27:39.950 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-04-26T20:27:39.959 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-26T20:27:40.019 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-04-26T20:27:40.027 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-26T20:27:40.037 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-04-26T20:27:40.083 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-26T20:27:40.092 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-04-26T20:27:40.128 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-26T20:27:40.137 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-04-26T20:27:40.146 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-26T20:27:40.156 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-04-26T20:27:40.164 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-26T20:27:40.173 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-04-26T20:27:40.227 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-26T20:27:40.235 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-04-26T20:27:40.244 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-26T20:27:40.252 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-04-26T20:27:40.261 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-26T20:27:40.270 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-04-26T20:27:40.318 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-26T20:27:40.352 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-04-26T20:27:40.359 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-26T20:27:40.370 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-04-26T20:27:40.379 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-26T20:27:40.419 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-04-26T20:27:40.428 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-26T20:27:40.436 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-04-26T20:27:40.445 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-26T20:27:40.454 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-04-26T20:27:40.463 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-26T20:27:40.552 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-04-26T20:27:40.560 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-26T20:27:40.568 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-04-26T20:27:40.576 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-26T20:27:40.585 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-04-26T20:27:40.592 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-26T20:27:40.660 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-04-26T20:27:40.699 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-26T20:27:40.707 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-04-26T20:27:40.714 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-26T20:27:40.723 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-04-26T20:27:40.782 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-26T20:27:40.788 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-04-26T20:27:40.794 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-26T20:27:40.802 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-04-26T20:27:40.809 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-26T20:27:40.815 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-04-26T20:27:40.879 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-26T20:27:40.887 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-04-26T20:27:40.895 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-26T20:27:40.902 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-04-26T20:27:40.909 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-26T20:27:40.917 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-04-26T20:27:40.963 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-26T20:27:41.009 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-04-26T20:27:41.018 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-26T20:27:41.026 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-04-26T20:27:41.034 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-26T20:27:41.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-04-26T20:27:41.084 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-26T20:27:41.090 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-04-26T20:27:41.096 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-26T20:27:41.103 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-04-26T20:27:41.110 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-26T20:27:41.164 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-04-26T20:27:41.170 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-26T20:27:41.177 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-04-26T20:27:41.183 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-26T20:27:41.190 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-04-26T20:27:41.197 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-26T20:27:41.253 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-04-26T20:27:41.307 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-26T20:27:41.315 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-04-26T20:27:41.324 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-26T20:27:41.332 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-04-26T20:27:41.380 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-26T20:27:41.387 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-04-26T20:27:41.397 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-26T20:27:41.406 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-04-26T20:27:41.414 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-26T20:27:41.420 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-04-26T20:27:41.481 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-26T20:27:41.490 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-04-26T20:27:41.499 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-26T20:27:41.508 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-04-26T20:27:41.517 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-26T20:27:41.526 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-04-26T20:27:41.597 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-26T20:27:41.659 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-04-26T20:27:41.666 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-26T20:27:41.674 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-04-26T20:27:41.682 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-26T20:27:41.773 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-04-26T20:27:41.783 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-26T20:27:41.792 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-04-26T20:27:41.801 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-26T20:27:41.810 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-04-26T20:27:41.819 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-26T20:27:41.893 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-04-26T20:27:41.900 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-26T20:27:41.911 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-04-26T20:27:41.920 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-26T20:27:41.928 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-04-26T20:27:41.937 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-26T20:27:41.997 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-04-26T20:27:42.041 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-26T20:27:42.050 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-04-26T20:27:42.060 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-26T20:27:42.069 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-04-26T20:27:42.127 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-26T20:27:42.135 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-04-26T20:27:42.142 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-26T20:27:42.150 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-04-26T20:27:42.158 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-26T20:27:42.165 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-04-26T20:27:42.225 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-26T20:27:42.232 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-04-26T20:27:42.239 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-26T20:27:42.247 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-04-26T20:27:42.255 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-26T20:27:42.262 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-04-26T20:27:42.317 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-26T20:27:42.357 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-04-26T20:27:42.364 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-26T20:27:42.371 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-04-26T20:27:42.378 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-26T20:27:42.439 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-04-26T20:27:42.446 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-26T20:27:42.453 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-04-26T20:27:42.459 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-26T20:27:42.466 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-04-26T20:27:42.472 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-26T20:27:42.530 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-04-26T20:27:42.538 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-26T20:27:42.545 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-04-26T20:27:42.553 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-26T20:27:42.561 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-04-26T20:27:42.568 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-26T20:27:42.617 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-04-26T20:27:42.674 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-26T20:27:42.680 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-04-26T20:27:42.690 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-26T20:27:42.695 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-04-26T20:27:42.735 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-26T20:27:42.743 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-04-26T20:27:42.750 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-26T20:27:42.757 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-04-26T20:27:42.764 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-26T20:27:42.772 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-04-26T20:27:42.796 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-26T20:27:42.826 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-04-26T20:27:42.835 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-26T20:27:42.844 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-04-26T20:27:42.853 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-26T20:27:42.862 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-04-26T20:27:42.915 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-26T20:27:42.965 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-04-26T20:27:42.975 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-26T20:27:42.984 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-04-26T20:27:42.993 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-26T20:27:43.025 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-04-26T20:27:43.034 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-26T20:27:43.044 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-04-26T20:27:43.052 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-26T20:27:43.061 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-04-26T20:27:43.069 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-26T20:27:43.135 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-04-26T20:27:43.142 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-26T20:27:43.151 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-04-26T20:27:43.160 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-26T20:27:43.169 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-04-26T20:27:43.178 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-26T20:27:43.218 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-04-26T20:27:43.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:43 smithi131 ceph-mon[27919]: pgmap v911: 129 pgs: 129 active+clean; 187 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 942 KiB/s wr, 237 op/s 2024-04-26T20:27:43.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:45 smithi131 ceph-mon[27919]: pgmap v912: 129 pgs: 129 active+clean; 200 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-04-26T20:27:43.282 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-26T20:27:43.290 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-04-26T20:27:43.330 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-26T20:27:43.339 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-04-26T20:27:43.348 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-26T20:27:43.356 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-04-26T20:27:43.365 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-26T20:27:43.373 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-04-26T20:27:43.401 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-26T20:27:43.409 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-04-26T20:27:43.418 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-26T20:27:43.427 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-04-26T20:27:43.435 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-26T20:27:43.443 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-04-26T20:27:43.492 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-26T20:27:43.532 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-04-26T20:27:43.541 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-26T20:27:43.550 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-04-26T20:27:43.558 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-26T20:27:43.605 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-04-26T20:27:43.613 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-26T20:27:43.620 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-04-26T20:27:43.627 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-26T20:27:43.635 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-04-26T20:27:43.642 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-26T20:27:43.692 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-04-26T20:27:43.699 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-26T20:27:43.706 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-04-26T20:27:43.713 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-26T20:27:43.721 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-04-26T20:27:43.729 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-26T20:27:43.770 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-04-26T20:27:43.807 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-26T20:27:43.813 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-04-26T20:27:43.820 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-26T20:27:43.827 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-04-26T20:27:43.902 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-26T20:27:43.909 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-04-26T20:27:43.917 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-26T20:27:43.924 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-04-26T20:27:43.931 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-26T20:27:43.938 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-04-26T20:27:43.976 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-26T20:27:43.999 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-04-26T20:27:44.007 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-26T20:27:44.013 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-04-26T20:27:44.020 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-26T20:27:44.029 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-04-26T20:27:44.072 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-26T20:27:44.107 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-04-26T20:27:44.114 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-26T20:27:44.122 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-04-26T20:27:44.130 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-26T20:27:44.173 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-04-26T20:27:44.181 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-26T20:27:44.190 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-04-26T20:27:44.198 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-26T20:27:44.207 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-04-26T20:27:44.214 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-26T20:27:44.238 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-04-26T20:27:44.276 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-26T20:27:44.283 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-04-26T20:27:44.290 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-26T20:27:44.298 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-04-26T20:27:44.307 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-26T20:27:44.343 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-04-26T20:27:44.423 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-26T20:27:44.433 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-04-26T20:27:44.442 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-26T20:27:44.451 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-04-26T20:27:44.507 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-26T20:27:44.517 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-04-26T20:27:44.526 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-26T20:27:44.557 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-04-26T20:27:44.565 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-26T20:27:44.574 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-04-26T20:27:44.609 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-26T20:27:44.619 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-04-26T20:27:44.628 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-26T20:27:44.637 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-04-26T20:27:44.648 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:47 smithi131 ceph-mon[27919]: pgmap v913: 129 pgs: 129 active+clean; 180 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 400 KiB/s wr, 140 op/s 2024-04-26T20:27:44.657 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-26T20:27:44.705 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-04-26T20:27:44.740 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-26T20:27:44.748 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-04-26T20:27:44.761 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-26T20:27:44.770 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-04-26T20:27:44.857 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-26T20:27:44.864 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-04-26T20:27:44.874 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-26T20:27:44.884 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-04-26T20:27:44.893 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-26T20:27:44.904 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-04-26T20:27:44.965 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-26T20:27:44.974 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-04-26T20:27:44.983 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-26T20:27:44.991 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-04-26T20:27:45.000 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-26T20:27:45.008 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-04-26T20:27:45.043 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-26T20:27:45.081 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-04-26T20:27:45.090 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-26T20:27:45.099 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-04-26T20:27:45.106 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-26T20:27:45.164 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-04-26T20:27:45.172 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-26T20:27:45.180 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-04-26T20:27:45.187 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-26T20:27:45.194 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-04-26T20:27:45.201 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-26T20:27:45.250 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-04-26T20:27:45.258 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-26T20:27:45.266 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-04-26T20:27:45.273 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-26T20:27:45.281 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-04-26T20:27:45.288 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-26T20:27:45.332 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-04-26T20:27:45.373 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-26T20:27:45.380 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-04-26T20:27:45.389 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-26T20:27:45.397 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-04-26T20:27:45.430 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-26T20:27:45.438 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-04-26T20:27:45.445 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-26T20:27:45.453 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-04-26T20:27:45.460 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-26T20:27:45.469 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-04-26T20:27:45.525 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-26T20:27:45.533 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-04-26T20:27:45.540 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-26T20:27:45.546 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-04-26T20:27:45.553 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-26T20:27:45.560 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-04-26T20:27:45.623 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-26T20:27:45.685 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-04-26T20:27:45.692 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-26T20:27:45.698 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-04-26T20:27:45.705 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-26T20:27:45.764 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-04-26T20:27:45.770 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-26T20:27:45.780 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-04-26T20:27:45.788 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-26T20:27:45.796 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-04-26T20:27:45.803 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-26T20:27:45.859 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-04-26T20:27:45.865 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-26T20:27:45.872 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-04-26T20:27:45.879 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-26T20:27:45.886 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-04-26T20:27:45.893 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-26T20:27:45.933 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-04-26T20:27:45.989 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-26T20:27:45.999 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-04-26T20:27:46.008 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-26T20:27:46.017 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-04-26T20:27:46.064 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-26T20:27:46.070 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-04-26T20:27:46.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-26T20:27:46.088 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-04-26T20:27:46.096 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-26T20:27:46.105 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-04-26T20:27:46.160 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-26T20:27:46.169 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-04-26T20:27:46.178 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-26T20:27:46.187 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-04-26T20:27:46.195 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-26T20:27:46.204 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-04-26T20:27:46.257 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-26T20:27:46.298 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-04-26T20:27:46.307 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-26T20:27:46.314 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-04-26T20:27:46.322 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-26T20:27:46.373 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-04-26T20:27:46.381 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-26T20:27:46.389 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-04-26T20:27:46.398 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-26T20:27:46.406 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-04-26T20:27:46.412 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-26T20:27:46.470 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-04-26T20:27:46.477 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-26T20:27:46.485 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-04-26T20:27:46.492 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-26T20:27:46.498 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-04-26T20:27:46.507 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-26T20:27:46.558 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-04-26T20:27:46.597 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-26T20:27:46.606 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-04-26T20:27:46.612 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-26T20:27:46.618 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-04-26T20:27:46.672 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-26T20:27:46.679 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-04-26T20:27:46.685 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-26T20:27:46.691 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-04-26T20:27:46.699 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-26T20:27:46.706 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-04-26T20:27:46.760 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-26T20:27:46.765 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-04-26T20:27:46.773 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-26T20:27:46.780 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-04-26T20:27:46.787 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-26T20:27:46.795 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-04-26T20:27:46.834 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-26T20:27:46.873 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-04-26T20:27:46.879 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-26T20:27:46.886 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-04-26T20:27:46.894 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-26T20:27:46.943 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-04-26T20:27:46.952 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-26T20:27:46.983 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-04-26T20:27:46.990 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-26T20:27:47.000 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-04-26T20:27:47.007 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-26T20:27:47.050 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-04-26T20:27:47.057 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-26T20:27:47.064 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-04-26T20:27:47.070 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-26T20:27:47.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-04-26T20:27:47.085 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-26T20:27:47.133 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-04-26T20:27:47.172 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-26T20:27:47.181 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-04-26T20:27:47.189 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-26T20:27:47.198 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-04-26T20:27:47.262 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-26T20:27:47.271 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-04-26T20:27:47.280 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-26T20:27:47.287 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-04-26T20:27:47.295 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-26T20:27:47.304 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-04-26T20:27:47.358 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-26T20:27:47.366 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-04-26T20:27:47.374 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-26T20:27:47.382 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-04-26T20:27:47.391 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-26T20:27:47.401 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-04-26T20:27:47.436 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-26T20:27:47.489 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-04-26T20:27:47.495 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-26T20:27:47.503 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-04-26T20:27:47.511 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-26T20:27:47.576 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-04-26T20:27:47.583 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-26T20:27:47.592 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-04-26T20:27:47.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:49 smithi131 ceph-mon[27919]: pgmap v914: 129 pgs: 129 active+clean; 209 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 497 KiB/s wr, 171 op/s 2024-04-26T20:27:47.611 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-26T20:27:47.620 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-04-26T20:27:47.670 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-26T20:27:47.679 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-04-26T20:27:47.688 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-26T20:27:47.698 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-04-26T20:27:47.705 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-26T20:27:47.716 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-04-26T20:27:47.781 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-26T20:27:47.831 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-04-26T20:27:47.837 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-26T20:27:47.845 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-04-26T20:27:47.851 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-26T20:27:47.917 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-04-26T20:27:47.923 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-26T20:27:47.929 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-04-26T20:27:47.942 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-26T20:27:47.949 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-04-26T20:27:47.956 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-26T20:27:48.005 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-04-26T20:27:48.020 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-26T20:27:48.027 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-04-26T20:27:48.033 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-26T20:27:48.039 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-04-26T20:27:48.046 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-26T20:27:48.091 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-04-26T20:27:48.149 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-26T20:27:48.157 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-04-26T20:27:48.164 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-26T20:27:48.171 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-04-26T20:27:48.227 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-26T20:27:48.233 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-04-26T20:27:48.239 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-26T20:27:48.246 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-04-26T20:27:48.253 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-26T20:27:48.261 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-04-26T20:27:48.327 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-26T20:27:48.335 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-04-26T20:27:48.342 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-26T20:27:48.350 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-04-26T20:27:48.357 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-26T20:27:48.365 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-04-26T20:27:48.421 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-26T20:27:48.464 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-04-26T20:27:48.470 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-26T20:27:48.481 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-04-26T20:27:48.490 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-26T20:27:48.539 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-04-26T20:27:48.549 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-26T20:27:48.556 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-04-26T20:27:48.564 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-26T20:27:48.573 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-04-26T20:27:48.582 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-26T20:27:48.630 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-04-26T20:27:48.637 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-26T20:27:48.645 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-04-26T20:27:48.652 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-26T20:27:48.660 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-04-26T20:27:48.671 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-26T20:27:48.717 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-04-26T20:27:48.772 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-26T20:27:48.778 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-04-26T20:27:48.787 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-26T20:27:48.797 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-04-26T20:27:48.847 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-26T20:27:48.857 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-04-26T20:27:48.867 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-26T20:27:48.876 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-04-26T20:27:48.884 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-26T20:27:48.892 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-04-26T20:27:48.947 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-26T20:27:48.956 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-04-26T20:27:48.965 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-26T20:27:48.973 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-04-26T20:27:48.982 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-26T20:27:48.990 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-04-26T20:27:49.041 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-26T20:27:49.098 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-04-26T20:27:49.104 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-26T20:27:49.113 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-04-26T20:27:49.122 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-26T20:27:49.182 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:39 smithi148 ceph-mon[32281]: pgmap v939: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 14 op/s 2024-04-26T20:27:49.190 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:41 smithi148 ceph-mon[32281]: pgmap v940: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-04-26T20:27:49.197 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:43 smithi148 ceph-mon[32281]: pgmap v941: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-26T20:27:49.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:45 smithi148 ceph-mon[32281]: pgmap v942: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T20:27:49.214 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:47 smithi148 ceph-mon[32281]: pgmap v943: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-04-26T20:27:49.223 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:39 smithi157 ceph-mon[32703]: pgmap v939: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 14 op/s 2024-04-26T20:27:49.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:41 smithi157 ceph-mon[32703]: pgmap v940: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-04-26T20:27:49.285 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:43 smithi157 ceph-mon[32703]: pgmap v941: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-26T20:27:49.293 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:45 smithi157 ceph-mon[32703]: pgmap v942: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T20:27:49.300 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:47 smithi157 ceph-mon[32703]: pgmap v943: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-04-26T20:27:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 -- 172.21.15.131:0/2156450960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48981111b0 msgr2=0x7f48981135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 --2- 172.21.15.131:0/2156450960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48981111b0 0x7f48981135a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4890009ca0 tx=0x7f489002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 -- 172.21.15.131:0/2156450960 shutdown_connections 2024-04-26T20:27:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 --2- 172.21.15.131:0/2156450960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48981111b0 0x7f48981135a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 --2- 172.21.15.131:0/2156450960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489810a290 0x7f4898110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 --2- 172.21.15.131:0/2156450960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4898109930 0x7f4898109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 -- 172.21.15.131:0/2156450960 >> 172.21.15.131:0/2156450960 conn(0x7f489806f9a0 msgr2=0x7f4898071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.620+0000 7f489d400640 1 -- 172.21.15.131:0/2156450960 shutdown_connections 2024-04-26T20:27:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.621+0000 7f489d400640 1 -- 172.21.15.131:0/2156450960 wait complete. 2024-04-26T20:27:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.621+0000 7f489d400640 1 Processor -- start 2024-04-26T20:27:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f489d400640 1 -- start start 2024-04-26T20:27:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f489d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4898109930 0x7f48981a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f489d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489810a290 0x7f48981a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f489d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48981111b0 0x7f48981a8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f489d400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4898116d20 con 0x7f489810a290 2024-04-26T20:27:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f489d400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4898116ba0 con 0x7f4898109930 2024-04-26T20:27:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f489d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4898116ea0 con 0x7f48981111b0 2024-04-26T20:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f4896800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4898109930 0x7f48981a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f488fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489810a290 0x7f48981a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f4896800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4898109930 0x7f48981a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57558/0 (socket says 172.21.15.131:57558) 2024-04-26T20:27:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.622+0000 7f4896800640 1 -- 172.21.15.131:0/1921905057 learned_addr learned my addr 172.21.15.131:0/1921905057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4897200640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48981111b0 0x7f48981a8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4896800640 1 -- 172.21.15.131:0/1921905057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48981111b0 msgr2=0x7f48981a8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4896800640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48981111b0 0x7f48981a8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4896800640 1 -- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489810a290 msgr2=0x7f48981a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4896800640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489810a290 0x7f48981a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4896800640 1 -- 172.21.15.131:0/1921905057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489810aa40 con 0x7f4898109930 2024-04-26T20:27:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4897200640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48981111b0 0x7f48981a8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f4896800640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4898109930 0x7f48981a11c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f488800e9f0 tx=0x7f488800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f488f400640 1 -- 172.21.15.131:0/1921905057 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488800ccc0 con 0x7f4898109930 2024-04-26T20:27:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f488f400640 1 -- 172.21.15.131:0/1921905057 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48880108e0 con 0x7f4898109930 2024-04-26T20:27:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f488f400640 1 -- 172.21.15.131:0/1921905057 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888018880 con 0x7f4898109930 2024-04-26T20:27:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.623+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489810ad30 con 0x7f4898109930 2024-04-26T20:27:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.624+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489810b2d0 con 0x7f4898109930 2024-04-26T20:27:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.627+0000 7f488f400640 1 -- 172.21.15.131:0/1921905057 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4888010430 con 0x7f4898109930 2024-04-26T20:27:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 Processor -- start 2024-04-26T20:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 -- start start 2024-04-26T20:27:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c63fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c647bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b47c5a4380 con 0x55b47c645730 2024-04-26T20:27:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b47c540c30 con 0x55b47c63fbd0 2024-04-26T20:27:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.629+0000 7f48a0375740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b47c5a0340 con 0x55b47c6405c0 2024-04-26T20:27:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f4896800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f4897200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c63fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c647bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f4897200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c63fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57574/0 (socket says 172.21.15.131:57574) 2024-04-26T20:27:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f4897200640 1 -- 172.21.15.131:0/1052261644 learned_addr learned my addr 172.21.15.131:0/1052261644 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488f400640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f487c0a3d70 0x7f487c0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488f400640 1 -- 172.21.15.131:0/1921905057 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f48880d0d70 con 0x7f4898109930 2024-04-26T20:27:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488fe00640 1 -- 172.21.15.131:0/1052261644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 msgr2=0x55b47c6450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488fe00640 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488fe00640 1 -- 172.21.15.131:0/1052261644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 msgr2=0x55b47c63fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488fe00640 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c63fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f488fe00640 1 -- 172.21.15.131:0/1052261644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47c562940 con 0x55b47c645730 2024-04-26T20:27:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.630+0000 7f4896800640 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.631+0000 7f4897200640 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c63fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.631+0000 7f4897200640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f487c0a3d70 0x7f487c0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.631+0000 7f488fe00640 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c647bc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f488000abc0 tx=0x7f488000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf428bee570e5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.631+0000 7f4887400640 1 -- 172.21.15.131:0/1052261644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880005220 con 0x55b47c645730 2024-04-26T20:27:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.631+0000 7f4887400640 1 -- 172.21.15.131:0/1052261644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4880004c50 con 0x55b47c645730 2024-04-26T20:27:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.631+0000 7f4887400640 1 -- 172.21.15.131:0/1052261644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880018800 con 0x55b47c645730 2024-04-26T20:27:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f48a0375740 1 -- 172.21.15.131:0/1052261644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 msgr2=0x55b47c647bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f48a0375740 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c647bc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f488000abc0 tx=0x7f488000f790 comp rx=0 tx=0).stop 2024-04-26T20:27:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f4897200640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f487c0a3d70 0x7f487c0a6230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f489002f540 tx=0x7f48900095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f48a0375740 1 -- 172.21.15.131:0/1052261644 shutdown_connections 2024-04-26T20:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f48a0375740 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c647bc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f48a0375740 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f48a0375740 1 --2- 172.21.15.131:0/1052261644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c63fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.632+0000 7f48a0375740 1 -- 172.21.15.131:0/1052261644 >> 172.21.15.131:0/1052261644 conn(0x55b47c63b240 msgr2=0x55b47c63b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.633+0000 7f48a0375740 1 -- 172.21.15.131:0/1052261644 shutdown_connections 2024-04-26T20:27:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.633+0000 7f48a0375740 1 -- 172.21.15.131:0/1052261644 wait complete. 2024-04-26T20:27:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.633+0000 7f48a0375740 1 -- start start 2024-04-26T20:27:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 1 Processor -- start 2024-04-26T20:27:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c649db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c64a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c651330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b47c6486e0 con 0x55b47c645730 2024-04-26T20:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b47c52bc60 con 0x55b47c63fbd0 2024-04-26T20:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b47c648860 con 0x55b47c6405c0 2024-04-26T20:27:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.634+0000 7f48a0375740 10 client.0 _ll_register_callbacks cb 0x55b47c557e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:27:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f488fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c64a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c649db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4897200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c651330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c649db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57576/0 (socket says 172.21.15.131:57576) 2024-04-26T20:27:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f488fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c64a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58314/0 (socket says 172.21.15.131:58314) 2024-04-26T20:27:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 -- 172.21.15.131:0/2226692961 learned_addr learned my addr 172.21.15.131:0/2226692961 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 -- 172.21.15.131:0/2226692961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 msgr2=0x55b47c64a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c64a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 -- 172.21.15.131:0/2226692961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 msgr2=0x55b47c651330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c651330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 -- 172.21.15.131:0/2226692961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47c53a510 con 0x55b47c63fbd0 2024-04-26T20:27:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f4896800640 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c649db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f488800e9c0 tx=0x7f488809c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506d2afa9c8df501 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f487be00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:27:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f487be00640 1 -- 172.21.15.131:0/2226692961 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488809cb00 con 0x55b47c63fbd0 2024-04-26T20:27:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f487be00640 1 -- 172.21.15.131:0/2226692961 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f48880ce580 con 0x55b47c63fbd0 2024-04-26T20:27:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.635+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4880004c50 con 0x55b47c63fbd0 2024-04-26T20:27:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f487be00640 1 -- 172.21.15.131:0/2226692961 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f48880a0eb0 con 0x55b47c63fbd0 2024-04-26T20:27:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 10 client.26732 fetch_fsmap learned FSMap version 27 2024-04-26T20:27:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b47c2b6210 con 0x55b47c63fbd0 2024-04-26T20:27:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f487be00640 1 -- 172.21.15.131:0/2226692961 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4880004c50 con 0x55b47c63fbd0 2024-04-26T20:27:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f487be00640 1 -- 172.21.15.131:0/2226692961 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f48880989c0 con 0x55b47c63fbd0 2024-04-26T20:27:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 10 client.26732 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:27:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 10 client.26732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:27:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 20 client.26732 populate_metadata read hostname 'smithi131' 2024-04-26T20:27:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c655870 0x55b47c657d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 4 client.26732 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:27:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b47c5b1f90 con 0x55b47c655870 2024-04-26T20:27:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.636+0000 7f488fe00640 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c655870 0x55b47c657d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.637+0000 7f488fe00640 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c655870 0x55b47c657d00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b71613193a7c87 server_cookie=cd31da61d32ca94c in_seq=0 out_seq=0 2024-04-26T20:27:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.637+0000 7f487be00640 10 client.26732 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.638+0000 7f487be00640 1 -- 172.21.15.131:0/2226692961 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b47c5b1f90 con 0x55b47c655870 2024-04-26T20:27:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.638+0000 7f487be00640 10 client.26732 handle_command_reply: tid=0 2024-04-26T20:27:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.638+0000 7f48a0375740 1 client.26732 shutdown 2024-04-26T20:27:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.638+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 msgr2=0x55b47c649db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.638+0000 7f48a0375740 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c649db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f488800e9c0 tx=0x7f488809c4e0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 shutdown_connections 2024-04-26T20:27:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c655870 0x55b47c657d00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c645730 0x55b47c651330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c64a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 --2- 172.21.15.131:0/2226692961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c649db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 >> 172.21.15.131:0/2226692961 conn(0x55b47c63b240 msgr2=0x55b47c63a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f487be00640 0 client.26732 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 shutdown_connections 2024-04-26T20:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 1 -- 172.21.15.131:0/2226692961 wait complete. 2024-04-26T20:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.639+0000 7f48a0375740 20 client.26732 trim_cache size 0 max 16384 2024-04-26T20:27:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 Processor -- start 2024-04-26T20:27:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 -- start start 2024-04-26T20:27:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c63fbd0 0x55b47c63e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c645730 0x55b47c6499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b8740 con 0x55b47c63fbd0 2024-04-26T20:27:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b6010 con 0x55b47c645730 2024-04-26T20:27:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.656+0000 7f48a0375740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b3e60 con 0x55b47c6405c0 2024-04-26T20:27:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f4896800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c63fbd0 0x55b47c63e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f4897200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c645730 0x55b47c6499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58330/0 (socket says 172.21.15.131:58330) 2024-04-26T20:27:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f4897200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c645730 0x55b47c6499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57586/0 (socket says 172.21.15.131:57586) 2024-04-26T20:27:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 -- 172.21.15.131:0/4158774535 learned_addr learned my addr 172.21.15.131:0/4158774535 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 -- 172.21.15.131:0/4158774535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c645730 msgr2=0x55b47c6499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 --2- 172.21.15.131:0/4158774535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c645730 0x55b47c6499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 -- 172.21.15.131:0/4158774535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c63fbd0 msgr2=0x55b47c63e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 --2- 172.21.15.131:0/4158774535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c63fbd0 0x55b47c63e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 -- 172.21.15.131:0/4158774535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47c382620 con 0x55b47c6405c0 2024-04-26T20:27:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f488fe00640 1 --2- 172.21.15.131:0/4158774535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6512f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f488000ab10 tx=0x7f48800057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a27f52be2027d8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.657+0000 7f4886a00640 1 -- 172.21.15.131:0/4158774535 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800291d0 con 0x55b47c6405c0 2024-04-26T20:27:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f4886a00640 1 -- 172.21.15.131:0/4158774535 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4880029820 con 0x55b47c6405c0 2024-04-26T20:27:52.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f4886a00640 1 -- 172.21.15.131:0/4158774535 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880018c30 con 0x55b47c6405c0 2024-04-26T20:27:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f48a0375740 1 -- 172.21.15.131:0/4158774535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 msgr2=0x55b47c6512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f48a0375740 1 --2- 172.21.15.131:0/4158774535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6512f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f488000ab10 tx=0x7f48800057f0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f48a0375740 1 -- 172.21.15.131:0/4158774535 shutdown_connections 2024-04-26T20:27:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f48a0375740 1 --2- 172.21.15.131:0/4158774535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c645730 0x55b47c6499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f48a0375740 1 --2- 172.21.15.131:0/4158774535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c6405c0 0x55b47c6512f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f48a0375740 1 --2- 172.21.15.131:0/4158774535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c63fbd0 0x55b47c63e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.658+0000 7f48a0375740 1 -- 172.21.15.131:0/4158774535 >> 172.21.15.131:0/4158774535 conn(0x55b47c63b240 msgr2=0x55b47c63e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.659+0000 7f48a0375740 1 -- 172.21.15.131:0/4158774535 shutdown_connections 2024-04-26T20:27:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.659+0000 7f48a0375740 1 -- 172.21.15.131:0/4158774535 wait complete. 2024-04-26T20:27:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.659+0000 7f48a0375740 1 -- start start 2024-04-26T20:27:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.659+0000 7f48a0375740 1 Processor -- start 2024-04-26T20:27:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f48a0375740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c64e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f48a0375740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c6405c0 0x55b47c64eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f48a0375740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c645730 0x55b47c64fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f48a0375740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b44d0 con 0x55b47c6405c0 2024-04-26T20:27:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f48a0375740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b5940 con 0x55b47c63fbd0 2024-04-26T20:27:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f48a0375740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b4ba0 con 0x55b47c645730 2024-04-26T20:27:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f48a0375740 10 client.0 _ll_register_callbacks cb 0x7f4888009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:27:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c64e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c64e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57600/0 (socket says 172.21.15.131:57600) 2024-04-26T20:27:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 -- 172.21.15.131:0/3028664722 learned_addr learned my addr 172.21.15.131:0/3028664722 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:27:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f488fe00640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c6405c0 0x55b47c64eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4897200640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c645730 0x55b47c64fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 -- 172.21.15.131:0/3028664722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c645730 msgr2=0x55b47c64fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c645730 0x55b47c64fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 -- 172.21.15.131:0/3028664722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c6405c0 msgr2=0x55b47c64eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c6405c0 0x55b47c64eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4896800640 1 -- 172.21.15.131:0/3028664722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47c55ecb0 con 0x55b47c63fbd0 2024-04-26T20:27:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f488fe00640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c6405c0 0x55b47c64eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.660+0000 7f4897200640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c645730 0x55b47c64fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f4896800640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c64e610 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4888098f60 tx=0x7f48880107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7626effe143df4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f487be00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:27:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f487be00640 1 -- 172.21.15.131:0/3028664722 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488809c640 con 0x55b47c63fbd0 2024-04-26T20:27:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f487be00640 1 -- 172.21.15.131:0/3028664722 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4888010a90 con 0x55b47c63fbd0 2024-04-26T20:27:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4880029820 con 0x55b47c63fbd0 2024-04-26T20:27:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f487be00640 1 -- 172.21.15.131:0/3028664722 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f48880a5770 con 0x55b47c63fbd0 2024-04-26T20:27:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f48a0375740 10 client.26738 fetch_fsmap learned FSMap version 27 2024-04-26T20:27:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b47c651240 con 0x55b47c63fbd0 2024-04-26T20:27:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f487be00640 1 -- 172.21.15.131:0/3028664722 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4880029820 con 0x55b47c63fbd0 2024-04-26T20:27:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f487be00640 1 -- 172.21.15.131:0/3028664722 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f48880a2040 con 0x55b47c63fbd0 2024-04-26T20:27:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f48a0375740 10 client.26738 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:27:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f48a0375740 10 client.26738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:27:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.661+0000 7f48a0375740 20 client.26738 populate_metadata read hostname 'smithi131' 2024-04-26T20:27:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.662+0000 7f48a0375740 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c6c47a0 0x55b47c6c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.662+0000 7f48a0375740 4 client.26738 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:27:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.662+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b47c5abe00 con 0x55b47c6c47a0 2024-04-26T20:27:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.662+0000 7f488fe00640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c6c47a0 0x55b47c6c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.662+0000 7f487be00640 10 client.26738 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.662+0000 7f488fe00640 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c6c47a0 0x55b47c6c6b90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be85762ade0efa0c server_cookie=48ac2b781c0449d4 in_seq=0 out_seq=0 2024-04-26T20:27:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f487be00640 1 -- 172.21.15.131:0/3028664722 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b47c5abe00 con 0x55b47c6c47a0 2024-04-26T20:27:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f487be00640 10 client.26738 handle_command_reply: tid=0 2024-04-26T20:27:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 client.26738 shutdown 2024-04-26T20:27:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 msgr2=0x55b47c64e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c64e610 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4888098f60 tx=0x7f48880107e0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 shutdown_connections 2024-04-26T20:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55b47c6c47a0 0x55b47c6c6b90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b47c645730 0x55b47c64fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f487be00640 0 client.26738 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:27:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b47c6405c0 0x55b47c64eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 --2- 172.21.15.131:0/3028664722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b47c63fbd0 0x55b47c64e610 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 >> 172.21.15.131:0/3028664722 conn(0x55b47c63b240 msgr2=0x55b47c6499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 shutdown_connections 2024-04-26T20:27:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.698+0000 7f48a0375740 1 -- 172.21.15.131:0/3028664722 wait complete. 2024-04-26T20:27:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f48a0375740 20 client.26738 trim_cache size 0 max 16384 2024-04-26T20:27:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:27:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:27:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:27:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:27:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f487c0a3d70 msgr2=0x7f487c0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f487c0a3d70 0x7f487c0a6230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f489002f540 tx=0x7f48900095c0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4898109930 msgr2=0x7f48981a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4898109930 0x7f48981a11c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f488800e9f0 tx=0x7f488800edf0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 shutdown_connections 2024-04-26T20:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f487c0a3d70 0x7f487c0a6230 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48981111b0 0x7f48981a8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489810a290 0x7f48981a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 --2- 172.21.15.131:0/1921905057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4898109930 0x7f48981a11c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.699+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 >> 172.21.15.131:0/1921905057 conn(0x7f489806f9a0 msgr2=0x7f489810cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.700+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 shutdown_connections 2024-04-26T20:27:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:26:51.700+0000 7f489d400640 1 -- 172.21.15.131:0/1921905057 wait complete. 2024-04-26T20:27:54.162 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:51 smithi131 ceph-mon[27919]: pgmap v915: 129 pgs: 129 active+clean; 217 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 356 KiB/s wr, 141 op/s 2024-04-26T20:27:54.171 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-04-26T20:27:54.180 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-26T20:27:54.188 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-04-26T20:27:54.224 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-26T20:27:54.262 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-04-26T20:27:54.271 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-26T20:27:54.279 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-04-26T20:27:54.286 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-26T20:27:54.293 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-04-26T20:27:54.300 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-26T20:27:54.363 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-04-26T20:27:54.412 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-26T20:27:54.419 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-04-26T20:27:54.426 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-26T20:27:54.434 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-04-26T20:27:54.528 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-26T20:27:54.535 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-04-26T20:27:54.543 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-26T20:27:54.550 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-04-26T20:27:54.557 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-26T20:27:54.616 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-04-26T20:27:54.623 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-26T20:27:54.684 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-04-26T20:27:54.691 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-26T20:27:54.699 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-04-26T20:27:54.706 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-26T20:27:54.714 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-04-26T20:27:54.753 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-26T20:27:54.816 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-04-26T20:27:54.823 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-26T20:27:54.831 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-04-26T20:27:54.839 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-26T20:27:54.912 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-04-26T20:27:54.920 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-26T20:27:54.927 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-04-26T20:27:54.934 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-26T20:27:54.941 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-04-26T20:27:54.978 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-26T20:27:55.028 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-04-26T20:27:55.035 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-26T20:27:55.043 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-04-26T20:27:55.049 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-26T20:27:55.057 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-04-26T20:27:55.064 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-26T20:27:55.124 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-04-26T20:27:55.170 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-26T20:27:55.178 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-04-26T20:27:55.185 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-26T20:27:55.193 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-04-26T20:27:55.254 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-26T20:27:55.262 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-04-26T20:27:55.269 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-26T20:27:55.275 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-04-26T20:27:55.282 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-26T20:27:55.321 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-04-26T20:27:55.375 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-26T20:27:55.383 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-04-26T20:27:55.391 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-26T20:27:55.400 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-04-26T20:27:55.409 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-26T20:27:55.418 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-04-26T20:27:55.482 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-26T20:27:55.528 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-04-26T20:27:55.535 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-26T20:27:55.543 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-04-26T20:27:55.553 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-26T20:27:55.596 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-04-26T20:27:55.606 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-26T20:27:55.615 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-04-26T20:27:55.625 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-26T20:27:55.635 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-04-26T20:27:55.682 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-26T20:27:55.739 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-04-26T20:27:55.746 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-26T20:27:55.753 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-04-26T20:27:55.761 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-26T20:27:55.769 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-04-26T20:27:55.776 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-26T20:27:55.827 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-04-26T20:27:55.880 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-26T20:27:55.889 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-04-26T20:27:55.898 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:53 smithi131 ceph-mon[27919]: pgmap v916: 129 pgs: 129 active+clean; 193 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 349 KiB/s wr, 145 op/s 2024-04-26T20:27:55.905 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-26T20:27:56.002 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-04-26T20:27:56.009 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-26T20:27:56.017 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-04-26T20:27:56.026 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-26T20:27:56.035 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-04-26T20:27:56.079 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-26T20:27:56.084 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-04-26T20:27:56.145 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-26T20:27:56.153 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-04-26T20:27:56.160 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-26T20:27:56.166 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-04-26T20:27:56.173 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-26T20:27:56.228 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-04-26T20:27:56.270 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-26T20:27:56.277 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-04-26T20:27:56.285 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-26T20:27:56.294 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-04-26T20:27:56.370 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-26T20:27:56.376 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-04-26T20:27:56.383 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-26T20:27:56.390 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-04-26T20:27:56.398 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-26T20:27:56.430 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-04-26T20:27:56.459 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-26T20:27:56.469 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-04-26T20:27:56.476 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-26T20:27:56.483 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-04-26T20:27:56.489 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-26T20:27:56.496 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-04-26T20:27:56.540 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-26T20:27:56.586 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-04-26T20:27:56.600 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-26T20:27:56.607 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-04-26T20:27:56.614 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-26T20:27:56.662 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-04-26T20:27:56.668 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-26T20:27:56.676 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-04-26T20:27:56.683 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-26T20:27:56.690 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-04-26T20:27:56.715 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-26T20:27:56.738 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-04-26T20:27:56.746 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-26T20:27:56.753 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-04-26T20:27:56.760 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-26T20:27:56.766 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-04-26T20:27:56.772 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-26T20:27:56.823 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-04-26T20:27:56.881 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-26T20:27:56.890 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-04-26T20:27:56.899 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-26T20:27:56.907 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-04-26T20:27:57.003 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-26T20:27:57.013 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-04-26T20:27:57.023 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-26T20:27:57.031 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-04-26T20:27:57.041 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-26T20:27:57.097 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-04-26T20:27:57.154 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-26T20:27:57.163 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-04-26T20:27:57.172 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-26T20:27:57.182 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-04-26T20:27:57.191 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-26T20:27:57.198 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-04-26T20:27:57.268 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-26T20:27:57.320 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-04-26T20:27:57.326 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-26T20:27:57.332 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-04-26T20:27:57.339 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-26T20:27:57.386 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-04-26T20:27:57.394 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-26T20:27:57.401 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-04-26T20:27:57.408 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-26T20:27:57.415 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-04-26T20:27:57.472 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-26T20:27:57.524 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-04-26T20:27:57.532 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-26T20:27:57.539 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-04-26T20:27:57.546 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-26T20:27:57.556 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-04-26T20:27:57.562 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-26T20:27:57.628 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-04-26T20:27:57.678 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-26T20:27:57.684 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-04-26T20:27:57.691 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-26T20:27:57.698 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-04-26T20:27:57.757 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-26T20:27:57.763 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-04-26T20:27:57.769 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-26T20:27:57.776 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-04-26T20:27:57.783 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-26T20:27:57.821 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-04-26T20:27:57.828 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-26T20:27:57.888 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-04-26T20:27:57.900 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-26T20:27:57.908 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-04-26T20:27:57.915 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-26T20:27:57.922 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-04-26T20:27:58.004 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-26T20:27:58.053 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-04-26T20:27:58.062 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-26T20:27:58.071 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-04-26T20:27:58.080 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-26T20:27:58.178 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-04-26T20:27:58.185 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-26T20:27:58.191 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-04-26T20:27:58.198 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-26T20:27:58.207 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-04-26T20:27:58.249 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-26T20:27:58.301 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-04-26T20:27:58.310 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-26T20:27:58.319 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-04-26T20:27:58.328 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-26T20:27:58.337 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-04-26T20:27:58.347 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-26T20:27:58.435 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-04-26T20:27:58.478 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-26T20:27:58.483 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-04-26T20:27:58.490 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-26T20:27:58.497 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-04-26T20:27:58.604 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-26T20:27:58.612 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-04-26T20:27:58.618 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-26T20:27:58.627 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-04-26T20:27:58.635 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-26T20:27:58.679 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-04-26T20:27:58.715 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-26T20:27:58.724 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-04-26T20:27:58.733 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-26T20:27:58.742 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-04-26T20:27:58.751 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-26T20:27:58.759 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-04-26T20:27:58.837 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-26T20:27:58.886 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-04-26T20:27:58.895 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-26T20:27:58.903 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-04-26T20:27:58.910 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-26T20:27:59.006 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-04-26T20:27:59.014 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-26T20:27:59.022 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-04-26T20:27:59.031 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-26T20:27:59.039 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-04-26T20:27:59.097 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-26T20:27:59.153 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-04-26T20:27:59.161 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-26T20:27:59.168 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-04-26T20:27:59.175 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-26T20:27:59.183 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-04-26T20:27:59.190 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-26T20:27:59.258 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-04-26T20:27:59.299 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-26T20:27:59.305 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-04-26T20:27:59.312 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-26T20:27:59.318 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-04-26T20:27:59.349 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-26T20:27:59.356 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-04-26T20:27:59.362 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-26T20:27:59.370 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-04-26T20:27:59.377 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-26T20:27:59.437 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-04-26T20:27:59.493 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-26T20:27:59.502 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:55 smithi131 ceph-mon[27919]: pgmap v917: 129 pgs: 129 active+clean; 146 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 497 KiB/s wr, 182 op/s 2024-04-26T20:27:59.510 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:49 smithi148 ceph-mon[32281]: pgmap v944: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T20:27:59.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:51 smithi148 ceph-mon[32281]: pgmap v945: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T20:27:59.525 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:53 smithi148 ceph-mon[32281]: pgmap v946: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T20:27:59.532 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:55 smithi148 ceph-mon[32281]: pgmap v947: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T20:27:59.593 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:57 smithi148 ceph-mon[32281]: pgmap v948: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T20:27:59.652 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:49 smithi157 ceph-mon[32703]: pgmap v944: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T20:27:59.660 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:51 smithi157 ceph-mon[32703]: pgmap v945: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T20:27:59.667 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:53 smithi157 ceph-mon[32703]: pgmap v946: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T20:27:59.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:55 smithi157 ceph-mon[32703]: pgmap v947: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T20:27:59.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:57 smithi157 ceph-mon[32703]: pgmap v948: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T20:27:59.747 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-04-26T20:27:59.753 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-26T20:27:59.761 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-04-26T20:27:59.770 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-26T20:27:59.834 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-04-26T20:27:59.892 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-26T20:27:59.901 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-04-26T20:27:59.909 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-26T20:27:59.918 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-04-26T20:27:59.927 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-26T20:27:59.937 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-04-26T20:28:00.006 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-26T20:28:00.055 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-04-26T20:28:00.065 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-26T20:28:00.075 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-04-26T20:28:00.084 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-26T20:28:00.198 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-04-26T20:28:00.207 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-26T20:28:00.216 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-04-26T20:28:00.225 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-26T20:28:00.234 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-04-26T20:28:00.292 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-26T20:28:00.301 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-04-26T20:28:00.355 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-26T20:28:00.364 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-04-26T20:28:00.373 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-26T20:28:00.381 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-04-26T20:28:00.390 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-26T20:28:00.448 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-04-26T20:28:00.488 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-26T20:28:00.497 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-04-26T20:28:00.506 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-26T20:28:00.515 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-04-26T20:28:00.577 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-26T20:28:00.586 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-04-26T20:28:00.595 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-26T20:28:00.603 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-04-26T20:28:00.612 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-26T20:28:00.662 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-04-26T20:28:00.716 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-26T20:28:00.725 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-04-26T20:28:00.733 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-26T20:28:00.742 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-04-26T20:28:00.750 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-26T20:28:00.759 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-04-26T20:28:00.825 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-26T20:28:00.878 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-04-26T20:28:00.887 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-26T20:28:00.895 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-04-26T20:28:00.904 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-26T20:28:00.968 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-04-26T20:28:00.975 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-26T20:28:00.983 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-04-26T20:28:00.992 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-26T20:28:01.000 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-04-26T20:28:01.045 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-26T20:28:01.096 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-04-26T20:28:01.103 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-26T20:28:01.109 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-04-26T20:28:01.115 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-26T20:28:01.123 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-04-26T20:28:01.132 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-26T20:28:01.204 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-04-26T20:28:01.243 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-26T20:28:01.251 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-04-26T20:28:01.259 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-26T20:28:01.266 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-04-26T20:28:01.358 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-26T20:28:01.366 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-04-26T20:28:01.373 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-26T20:28:01.381 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-04-26T20:28:01.387 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-26T20:28:01.445 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-04-26T20:28:01.453 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-26T20:28:01.504 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-04-26T20:28:01.512 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-26T20:28:01.520 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-04-26T20:28:01.528 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-26T20:28:01.535 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-04-26T20:28:01.599 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-26T20:28:01.637 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-04-26T20:28:01.646 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:57 smithi131 ceph-mon[27919]: pgmap v918: 129 pgs: 129 active+clean; 158 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 352 KiB/s wr, 151 op/s 2024-04-26T20:28:01.653 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:26:59 smithi131 ceph-mon[27919]: pgmap v919: 129 pgs: 129 active+clean; 124 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 508 KiB/s wr, 188 op/s 2024-04-26T20:28:01.661 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-26T20:28:01.780 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-04-26T20:28:01.787 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-26T20:28:01.795 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-04-26T20:28:01.802 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-26T20:28:01.809 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-04-26T20:28:01.861 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-26T20:28:01.916 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-04-26T20:28:01.923 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-26T20:28:01.931 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-04-26T20:28:01.938 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-26T20:28:01.945 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-04-26T20:28:01.953 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-26T20:28:02.017 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-04-26T20:28:02.054 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-26T20:28:02.062 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-04-26T20:28:02.069 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-26T20:28:02.076 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-04-26T20:28:02.160 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-26T20:28:02.168 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-04-26T20:28:02.175 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-26T20:28:02.182 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-04-26T20:28:02.189 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-26T20:28:02.245 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-04-26T20:28:02.298 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-26T20:28:02.304 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-04-26T20:28:02.311 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-26T20:28:02.318 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-04-26T20:28:02.327 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-26T20:28:02.334 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-04-26T20:28:02.421 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-26T20:28:02.463 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-04-26T20:28:02.485 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-26T20:28:02.494 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-04-26T20:28:02.503 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-26T20:28:02.559 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-04-26T20:28:02.565 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-26T20:28:02.574 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-04-26T20:28:02.580 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-26T20:28:02.589 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-04-26T20:28:02.666 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-26T20:28:02.728 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-04-26T20:28:02.734 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-26T20:28:02.740 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-04-26T20:28:02.748 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-26T20:28:02.756 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-04-26T20:28:02.765 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-26T20:28:02.827 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-04-26T20:28:02.868 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-26T20:28:02.874 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-04-26T20:28:02.883 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-26T20:28:02.891 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-04-26T20:28:02.951 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-26T20:28:02.960 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-04-26T20:28:02.969 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-26T20:28:02.976 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-04-26T20:28:02.985 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-26T20:28:03.028 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-04-26T20:28:03.081 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-26T20:28:03.089 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-04-26T20:28:03.096 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-26T20:28:03.103 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-04-26T20:28:03.110 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-26T20:28:03.117 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-04-26T20:28:03.171 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-26T20:28:03.210 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-04-26T20:28:03.217 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-26T20:28:03.226 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-04-26T20:28:03.233 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-26T20:28:03.342 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-04-26T20:28:03.348 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-26T20:28:03.355 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-04-26T20:28:03.362 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-26T20:28:03.368 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-04-26T20:28:03.395 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-26T20:28:03.446 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-04-26T20:28:03.453 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-26T20:28:03.462 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-04-26T20:28:03.471 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-26T20:28:03.478 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-04-26T20:28:03.486 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-26T20:28:03.571 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:00 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:28:03.622 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-04-26T20:28:03.630 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-26T20:28:03.637 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-04-26T20:28:03.644 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-26T20:28:03.709 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-04-26T20:28:03.715 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-26T20:28:03.721 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-04-26T20:28:03.728 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-26T20:28:03.735 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-04-26T20:28:03.763 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-26T20:28:03.818 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-04-26T20:28:03.826 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-26T20:28:03.835 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-04-26T20:28:03.844 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-26T20:28:03.852 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-04-26T20:28:03.861 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-26T20:28:03.938 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-04-26T20:28:04.011 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-26T20:28:04.022 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-04-26T20:28:04.031 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-26T20:28:04.037 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-04-26T20:28:04.116 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-26T20:28:04.125 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-04-26T20:28:04.135 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-26T20:28:04.144 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-04-26T20:28:04.153 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-26T20:28:04.222 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-04-26T20:28:04.283 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-26T20:28:04.293 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-04-26T20:28:04.302 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-26T20:28:04.310 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-04-26T20:28:04.319 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-26T20:28:04.328 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-04-26T20:28:04.415 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-26T20:28:04.470 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-04-26T20:28:04.479 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-26T20:28:04.488 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-04-26T20:28:04.496 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-26T20:28:04.577 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-04-26T20:28:04.583 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-26T20:28:04.591 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-04-26T20:28:04.598 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-26T20:28:04.606 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-04-26T20:28:04.613 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-26T20:28:04.670 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-04-26T20:28:04.685 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-26T20:28:04.704 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-04-26T20:28:04.712 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-26T20:28:04.719 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-04-26T20:28:04.726 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-26T20:28:04.787 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-04-26T20:28:04.828 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-26T20:28:04.836 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-04-26T20:28:04.843 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-26T20:28:04.851 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-04-26T20:28:04.951 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-26T20:28:04.958 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-04-26T20:28:04.964 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-26T20:28:04.970 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-04-26T20:28:04.977 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-26T20:28:05.029 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-04-26T20:28:05.066 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-26T20:28:05.074 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-04-26T20:28:05.083 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-26T20:28:05.090 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-04-26T20:28:05.096 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-26T20:28:05.104 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-04-26T20:28:05.178 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-26T20:28:05.220 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-04-26T20:28:05.229 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-26T20:28:05.236 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-04-26T20:28:05.243 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-26T20:28:05.297 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-04-26T20:28:05.304 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-26T20:28:05.313 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-04-26T20:28:05.321 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-26T20:28:05.328 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-04-26T20:28:05.388 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-26T20:28:05.447 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-04-26T20:28:05.456 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-26T20:28:05.464 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-04-26T20:28:05.473 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-26T20:28:05.482 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-04-26T20:28:05.488 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-26T20:28:05.566 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-04-26T20:28:05.603 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-26T20:28:05.615 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-04-26T20:28:05.624 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-26T20:28:05.631 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-04-26T20:28:05.692 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-26T20:28:05.701 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-04-26T20:28:05.710 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-26T20:28:05.719 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-04-26T20:28:05.727 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-26T20:28:05.785 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-04-26T20:28:05.792 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-26T20:28:05.853 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-04-26T20:28:05.862 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-26T20:28:05.870 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-04-26T20:28:05.881 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-26T20:28:05.888 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-04-26T20:28:05.953 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-26T20:28:05.995 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-04-26T20:28:06.002 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-26T20:28:06.011 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-04-26T20:28:06.020 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-26T20:28:06.100 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-04-26T20:28:06.106 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-26T20:28:06.113 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-04-26T20:28:06.120 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-26T20:28:06.127 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-04-26T20:28:06.169 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-26T20:28:06.224 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-04-26T20:28:06.231 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-26T20:28:06.237 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-04-26T20:28:06.244 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-26T20:28:06.251 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-04-26T20:28:06.260 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-26T20:28:06.340 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-04-26T20:28:06.392 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-26T20:28:06.399 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-04-26T20:28:06.406 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-26T20:28:06.413 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-04-26T20:28:06.457 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-26T20:28:06.464 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-04-26T20:28:06.471 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-26T20:28:06.478 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-04-26T20:28:06.486 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-26T20:28:06.535 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-04-26T20:28:06.588 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-26T20:28:06.595 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-04-26T20:28:06.601 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-26T20:28:06.609 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-04-26T20:28:06.616 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-26T20:28:06.626 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-04-26T20:28:06.673 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-26T20:28:06.719 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-04-26T20:28:06.726 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-26T20:28:06.733 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-04-26T20:28:06.739 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-26T20:28:06.796 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-04-26T20:28:06.804 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-26T20:28:06.813 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-04-26T20:28:06.822 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-26T20:28:06.831 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-04-26T20:28:06.877 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-26T20:28:06.935 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-04-26T20:28:06.941 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-26T20:28:06.950 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-04-26T20:28:06.959 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-26T20:28:06.968 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-04-26T20:28:06.977 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-26T20:28:07.037 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-04-26T20:28:07.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-26T20:28:07.087 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-04-26T20:28:07.096 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-26T20:28:07.104 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-04-26T20:28:07.212 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-26T20:28:07.219 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-04-26T20:28:07.227 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-26T20:28:07.235 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-04-26T20:28:07.245 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: pgmap v920: 129 pgs: 129 active+clean; 108 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 419 KiB/s wr, 164 op/s 2024-04-26T20:28:07.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.351 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.360 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.378 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.444 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:28:07.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:28:07.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:28:07.515 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:28:07.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:28:07.627 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:28:07.638 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:28:07.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:28:07.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:03 smithi131 ceph-mon[27919]: pgmap v921: 129 pgs: 129 active+clean; 108 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 428 KiB/s wr, 169 op/s 2024-04-26T20:28:07.658 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-26T20:28:07.745 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-04-26T20:28:07.752 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-26T20:28:07.780 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-04-26T20:28:07.787 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-26T20:28:07.794 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-04-26T20:28:07.802 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-26T20:28:07.808 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-04-26T20:28:07.879 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-26T20:28:07.936 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-04-26T20:28:07.942 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-26T20:28:07.949 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-04-26T20:28:07.957 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-26T20:28:08.022 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-04-26T20:28:08.028 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-26T20:28:08.034 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-04-26T20:28:08.041 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-26T20:28:08.049 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-04-26T20:28:08.091 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-26T20:28:08.143 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-04-26T20:28:08.149 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-26T20:28:08.155 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-04-26T20:28:08.161 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-26T20:28:08.169 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-04-26T20:28:08.175 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-26T20:28:08.238 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-04-26T20:28:08.286 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-26T20:28:08.295 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-04-26T20:28:08.305 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-26T20:28:08.314 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-04-26T20:28:08.323 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-26T20:28:08.336 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-04-26T20:28:08.411 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-26T20:28:08.420 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-04-26T20:28:08.429 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-26T20:28:08.493 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-04-26T20:28:08.548 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-26T20:28:08.556 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-04-26T20:28:08.565 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-26T20:28:08.573 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-04-26T20:28:08.581 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-26T20:28:08.590 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-04-26T20:28:08.635 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-26T20:28:08.686 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-04-26T20:28:08.692 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-26T20:28:08.699 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-04-26T20:28:08.706 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-26T20:28:08.774 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-04-26T20:28:08.781 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-26T20:28:08.788 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-04-26T20:28:08.795 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-26T20:28:08.802 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-04-26T20:28:08.858 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-26T20:28:08.914 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-04-26T20:28:08.921 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-26T20:28:08.929 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-04-26T20:28:08.936 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-26T20:28:08.943 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-04-26T20:28:08.951 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-26T20:28:09.012 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-04-26T20:28:09.061 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-26T20:28:09.067 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-04-26T20:28:09.074 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-26T20:28:09.081 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-04-26T20:28:09.088 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-26T20:28:09.170 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-04-26T20:28:09.176 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-26T20:28:09.185 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-04-26T20:28:09.192 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-26T20:28:09.241 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-04-26T20:28:09.246 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-26T20:28:09.302 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-04-26T20:28:09.310 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-26T20:28:09.317 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-04-26T20:28:09.325 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-26T20:28:09.330 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-04-26T20:28:09.437 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-26T20:28:09.520 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-04-26T20:28:09.527 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-26T20:28:09.535 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-04-26T20:28:09.544 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-26T20:28:09.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:27:59 smithi148 ceph-mon[32281]: pgmap v949: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T20:28:09.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:01 smithi148 ceph-mon[32281]: pgmap v950: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T20:28:09.665 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:01 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:28:09.674 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:28:09.683 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:28:09.742 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:28:09.774 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:03 smithi148 ceph-mon[32281]: pgmap v951: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:28:09.784 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:05 smithi148 ceph-mon[32281]: pgmap v952: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:28:09.792 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:07 smithi148 ceph-mon[32281]: pgmap v953: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:28:09.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:27:59 smithi157 ceph-mon[32703]: pgmap v949: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T20:28:09.809 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:01 smithi157 ceph-mon[32703]: pgmap v950: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T20:28:09.818 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:01 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:28:09.885 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:28:09.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:28:09.945 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:28:09.954 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:03 smithi157 ceph-mon[32703]: pgmap v951: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:28:09.962 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:05 smithi157 ceph-mon[32703]: pgmap v952: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:28:10.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:07 smithi157 ceph-mon[32703]: pgmap v953: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:28:10.033 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:09 smithi148 ceph-mon[32281]: pgmap v954: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T20:28:10.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:09 smithi157 ceph-mon[32703]: pgmap v954: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T20:28:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.849+0000 7f9f14c00640 1 -- 172.21.15.131:0/1717636272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 msgr2=0x7f9f10105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.849+0000 7f9f14c00640 1 --2- 172.21.15.131:0/1717636272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 0x7f9f10105800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009900 tx=0x7f9f0002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:28:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.849+0000 7f9f14c00640 1 -- 172.21.15.131:0/1717636272 shutdown_connections 2024-04-26T20:28:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.849+0000 7f9f14c00640 1 --2- 172.21.15.131:0/1717636272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10108670 0x7f9f1010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.850+0000 7f9f14c00640 1 --2- 172.21.15.131:0/1717636272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 0x7f9f10108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.850+0000 7f9f14c00640 1 --2- 172.21.15.131:0/1717636272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 0x7f9f10105800 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.850+0000 7f9f14c00640 1 -- 172.21.15.131:0/1717636272 >> 172.21.15.131:0/1717636272 conn(0x7f9f100fce00 msgr2=0x7f9f100ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.850+0000 7f9f14c00640 1 -- 172.21.15.131:0/1717636272 shutdown_connections 2024-04-26T20:28:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.850+0000 7f9f14c00640 1 -- 172.21.15.131:0/1717636272 wait complete. 2024-04-26T20:28:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.851+0000 7f9f14c00640 1 Processor -- start 2024-04-26T20:28:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.851+0000 7f9f14c00640 1 -- start start 2024-04-26T20:28:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f14c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 0x7f9f10198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f14c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 0x7f9f10199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f14c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10108670 0x7f9f101a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f14c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010e4e0 con 0x7f9f10103410 2024-04-26T20:28:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f14c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010e360 con 0x7f9f10105d40 2024-04-26T20:28:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f14c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010e660 con 0x7f9f10108670 2024-04-26T20:28:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 0x7f9f10198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 0x7f9f10199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10108670 0x7f9f101a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 0x7f9f10199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:53280/0 (socket says 172.21.15.131:53280) 2024-04-26T20:28:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 -- 172.21.15.131:0/445012491 learned_addr learned my addr 172.21.15.131:0/445012491 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 -- 172.21.15.131:0/445012491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10108670 msgr2=0x7f9f101a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10108670 0x7f9f101a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 -- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 msgr2=0x7f9f10198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 0x7f9f10198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.852+0000 7f9f0d400640 1 -- 172.21.15.131:0/445012491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f101a0750 con 0x7f9f10105d40 2024-04-26T20:28:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.853+0000 7f9f0d400640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 0x7f9f10199070 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9efc005fd0 tx=0x7f9efc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.853+0000 7f9f06a00640 1 -- 172.21.15.131:0/445012491 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc017070 con 0x7f9f10105d40 2024-04-26T20:28:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.853+0000 7f9f06a00640 1 -- 172.21.15.131:0/445012491 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc009c30 con 0x7f9f10105d40 2024-04-26T20:28:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.853+0000 7f9f06a00640 1 -- 172.21.15.131:0/445012491 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc003d30 con 0x7f9f10105d40 2024-04-26T20:28:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.853+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f101a0a40 con 0x7f9f10105d40 2024-04-26T20:28:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.853+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101a0f90 con 0x7f9f10105d40 2024-04-26T20:28:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.856+0000 7f9f06a00640 1 -- 172.21.15.131:0/445012491 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9efc003ed0 con 0x7f9f10105d40 2024-04-26T20:28:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 Processor -- start 2024-04-26T20:28:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 -- start start 2024-04-26T20:28:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558b2b020380 con 0x558b2b0c74b0 2024-04-26T20:28:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558b2afbcc30 con 0x558b2b0c4af0 2024-04-26T20:28:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f179d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558b2b01c340 con 0x558b2b0c9f90 2024-04-26T20:28:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.857+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39510/0 (socket says 172.21.15.131:39510) 2024-04-26T20:28:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0de00640 1 -- 172.21.15.131:0/2347758259 learned_addr learned my addr 172.21.15.131:0/2347758259 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0d400640 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0e800640 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0e800640 1 -- 172.21.15.131:0/2347758259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 msgr2=0x558b2b0cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0e800640 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0e800640 1 -- 172.21.15.131:0/2347758259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 msgr2=0x558b2b0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0e800640 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0e800640 1 -- 172.21.15.131:0/2347758259 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2afde940 con 0x558b2b0c4af0 2024-04-26T20:28:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f06a00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9ef40a3cf0 0x7f9ef40a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f0e800640 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0c6ee0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009eb0 tx=0x7f9f08010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c610100b78ee096 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.858+0000 7f9f06a00640 1 -- 172.21.15.131:0/445012491 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9efc0d0600 con 0x7f9f10105d40 2024-04-26T20:28:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.859+0000 7f9f0de00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9ef40a3cf0 0x7f9ef40a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.859+0000 7f9efb000640 1 -- 172.21.15.131:0/2347758259 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08016070 con 0x558b2b0c4af0 2024-04-26T20:28:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.859+0000 7f9efb000640 1 -- 172.21.15.131:0/2347758259 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004d20 con 0x558b2b0c4af0 2024-04-26T20:28:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.859+0000 7f9efb000640 1 -- 172.21.15.131:0/2347758259 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08012780 con 0x558b2b0c4af0 2024-04-26T20:28:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 -- 172.21.15.131:0/2347758259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 msgr2=0x558b2b0c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f0de00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9ef40a3cf0 0x7f9ef40a61b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9f000096c0 tx=0x7f9f000314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0c6ee0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009eb0 tx=0x7f9f08010740 comp rx=0 tx=0).stop 2024-04-26T20:28:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 -- 172.21.15.131:0/2347758259 shutdown_connections 2024-04-26T20:28:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2347758259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0c6ee0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 -- 172.21.15.131:0/2347758259 >> 172.21.15.131:0/2347758259 conn(0x558b2b0b71c0 msgr2=0x558b2b0b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 -- 172.21.15.131:0/2347758259 shutdown_connections 2024-04-26T20:28:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.860+0000 7f9f179d0740 1 -- 172.21.15.131:0/2347758259 wait complete. 2024-04-26T20:28:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 -- start start 2024-04-26T20:28:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 Processor -- start 2024-04-26T20:28:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558b2b0ccf40 con 0x558b2b0c74b0 2024-04-26T20:28:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558b2afa7c60 con 0x558b2b0c4af0 2024-04-26T20:28:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558b2b0cd0c0 con 0x558b2b0c9f90 2024-04-26T20:28:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.861+0000 7f9f179d0740 10 client.0 _ll_register_callbacks cb 0x558b2afd3e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39520/0 (socket says 172.21.15.131:39520) 2024-04-26T20:28:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0d400640 1 -- 172.21.15.131:0/1514671722 learned_addr learned my addr 172.21.15.131:0/1514671722 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0e800640 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0de00640 1 -- 172.21.15.131:0/1514671722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 msgr2=0x558b2b0d5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0de00640 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0de00640 1 -- 172.21.15.131:0/1514671722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 msgr2=0x558b2b0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0de00640 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0de00640 1 -- 172.21.15.131:0/1514671722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2afb6510 con 0x558b2b0c4af0 2024-04-26T20:28:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.862+0000 7f9f0d400640 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9f0de00640 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x558b2b0baf90 tx=0x7f9f00009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=343b7d3c8a48e1dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9eef400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9eef400640 1 -- 172.21.15.131:0/1514671722 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00035040 con 0x558b2b0c4af0 2024-04-26T20:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f08004d20 con 0x558b2b0c4af0 2024-04-26T20:28:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9eef400640 1 -- 172.21.15.131:0/1514671722 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9f000037f0 con 0x558b2b0c4af0 2024-04-26T20:28:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9eef400640 1 -- 172.21.15.131:0/1514671722 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f000039f0 con 0x558b2b0c4af0 2024-04-26T20:28:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9f179d0740 10 client.26750 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b2ad32210 con 0x558b2b0c4af0 2024-04-26T20:28:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.863+0000 7f9eef400640 1 -- 172.21.15.131:0/1514671722 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004d20 con 0x558b2b0c4af0 2024-04-26T20:28:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9eef400640 1 -- 172.21.15.131:0/1514671722 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9f00044070 con 0x558b2b0c4af0 2024-04-26T20:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9f179d0740 10 client.26750 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9f179d0740 10 client.26750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9f179d0740 20 client.26750 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9f179d0740 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b0da090 0x558b2b0dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9f179d0740 4 client.26750 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b2b02df90 con 0x558b2b0da090 2024-04-26T20:28:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.864+0000 7f9f0d400640 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b0da090 0x558b2b0dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.865+0000 7f9eef400640 10 client.26750 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.865+0000 7f9f0d400640 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b0da090 0x558b2b0dc480 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db31107df7a2afbf server_cookie=d2463b47b8a662fe in_seq=0 out_seq=0 2024-04-26T20:28:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9eef400640 1 -- 172.21.15.131:0/1514671722 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b2b02df90 con 0x558b2b0da090 2024-04-26T20:28:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9eef400640 10 client.26750 handle_command_reply: tid=0 2024-04-26T20:28:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 client.26750 shutdown 2024-04-26T20:28:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 msgr2=0x558b2b0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x558b2b0baf90 tx=0x7f9f00009260 comp rx=0 tx=0).stop 2024-04-26T20:28:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 shutdown_connections 2024-04-26T20:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b0da090 0x558b2b0dc480 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c74b0 0x558b2b0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 --2- 172.21.15.131:0/1514671722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce620 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9eef400640 0 client.26750 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 >> 172.21.15.131:0/1514671722 conn(0x558b2b0b71c0 msgr2=0x558b2b0c4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 shutdown_connections 2024-04-26T20:28:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 1 -- 172.21.15.131:0/1514671722 wait complete. 2024-04-26T20:28:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.866+0000 7f9f179d0740 20 client.26750 trim_cache size 0 max 16384 2024-04-26T20:28:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.885+0000 7f9f179d0740 1 Processor -- start 2024-04-26T20:28:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.885+0000 7f9f179d0740 1 -- start start 2024-04-26T20:28:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c9f90 0x558b2b0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f179d0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101abc90 con 0x558b2b0c9f90 2024-04-26T20:28:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f179d0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101a9560 con 0x558b2b0c74b0 2024-04-26T20:28:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f179d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101a73b0 con 0x558b2b0c4af0 2024-04-26T20:28:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f0e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c9f90 0x558b2b0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f0d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46536/0 (socket says 172.21.15.131:46536) 2024-04-26T20:28:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f0e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c9f90 0x558b2b0ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39536/0 (socket says 172.21.15.131:39536) 2024-04-26T20:28:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f0de00640 1 -- 172.21.15.131:0/2598148674 learned_addr learned my addr 172.21.15.131:0/2598148674 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.886+0000 7f9f0d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:53308/0 (socket says 172.21.15.131:53308) 2024-04-26T20:28:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9f0d400640 1 -- 172.21.15.131:0/2598148674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c4af0 msgr2=0x558b2b0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9f0d400640 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9f0d400640 1 -- 172.21.15.131:0/2598148674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c9f90 msgr2=0x558b2b0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9f0d400640 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c9f90 0x558b2b0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9f0d400640 1 -- 172.21.15.131:0/2598148674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2adfe620 con 0x558b2b0c74b0 2024-04-26T20:28:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9f0de00640 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9f0d400640 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d5b60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9efc098d00 tx=0x7f9efc0d6110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d74ece67907200e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9efa600640 1 -- 172.21.15.131:0/2598148674 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0cf040 con 0x558b2b0c74b0 2024-04-26T20:28:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9efa600640 1 -- 172.21.15.131:0/2598148674 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc09ca30 con 0x558b2b0c74b0 2024-04-26T20:28:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.887+0000 7f9efa600640 1 -- 172.21.15.131:0/2598148674 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc09a6c0 con 0x558b2b0c74b0 2024-04-26T20:28:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 -- 172.21.15.131:0/2598148674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 msgr2=0x558b2b0d5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d5b60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9efc098d00 tx=0x7f9efc0d6110 comp rx=0 tx=0).stop 2024-04-26T20:28:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 -- 172.21.15.131:0/2598148674 shutdown_connections 2024-04-26T20:28:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c9f90 0x558b2b0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d5b60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 --2- 172.21.15.131:0/2598148674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c4af0 0x558b2b0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 -- 172.21.15.131:0/2598148674 >> 172.21.15.131:0/2598148674 conn(0x558b2b0b71c0 msgr2=0x558b2b0c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 -- 172.21.15.131:0/2598148674 shutdown_connections 2024-04-26T20:28:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.888+0000 7f9f179d0740 1 -- 172.21.15.131:0/2598148674 wait complete. 2024-04-26T20:28:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 -- start start 2024-04-26T20:28:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 Processor -- start 2024-04-26T20:28:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c4af0 0x558b2b0d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101a7a20 con 0x558b2b0c4af0 2024-04-26T20:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101a8e90 con 0x558b2b0c74b0 2024-04-26T20:28:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.889+0000 7f9f179d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101a80f0 con 0x558b2b0c9f90 2024-04-26T20:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f179d0740 10 client.0 _ll_register_callbacks cb 0x7f9f00031670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c4af0 0x558b2b0d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46548/0 (socket says 172.21.15.131:46548) 2024-04-26T20:28:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c4af0 0x558b2b0d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39546/0 (socket says 172.21.15.131:39546) 2024-04-26T20:28:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0e800640 1 -- 172.21.15.131:0/847690011 learned_addr learned my addr 172.21.15.131:0/847690011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0d400640 1 -- 172.21.15.131:0/847690011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 msgr2=0x558b2b0d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0d400640 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0d400640 1 -- 172.21.15.131:0/847690011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c4af0 msgr2=0x558b2b0d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0d400640 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c4af0 0x558b2b0d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0d400640 1 -- 172.21.15.131:0/847690011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2afdacb0 con 0x558b2b0c74b0 2024-04-26T20:28:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9f0d400640 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d3800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x558b2b0cf790 tx=0x7f9efc0a06e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e773ad81195960d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.890+0000 7f9eef400640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:28:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9eef400640 1 -- 172.21.15.131:0/847690011 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0b4020 con 0x558b2b0c74b0 2024-04-26T20:28:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9eef400640 1 -- 172.21.15.131:0/847690011 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9efc0989a0 con 0x558b2b0c74b0 2024-04-26T20:28:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9efc09ca30 con 0x558b2b0c74b0 2024-04-26T20:28:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9eef400640 1 -- 172.21.15.131:0/847690011 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9efc0cf070 con 0x558b2b0c74b0 2024-04-26T20:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 10 client.26756 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b2b140600 con 0x558b2b0c74b0 2024-04-26T20:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9eef400640 1 -- 172.21.15.131:0/847690011 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc09ca30 con 0x558b2b0c74b0 2024-04-26T20:28:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9eef400640 1 -- 172.21.15.131:0/847690011 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9efc0cddb0 con 0x558b2b0c74b0 2024-04-26T20:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 10 client.26756 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 10 client.26756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 20 client.26756 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b140c20 0x558b2b143040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 4 client.26756 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:28:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.891+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b2b027e00 con 0x558b2b140c20 2024-04-26T20:28:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.892+0000 7f9f0de00640 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b140c20 0x558b2b143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.892+0000 7f9f0de00640 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b140c20 0x558b2b143040 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1158f1c987aacc1 server_cookie=904d6916d3298536 in_seq=0 out_seq=0 2024-04-26T20:28:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.892+0000 7f9eef400640 10 client.26756 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.972+0000 7f9eef400640 1 -- 172.21.15.131:0/847690011 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b2b027e00 con 0x558b2b140c20 2024-04-26T20:28:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.972+0000 7f9eef400640 10 client.26756 handle_command_reply: tid=0 2024-04-26T20:28:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.972+0000 7f9f179d0740 1 client.26756 shutdown 2024-04-26T20:28:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 msgr2=0x558b2b0d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d3800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x558b2b0cf790 tx=0x7f9efc0a06e0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 shutdown_connections 2024-04-26T20:28:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x558b2b140c20 0x558b2b143040 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b2b0c9f90 0x558b2b0d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b2b0c74b0 0x558b2b0d3800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 --2- 172.21.15.131:0/847690011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b2b0c4af0 0x558b2b0d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9eef400640 0 client.26756 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 >> 172.21.15.131:0/847690011 conn(0x558b2b0b71c0 msgr2=0x558b2b0ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 shutdown_connections 2024-04-26T20:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 1 -- 172.21.15.131:0/847690011 wait complete. 2024-04-26T20:28:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.974+0000 7f9f179d0740 20 client.26756 trim_cache size 0 max 16384 2024-04-26T20:28:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:28:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:28:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:28:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:28:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9ef40a3cf0 msgr2=0x7f9ef40a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9ef40a3cf0 0x7f9ef40a61b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9f000096c0 tx=0x7f9f000314a0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 msgr2=0x7f9f10199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 0x7f9f10199070 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9efc005fd0 tx=0x7f9efc00dde0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 shutdown_connections 2024-04-26T20:28:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9ef40a3cf0 0x7f9ef40a61b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10108670 0x7f9f101a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10105d40 0x7f9f10199070 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 --2- 172.21.15.131:0/445012491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f10103410 0x7f9f10198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:14.559 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 >> 172.21.15.131:0/445012491 conn(0x7f9f100fce00 msgr2=0x7f9f10103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 shutdown_connections 2024-04-26T20:28:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:04.975+0000 7f9f14c00640 1 -- 172.21.15.131:0/445012491 wait complete. 2024-04-26T20:28:14.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:05 smithi131 ceph-mon[27919]: pgmap v922: 129 pgs: 129 active+clean; 125 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 575 KiB/s wr, 198 op/s 2024-04-26T20:28:14.593 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-04-26T20:28:14.601 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-26T20:28:14.663 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-04-26T20:28:14.709 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-26T20:28:14.715 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-04-26T20:28:14.724 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-26T20:28:14.734 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-04-26T20:28:14.830 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-26T20:28:14.837 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-04-26T20:28:14.845 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-26T20:28:14.852 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-04-26T20:28:14.858 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-26T20:28:14.919 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-04-26T20:28:14.981 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-26T20:28:14.989 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-04-26T20:28:14.996 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-26T20:28:15.003 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-04-26T20:28:15.011 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-26T20:28:15.018 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-04-26T20:28:15.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-26T20:28:15.109 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-04-26T20:28:15.115 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-26T20:28:15.124 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-04-26T20:28:15.134 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-26T20:28:15.213 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-04-26T20:28:15.222 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-26T20:28:15.231 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-04-26T20:28:15.240 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-26T20:28:15.249 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-04-26T20:28:15.302 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-26T20:28:15.361 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-04-26T20:28:15.370 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-26T20:28:15.379 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-04-26T20:28:15.388 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-26T20:28:15.397 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-04-26T20:28:15.403 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-26T20:28:15.445 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-04-26T20:28:15.492 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-26T20:28:15.499 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-04-26T20:28:15.507 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-26T20:28:15.516 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-04-26T20:28:15.571 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-26T20:28:15.580 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-04-26T20:28:15.586 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-26T20:28:15.598 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-04-26T20:28:15.607 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-26T20:28:15.666 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-04-26T20:28:15.700 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-26T20:28:15.707 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-04-26T20:28:15.714 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-26T20:28:15.721 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-04-26T20:28:15.729 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-26T20:28:15.737 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-04-26T20:28:15.784 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-26T20:28:15.817 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-04-26T20:28:15.824 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-26T20:28:15.831 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-04-26T20:28:15.837 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-26T20:28:15.888 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-04-26T20:28:15.895 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-26T20:28:15.901 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-04-26T20:28:15.908 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-26T20:28:15.915 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-04-26T20:28:15.921 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-26T20:28:15.947 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-04-26T20:28:15.972 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-26T20:28:15.979 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-04-26T20:28:15.985 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-26T20:28:15.992 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-04-26T20:28:15.998 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-26T20:28:16.056 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-04-26T20:28:16.107 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-26T20:28:16.115 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-04-26T20:28:16.122 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-26T20:28:16.130 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-04-26T20:28:16.196 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-26T20:28:16.202 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-04-26T20:28:16.209 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-26T20:28:16.215 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-04-26T20:28:16.221 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-26T20:28:16.282 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-04-26T20:28:16.336 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-26T20:28:16.343 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-04-26T20:28:16.352 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-26T20:28:16.361 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-04-26T20:28:16.369 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-26T20:28:16.377 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-04-26T20:28:16.437 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-26T20:28:16.480 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-04-26T20:28:16.491 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-26T20:28:16.501 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-04-26T20:28:16.510 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-26T20:28:16.571 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-04-26T20:28:16.577 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-26T20:28:16.587 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-04-26T20:28:16.594 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-26T20:28:16.603 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-04-26T20:28:16.652 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-26T20:28:16.675 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-04-26T20:28:16.683 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-26T20:28:16.692 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-04-26T20:28:16.700 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-26T20:28:16.708 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-04-26T20:28:16.715 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-26T20:28:16.754 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-04-26T20:28:16.792 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-26T20:28:16.799 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-04-26T20:28:16.808 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-26T20:28:16.817 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-04-26T20:28:16.870 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-26T20:28:16.877 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-04-26T20:28:16.883 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-26T20:28:16.890 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-04-26T20:28:16.897 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-26T20:28:16.905 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-04-26T20:28:16.930 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-26T20:28:16.951 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-04-26T20:28:16.958 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-26T20:28:16.965 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-04-26T20:28:16.973 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-26T20:28:16.980 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-04-26T20:28:17.037 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-26T20:28:17.090 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-04-26T20:28:17.097 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-26T20:28:17.103 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-04-26T20:28:17.109 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-26T20:28:17.185 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-04-26T20:28:17.193 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-26T20:28:17.200 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-04-26T20:28:17.207 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-26T20:28:17.214 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-04-26T20:28:17.271 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-26T20:28:17.328 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-04-26T20:28:17.337 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-26T20:28:17.346 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-04-26T20:28:17.354 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-26T20:28:17.362 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-04-26T20:28:17.369 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-26T20:28:17.428 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-04-26T20:28:17.467 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-26T20:28:17.475 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-04-26T20:28:17.482 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-26T20:28:17.488 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-04-26T20:28:17.562 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-26T20:28:17.570 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-04-26T20:28:17.577 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-26T20:28:17.585 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-04-26T20:28:17.592 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-26T20:28:17.642 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-04-26T20:28:17.700 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-26T20:28:17.708 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-04-26T20:28:17.716 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-26T20:28:17.725 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-04-26T20:28:17.733 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-26T20:28:17.741 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-04-26T20:28:17.799 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-26T20:28:17.842 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-04-26T20:28:17.849 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-26T20:28:17.859 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-04-26T20:28:17.868 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-26T20:28:17.943 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-04-26T20:28:17.950 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-26T20:28:17.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:07 smithi131 ceph-mon[27919]: pgmap v923: 129 pgs: 129 active+clean; 118 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 474 KiB/s wr, 173 op/s 2024-04-26T20:28:17.966 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-04-26T20:28:17.974 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-26T20:28:18.013 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-04-26T20:28:18.071 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-26T20:28:18.078 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-04-26T20:28:18.086 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-26T20:28:18.094 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-04-26T20:28:18.103 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-26T20:28:18.112 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-04-26T20:28:18.179 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-26T20:28:18.217 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-04-26T20:28:18.226 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-26T20:28:18.235 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-04-26T20:28:18.244 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-26T20:28:18.302 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-04-26T20:28:18.311 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-26T20:28:18.320 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-04-26T20:28:18.328 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-26T20:28:18.336 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-04-26T20:28:18.418 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-26T20:28:18.477 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-04-26T20:28:18.484 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-26T20:28:18.492 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-04-26T20:28:18.499 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-26T20:28:18.507 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-04-26T20:28:18.515 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-26T20:28:18.574 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-04-26T20:28:18.633 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-26T20:28:18.640 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-04-26T20:28:18.647 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-26T20:28:18.655 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-04-26T20:28:18.721 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-26T20:28:18.730 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-04-26T20:28:18.737 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-26T20:28:18.744 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-04-26T20:28:18.752 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-26T20:28:18.788 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-04-26T20:28:18.796 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-26T20:28:18.850 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-04-26T20:28:18.859 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-26T20:28:18.866 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-04-26T20:28:18.874 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-26T20:28:18.882 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-04-26T20:28:18.936 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-26T20:28:18.983 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-04-26T20:28:18.989 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-26T20:28:18.996 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-04-26T20:28:19.003 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-26T20:28:19.060 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-04-26T20:28:19.068 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-26T20:28:19.075 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-04-26T20:28:19.082 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-26T20:28:19.089 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-04-26T20:28:19.177 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-26T20:28:19.185 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-04-26T20:28:19.238 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-26T20:28:19.249 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-04-26T20:28:19.258 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-26T20:28:19.271 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-04-26T20:28:19.279 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-26T20:28:19.347 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-04-26T20:28:19.427 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-26T20:28:19.432 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-04-26T20:28:19.439 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-26T20:28:19.448 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-04-26T20:28:19.530 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-26T20:28:19.540 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-04-26T20:28:19.549 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-26T20:28:19.558 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-04-26T20:28:19.567 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-26T20:28:19.593 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-04-26T20:28:19.642 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-26T20:28:19.651 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-04-26T20:28:19.660 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-26T20:28:19.668 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-04-26T20:28:19.678 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-26T20:28:19.687 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-04-26T20:28:19.762 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-26T20:28:19.799 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-04-26T20:28:19.809 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-26T20:28:19.818 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:09 smithi131 ceph-mon[27919]: pgmap v924: 129 pgs: 129 active+clean; 94 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 499 KiB/s wr, 209 op/s 2024-04-26T20:28:19.827 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-04-26T20:28:19.898 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-26T20:28:19.907 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-04-26T20:28:19.916 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-26T20:28:19.924 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-04-26T20:28:19.933 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-26T20:28:19.941 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-04-26T20:28:20.000 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-26T20:28:20.009 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-04-26T20:28:20.018 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-26T20:28:20.027 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-04-26T20:28:20.035 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-26T20:28:20.044 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-04-26T20:28:20.112 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-26T20:28:20.154 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-04-26T20:28:20.161 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-26T20:28:20.168 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-04-26T20:28:20.176 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-26T20:28:20.229 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-04-26T20:28:20.235 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-26T20:28:20.242 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-04-26T20:28:20.248 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-26T20:28:20.254 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-04-26T20:28:20.286 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-26T20:28:20.293 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-04-26T20:28:20.347 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-26T20:28:20.356 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-04-26T20:28:20.364 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-26T20:28:20.371 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-04-26T20:28:20.378 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-26T20:28:20.429 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-04-26T20:28:20.481 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-26T20:28:20.490 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-04-26T20:28:20.497 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-26T20:28:20.505 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-04-26T20:28:20.570 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-26T20:28:20.578 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-04-26T20:28:20.586 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-26T20:28:20.594 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-04-26T20:28:20.602 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-26T20:28:20.647 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-04-26T20:28:20.709 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-26T20:28:20.718 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-04-26T20:28:20.725 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-26T20:28:20.732 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-04-26T20:28:20.741 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-26T20:28:20.748 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-04-26T20:28:20.800 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-26T20:28:20.841 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-04-26T20:28:20.848 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-26T20:28:20.856 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-04-26T20:28:20.862 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-26T20:28:20.903 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-04-26T20:28:20.912 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-26T20:28:20.923 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-04-26T20:28:20.932 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-26T20:28:20.942 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-04-26T20:28:20.971 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-26T20:28:21.023 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-04-26T20:28:21.031 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-26T20:28:21.040 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-04-26T20:28:21.049 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-26T20:28:21.057 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-04-26T20:28:21.066 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-26T20:28:21.144 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-04-26T20:28:21.191 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-26T20:28:21.200 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-04-26T20:28:21.209 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-26T20:28:21.218 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-04-26T20:28:21.286 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-26T20:28:21.295 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-04-26T20:28:21.304 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-26T20:28:21.313 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-04-26T20:28:21.322 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-26T20:28:21.380 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-04-26T20:28:21.421 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-26T20:28:21.430 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-04-26T20:28:21.438 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-26T20:28:21.447 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-04-26T20:28:21.456 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-26T20:28:21.465 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-04-26T20:28:21.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:11 smithi131 ceph-mon[27919]: pgmap v925: 129 pgs: 129 active+clean; 114 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 345 KiB/s wr, 192 op/s 2024-04-26T20:28:21.563 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-26T20:28:21.572 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-04-26T20:28:21.581 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-26T20:28:21.589 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-04-26T20:28:21.636 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-26T20:28:21.645 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-04-26T20:28:21.653 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-26T20:28:21.663 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-04-26T20:28:21.671 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-26T20:28:21.733 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-04-26T20:28:21.784 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-26T20:28:21.791 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-04-26T20:28:21.800 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-26T20:28:21.807 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-04-26T20:28:21.813 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-26T20:28:21.820 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-04-26T20:28:21.884 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-26T20:28:21.932 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-04-26T20:28:21.942 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-26T20:28:21.949 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-04-26T20:28:21.956 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-26T20:28:22.024 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-04-26T20:28:22.031 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-26T20:28:22.039 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-04-26T20:28:22.046 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-26T20:28:22.053 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-04-26T20:28:22.120 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-26T20:28:22.172 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-04-26T20:28:22.181 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-26T20:28:22.197 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-04-26T20:28:22.205 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-26T20:28:22.212 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-04-26T20:28:22.220 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-26T20:28:22.287 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-04-26T20:28:22.329 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-26T20:28:22.334 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-04-26T20:28:22.340 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-26T20:28:22.347 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-04-26T20:28:22.398 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-26T20:28:22.405 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-04-26T20:28:22.413 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-26T20:28:22.421 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-04-26T20:28:22.428 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-26T20:28:22.461 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-04-26T20:28:22.468 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-26T20:28:22.523 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-04-26T20:28:22.529 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-26T20:28:22.535 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-04-26T20:28:22.541 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-26T20:28:22.548 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-04-26T20:28:22.592 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-26T20:28:22.640 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-04-26T20:28:22.649 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-26T20:28:22.658 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-04-26T20:28:22.667 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-26T20:28:22.711 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-04-26T20:28:22.718 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-26T20:28:22.724 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-04-26T20:28:22.733 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-26T20:28:22.742 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-04-26T20:28:22.802 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-26T20:28:22.858 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-04-26T20:28:22.867 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-26T20:28:22.876 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-04-26T20:28:22.882 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-26T20:28:22.889 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-04-26T20:28:22.898 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-26T20:28:22.972 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-04-26T20:28:23.007 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-26T20:28:23.028 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-04-26T20:28:23.036 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-26T20:28:23.045 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-04-26T20:28:23.121 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-26T20:28:23.131 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-04-26T20:28:23.140 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-26T20:28:23.148 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-04-26T20:28:23.156 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-26T20:28:23.262 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-04-26T20:28:23.313 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-26T20:28:23.323 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-04-26T20:28:23.331 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-26T20:28:23.340 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-04-26T20:28:23.349 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-26T20:28:23.357 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-04-26T20:28:23.449 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-26T20:28:23.515 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-04-26T20:28:23.522 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-26T20:28:23.528 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-04-26T20:28:23.534 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-26T20:28:23.594 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-04-26T20:28:23.602 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-26T20:28:23.608 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-04-26T20:28:23.615 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-26T20:28:23.624 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-04-26T20:28:23.666 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-26T20:28:23.673 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-04-26T20:28:23.736 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-26T20:28:23.743 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-04-26T20:28:23.750 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-26T20:28:23.758 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-04-26T20:28:23.768 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-26T20:28:23.833 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-04-26T20:28:23.873 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-26T20:28:23.881 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-04-26T20:28:23.888 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-26T20:28:23.895 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-04-26T20:28:23.960 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-26T20:28:23.967 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-04-26T20:28:23.975 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-26T20:28:23.983 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-04-26T20:28:23.994 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-26T20:28:24.051 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-04-26T20:28:24.107 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-26T20:28:24.114 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-04-26T20:28:24.122 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-26T20:28:24.132 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-04-26T20:28:24.141 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-26T20:28:24.149 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-04-26T20:28:24.207 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-26T20:28:24.256 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-04-26T20:28:24.264 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-26T20:28:24.271 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-04-26T20:28:24.278 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-26T20:28:24.335 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-04-26T20:28:24.345 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-26T20:28:24.354 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-04-26T20:28:24.363 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-26T20:28:24.372 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-04-26T20:28:24.433 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-26T20:28:24.489 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-04-26T20:28:24.498 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-26T20:28:24.507 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-04-26T20:28:24.516 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-26T20:28:24.525 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-04-26T20:28:24.534 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-26T20:28:24.581 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-04-26T20:28:24.620 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-26T20:28:24.629 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-04-26T20:28:24.640 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-26T20:28:24.649 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-04-26T20:28:24.744 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-26T20:28:24.753 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-04-26T20:28:24.762 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-26T20:28:24.768 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-04-26T20:28:24.777 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-26T20:28:24.830 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-04-26T20:28:24.837 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-26T20:28:24.893 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-04-26T20:28:24.902 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-26T20:28:24.910 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-04-26T20:28:24.919 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-26T20:28:24.926 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-04-26T20:28:24.967 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-26T20:28:25.006 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-04-26T20:28:25.015 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-26T20:28:25.024 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-04-26T20:28:25.033 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-26T20:28:25.110 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-04-26T20:28:25.118 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-26T20:28:25.124 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-04-26T20:28:25.130 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-26T20:28:25.138 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-04-26T20:28:25.183 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:13 smithi131 ceph-mon[27919]: pgmap v926: 129 pgs: 129 active+clean; 139 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 478 KiB/s wr, 217 op/s 2024-04-26T20:28:25.233 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-26T20:28:25.249 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-04-26T20:28:25.255 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-26T20:28:25.263 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-04-26T20:28:25.269 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-26T20:28:25.278 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-04-26T20:28:25.345 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-26T20:28:25.386 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-04-26T20:28:25.394 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-26T20:28:25.400 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-04-26T20:28:25.407 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-26T20:28:25.471 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-04-26T20:28:25.478 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-26T20:28:25.487 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-04-26T20:28:25.494 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-26T20:28:25.502 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-04-26T20:28:25.568 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-26T20:28:25.575 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-04-26T20:28:25.631 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-26T20:28:25.638 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-04-26T20:28:25.645 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-26T20:28:25.653 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-04-26T20:28:25.660 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-26T20:28:25.723 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-04-26T20:28:25.781 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-26T20:28:25.789 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-04-26T20:28:25.796 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-26T20:28:25.803 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-04-26T20:28:25.861 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-26T20:28:25.870 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-04-26T20:28:25.879 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-26T20:28:25.887 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-04-26T20:28:25.896 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-26T20:28:25.932 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-04-26T20:28:25.939 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-26T20:28:26.004 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-04-26T20:28:26.013 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-26T20:28:26.022 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-04-26T20:28:26.031 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-26T20:28:26.040 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-04-26T20:28:26.111 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-26T20:28:26.148 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-04-26T20:28:26.154 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-26T20:28:26.163 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-04-26T20:28:26.172 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-26T20:28:26.239 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-04-26T20:28:26.249 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-26T20:28:26.258 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-04-26T20:28:26.265 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-26T20:28:26.275 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-04-26T20:28:26.355 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-26T20:28:26.398 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-04-26T20:28:26.406 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-26T20:28:26.412 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-04-26T20:28:26.420 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-26T20:28:26.429 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-04-26T20:28:26.438 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-26T20:28:26.493 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-04-26T20:28:26.540 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-26T20:28:26.548 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-04-26T20:28:26.557 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-26T20:28:26.565 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-04-26T20:28:26.619 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-26T20:28:26.629 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-04-26T20:28:26.637 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-26T20:28:26.646 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-04-26T20:28:26.654 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-26T20:28:26.702 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-04-26T20:28:26.762 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-26T20:28:26.769 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-04-26T20:28:26.777 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-26T20:28:26.785 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-04-26T20:28:26.793 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-26T20:28:26.799 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-04-26T20:28:26.874 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-26T20:28:26.920 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-04-26T20:28:26.928 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-26T20:28:26.935 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-04-26T20:28:26.943 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-26T20:28:26.985 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-04-26T20:28:26.991 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-26T20:28:26.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:15 smithi131 ceph-mon[27919]: pgmap v927: 129 pgs: 129 active+clean; 143 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 471 KiB/s wr, 219 op/s 2024-04-26T20:28:27.005 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:17 smithi131 ceph-mon[27919]: pgmap v928: 129 pgs: 129 active+clean; 120 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 458 KiB/s wr, 211 op/s 2024-04-26T20:28:27.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:11 smithi157 ceph-mon[32703]: pgmap v955: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:28:27.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:13 smithi157 ceph-mon[32703]: pgmap v956: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:28:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:15 smithi157 ceph-mon[32703]: pgmap v957: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:27.140 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:17 smithi157 ceph-mon[32703]: pgmap v958: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:27.146 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:19 smithi157 ceph-mon[32703]: pgmap v959: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:27.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:21 smithi157 ceph-mon[32703]: pgmap v960: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:28:27.160 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:23 smithi157 ceph-mon[32703]: pgmap v961: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:27.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:25 smithi157 ceph-mon[32703]: pgmap v962: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:27.214 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:11 smithi148 ceph-mon[32281]: pgmap v955: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:28:27.256 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:13 smithi148 ceph-mon[32281]: pgmap v956: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:28:27.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:15 smithi148 ceph-mon[32281]: pgmap v957: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:27.270 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:17 smithi148 ceph-mon[32281]: pgmap v958: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:27.277 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:19 smithi148 ceph-mon[32281]: pgmap v959: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:27.327 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:21 smithi148 ceph-mon[32281]: pgmap v960: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:28:27.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:23 smithi148 ceph-mon[32281]: pgmap v961: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:27.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:25 smithi148 ceph-mon[32281]: pgmap v962: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 -- 172.21.15.131:0/777282868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c1027d0 msgr2=0x7f8c3c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 --2- 172.21.15.131:0/777282868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c1027d0 0x7f8c3c102bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009900 tx=0x7f8c2c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:28:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 -- 172.21.15.131:0/777282868 shutdown_connections 2024-04-26T20:28:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 --2- 172.21.15.131:0/777282868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c3c104380 0x7f8c3c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 --2- 172.21.15.131:0/777282868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 0x7f8c3c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 --2- 172.21.15.131:0/777282868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c1027d0 0x7f8c3c102bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 -- 172.21.15.131:0/777282868 >> 172.21.15.131:0/777282868 conn(0x7f8c3c0fdd40 msgr2=0x7f8c3c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 -- 172.21.15.131:0/777282868 shutdown_connections 2024-04-26T20:28:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.122+0000 7f8c40c00640 1 -- 172.21.15.131:0/777282868 wait complete. 2024-04-26T20:28:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.123+0000 7f8c40c00640 1 Processor -- start 2024-04-26T20:28:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.123+0000 7f8c40c00640 1 -- start start 2024-04-26T20:28:27.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c40c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c3c1027d0 0x7f8c3c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c40c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 0x7f8c3c19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c40c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c104380 0x7f8c3c1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c40c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c10e340 con 0x7f8c3c1027d0 2024-04-26T20:28:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c40c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c10e1c0 con 0x7f8c3c1039c0 2024-04-26T20:28:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c40c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c10e4c0 con 0x7f8c3c104380 2024-04-26T20:28:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 0x7f8c3c19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 0x7f8c3c19b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:48482/0 (socket says 172.21.15.131:48482) 2024-04-26T20:28:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 -- 172.21.15.131:0/2482951003 learned_addr learned my addr 172.21.15.131:0/2482951003 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c3be00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c104380 0x7f8c3c1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c3b400640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c3c1027d0 0x7f8c3c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 -- 172.21.15.131:0/2482951003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c104380 msgr2=0x7f8c3c1a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c104380 0x7f8c3c1a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 -- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c3c1027d0 msgr2=0x7f8c3c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c3c1027d0 0x7f8c3c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.124+0000 7f8c33e00640 1 -- 172.21.15.131:0/2482951003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3c1a2900 con 0x7f8c3c1039c0 2024-04-26T20:28:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.125+0000 7f8c33e00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 0x7f8c3c19b200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d770 tx=0x7f8c2400db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.125+0000 7f8c38c00640 1 -- 172.21.15.131:0/2482951003 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24017070 con 0x7f8c3c1039c0 2024-04-26T20:28:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.125+0000 7f8c38c00640 1 -- 172.21.15.131:0/2482951003 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c24009820 con 0x7f8c3c1039c0 2024-04-26T20:28:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.125+0000 7f8c38c00640 1 -- 172.21.15.131:0/2482951003 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24004bc0 con 0x7f8c3c1039c0 2024-04-26T20:28:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.125+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3c1a2bf0 con 0x7f8c3c1039c0 2024-04-26T20:28:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.125+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3c1a30c0 con 0x7f8c3c1039c0 2024-04-26T20:28:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.128+0000 7f8c4504b740 1 Processor -- start 2024-04-26T20:28:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c4504b740 1 -- start start 2024-04-26T20:28:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda4d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda54dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c4504b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a2fd9b1380 con 0x55a2fda4d7d0 2024-04-26T20:28:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c4504b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2fd94dc30 con 0x55a2fda52940 2024-04-26T20:28:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c4504b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a2fd9ad340 con 0x55a2fda4cde0 2024-04-26T20:28:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda4d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda4d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46460/0 (socket says 172.21.15.131:46460) 2024-04-26T20:28:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c3be00640 1 -- 172.21.15.131:0/4065531875 learned_addr learned my addr 172.21.15.131:0/4065531875 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.129+0000 7f8c3b400640 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c38c00640 1 -- 172.21.15.131:0/2482951003 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8c240099d0 con 0x7f8c3c1039c0 2024-04-26T20:28:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c3be00640 1 -- 172.21.15.131:0/4065531875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 msgr2=0x55a2fda54dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:28:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c3be00640 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda54dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c3be00640 1 -- 172.21.15.131:0/4065531875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 msgr2=0x55a2fda522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c3be00640 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c3be00640 1 -- 172.21.15.131:0/4065531875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2fd96f940 con 0x55a2fda4cde0 2024-04-26T20:28:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c3b400640 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:28:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c3be00640 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda4d200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8c34011440 tx=0x7f8c34011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6373b60e759db9f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c2be00640 1 -- 172.21.15.131:0/4065531875 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34003c30 con 0x55a2fda4cde0 2024-04-26T20:28:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c2be00640 1 -- 172.21.15.131:0/4065531875 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c34003dd0 con 0x55a2fda4cde0 2024-04-26T20:28:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.130+0000 7f8c2be00640 1 -- 172.21.15.131:0/4065531875 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400fa80 con 0x55a2fda4cde0 2024-04-26T20:28:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.131+0000 7f8c4504b740 1 -- 172.21.15.131:0/4065531875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 msgr2=0x55a2fda4d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.131+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda4d200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8c34011440 tx=0x7f8c34011900 comp rx=0 tx=0).stop 2024-04-26T20:28:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.131+0000 7f8c4504b740 1 -- 172.21.15.131:0/4065531875 shutdown_connections 2024-04-26T20:28:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.131+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda54dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.131+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.131+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4065531875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda4d200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.131+0000 7f8c4504b740 1 -- 172.21.15.131:0/4065531875 >> 172.21.15.131:0/4065531875 conn(0x55a2fda48450 msgr2=0x55a2fda48850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.132+0000 7f8c4504b740 1 -- 172.21.15.131:0/4065531875 shutdown_connections 2024-04-26T20:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.132+0000 7f8c4504b740 1 -- 172.21.15.131:0/4065531875 wait complete. 2024-04-26T20:28:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.132+0000 7f8c4504b740 1 -- start start 2024-04-26T20:28:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.132+0000 7f8c38c00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8c200a3b40 0x7f8c200a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.132+0000 7f8c4504b740 1 Processor -- start 2024-04-26T20:28:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.132+0000 7f8c38c00640 1 -- 172.21.15.131:0/2482951003 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8c240d0390 con 0x7f8c3c1039c0 2024-04-26T20:28:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c3be00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8c200a3b40 0x7f8c200a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda57050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda57590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c4504b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a2fda558f0 con 0x55a2fda4d7d0 2024-04-26T20:28:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c4504b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2fd938c60 con 0x55a2fda52940 2024-04-26T20:28:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c4504b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a2fda55a70 con 0x55a2fda4cde0 2024-04-26T20:28:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c4504b740 10 client.0 _ll_register_callbacks cb 0x55a2fd964e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda57590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda57590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51604/0 (socket says 172.21.15.131:51604) 2024-04-26T20:28:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c33e00640 1 -- 172.21.15.131:0/4229410461 learned_addr learned my addr 172.21.15.131:0/4229410461 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.133+0000 7f8c3b400640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda57050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c3be00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8c200a3b40 0x7f8c200a6000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8c3401b040 tx=0x7f8c34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c3be00640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c33e00640 1 -- 172.21.15.131:0/4229410461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 msgr2=0x55a2fda57050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c33e00640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda57050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c33e00640 1 -- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 msgr2=0x55a2fda5e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c33e00640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda5e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c33e00640 1 -- 172.21.15.131:0/4229410461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2fd947510 con 0x55a2fda4d7d0 2024-04-26T20:28:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c3be00640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda5e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:28:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c33e00640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda57590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c2409c0b0 tx=0x7f8c2409c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143e47c9b8c0470 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c28c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:28:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c28c00640 1 -- 172.21.15.131:0/4229410461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24020040 con 0x55a2fda4d7d0 2024-04-26T20:28:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c28c00640 1 -- 172.21.15.131:0/4229410461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8c240b5580 con 0x55a2fda4d7d0 2024-04-26T20:28:29.470 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-04-26T20:28:29.540 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-26T20:28:29.577 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-04-26T20:28:29.583 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-26T20:28:29.592 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-04-26T20:28:29.601 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-26T20:28:29.672 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-04-26T20:28:29.680 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-26T20:28:29.689 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-04-26T20:28:29.698 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-26T20:28:29.705 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-04-26T20:28:29.748 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-26T20:28:29.795 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-04-26T20:28:29.802 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-26T20:28:29.810 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-04-26T20:28:29.818 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-26T20:28:29.825 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-04-26T20:28:29.833 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-26T20:28:29.891 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-04-26T20:28:29.923 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-26T20:28:29.931 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-04-26T20:28:29.940 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-26T20:28:29.948 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-04-26T20:28:30.010 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-26T20:28:30.017 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-04-26T20:28:30.025 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-26T20:28:30.032 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-04-26T20:28:30.039 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-26T20:28:30.080 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-04-26T20:28:30.133 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-26T20:28:30.140 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-04-26T20:28:30.148 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-26T20:28:30.155 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-04-26T20:28:30.162 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-26T20:28:30.169 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-04-26T20:28:30.215 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-26T20:28:30.247 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-04-26T20:28:30.256 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-26T20:28:30.269 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-04-26T20:28:30.278 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-26T20:28:30.316 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-04-26T20:28:30.324 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-26T20:28:30.330 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-04-26T20:28:30.336 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-26T20:28:30.342 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-04-26T20:28:30.389 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-26T20:28:30.447 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-04-26T20:28:30.457 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-26T20:28:30.465 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-04-26T20:28:30.474 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-26T20:28:30.483 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-04-26T20:28:30.492 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-26T20:28:30.566 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-04-26T20:28:30.597 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-26T20:28:30.605 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-04-26T20:28:30.614 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-26T20:28:30.623 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-04-26T20:28:30.683 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-26T20:28:30.690 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-04-26T20:28:30.699 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-26T20:28:30.708 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-04-26T20:28:30.717 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-26T20:28:30.811 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-04-26T20:28:30.860 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-26T20:28:30.870 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-04-26T20:28:30.879 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-26T20:28:30.887 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-04-26T20:28:30.896 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-26T20:28:30.904 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-04-26T20:28:30.961 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-26T20:28:31.010 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-04-26T20:28:31.018 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-26T20:28:31.027 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-04-26T20:28:31.036 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-26T20:28:31.133 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-04-26T20:28:31.142 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-26T20:28:31.151 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-04-26T20:28:31.160 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-26T20:28:31.169 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-04-26T20:28:31.214 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-26T20:28:31.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:27 smithi157 ceph-mon[32703]: pgmap v963: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:31.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:29 smithi157 ceph-mon[32703]: pgmap v964: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:31.283 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:27 smithi148 ceph-mon[32281]: pgmap v963: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:31.291 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:29 smithi148 ceph-mon[32281]: pgmap v964: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.134+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c34003dd0 con 0x55a2fda4d7d0 2024-04-26T20:28:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c28c00640 1 -- 172.21.15.131:0/4229410461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c240ba050 con 0x55a2fda4d7d0 2024-04-26T20:28:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 10 client.16644 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2fd6c3210 con 0x55a2fda4d7d0 2024-04-26T20:28:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c28c00640 1 -- 172.21.15.131:0/4229410461 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c34003dd0 con 0x55a2fda4d7d0 2024-04-26T20:28:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c28c00640 1 -- 172.21.15.131:0/4229410461 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8c240b5250 con 0x55a2fda4d7d0 2024-04-26T20:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 10 client.16644 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 20 client.16644 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fda62af0 0x55a2fda64ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 4 client.16644 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.135+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2fd9bef90 con 0x55a2fda62af0 2024-04-26T20:28:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.136+0000 7f8c3b400640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fda62af0 0x55a2fda64ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.136+0000 7f8c3b400640 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fda62af0 0x55a2fda64ee0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7825a7ecffd294fe server_cookie=9264fd4a7ed0b8be in_seq=0 out_seq=0 2024-04-26T20:28:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.136+0000 7f8c28c00640 10 client.16644 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c28c00640 1 -- 172.21.15.131:0/4229410461 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2fd9bef90 con 0x55a2fda62af0 2024-04-26T20:28:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c28c00640 10 client.16644 handle_command_reply: tid=0 2024-04-26T20:28:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 client.16644 shutdown 2024-04-26T20:28:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 msgr2=0x55a2fda57590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda57590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c2409c0b0 tx=0x7f8c2409c1e0 comp rx=0 tx=0).stop 2024-04-26T20:28:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 shutdown_connections 2024-04-26T20:28:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fda62af0 0x55a2fda64ee0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda52940 0x55a2fda5e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda4d7d0 0x55a2fda57590 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c28c00640 0 client.16644 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 --2- 172.21.15.131:0/4229410461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda57050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 >> 172.21.15.131:0/4229410461 conn(0x55a2fda48450 msgr2=0x55a2fda47650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 shutdown_connections 2024-04-26T20:28:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.137+0000 7f8c4504b740 1 -- 172.21.15.131:0/4229410461 wait complete. 2024-04-26T20:28:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.138+0000 7f8c4504b740 20 client.16644 trim_cache size 0 max 16384 2024-04-26T20:28:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.158+0000 7f8c4504b740 1 Processor -- start 2024-04-26T20:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.158+0000 7f8c4504b740 1 -- start start 2024-04-26T20:28:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda56520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 0x55a2fda5e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda46dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c4504b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1b0050 con 0x55a2fda52940 2024-04-26T20:28:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c4504b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1ad920 con 0x55a2fda4d7d0 2024-04-26T20:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c4504b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1ab770 con 0x55a2fda4cde0 2024-04-26T20:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda56520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda56520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46476/0 (socket says 172.21.15.131:46476) 2024-04-26T20:28:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 -- 172.21.15.131:0/2268876014 learned_addr learned my addr 172.21.15.131:0/2268876014 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3be00640 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda46dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 -- 172.21.15.131:0/2268876014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 msgr2=0x55a2fda5e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 0x55a2fda5e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 -- 172.21.15.131:0/2268876014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 msgr2=0x55a2fda46dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda46dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3b400640 1 -- 172.21.15.131:0/2268876014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2fd78f620 con 0x55a2fda4cde0 2024-04-26T20:28:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.159+0000 7f8c3be00640 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda46dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:28:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.160+0000 7f8c3b400640 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda56520 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c103820 tx=0x7f8c2c004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea58d1f1311ec52e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.160+0000 7f8c2b400640 1 -- 172.21.15.131:0/2268876014 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c035040 con 0x55a2fda4cde0 2024-04-26T20:28:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.160+0000 7f8c2b400640 1 -- 172.21.15.131:0/2268876014 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c2c0044c0 con 0x55a2fda4cde0 2024-04-26T20:28:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.160+0000 7f8c2b400640 1 -- 172.21.15.131:0/2268876014 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0315a0 con 0x55a2fda4cde0 2024-04-26T20:28:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 -- 172.21.15.131:0/2268876014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 msgr2=0x55a2fda56520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda56520 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c103820 tx=0x7f8c2c004290 comp rx=0 tx=0).stop 2024-04-26T20:28:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 -- 172.21.15.131:0/2268876014 shutdown_connections 2024-04-26T20:28:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda46dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 0x55a2fda5e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 --2- 172.21.15.131:0/2268876014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda56520 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 -- 172.21.15.131:0/2268876014 >> 172.21.15.131:0/2268876014 conn(0x55a2fda48450 msgr2=0x55a2fda55e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 -- 172.21.15.131:0/2268876014 shutdown_connections 2024-04-26T20:28:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 -- 172.21.15.131:0/2268876014 wait complete. 2024-04-26T20:28:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.161+0000 7f8c4504b740 1 -- start start 2024-04-26T20:28:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 1 Processor -- start 2024-04-26T20:28:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 0x55a2fda5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda5cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1abde0 con 0x55a2fda52940 2024-04-26T20:28:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1ad250 con 0x55a2fda4d7d0 2024-04-26T20:28:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1ac4b0 con 0x55a2fda4cde0 2024-04-26T20:28:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c4504b740 10 client.0 _ll_register_callbacks cb 0x7f8c240983c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c3b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c3b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46482/0 (socket says 172.21.15.131:46482) 2024-04-26T20:28:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c3b400640 1 -- 172.21.15.131:0/883072045 learned_addr learned my addr 172.21.15.131:0/883072045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.162+0000 7f8c33e00640 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 0x55a2fda5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c3be00640 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda5cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c3b400640 1 -- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 msgr2=0x55a2fda5bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c3b400640 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 0x55a2fda5bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c3b400640 1 -- 172.21.15.131:0/883072045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 msgr2=0x55a2fda5cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c3b400640 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda5cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c3b400640 1 -- 172.21.15.131:0/883072045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2fd96bcb0 con 0x55a2fda4cde0 2024-04-26T20:28:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c3b400640 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda5b830 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c02a910 tx=0x7f8c2c003e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d46ec352161c2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c28c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:28:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.163+0000 7f8c28c00640 1 -- 172.21.15.131:0/883072045 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c052070 con 0x55a2fda4cde0 2024-04-26T20:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.164+0000 7f8c28c00640 1 -- 172.21.15.131:0/883072045 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f8c2c0453c0 con 0x55a2fda4cde0 2024-04-26T20:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.164+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2fd9b8e00 con 0x55a2fda4cde0 2024-04-26T20:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.164+0000 7f8c28c00640 1 -- 172.21.15.131:0/883072045 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c05e050 con 0x55a2fda4cde0 2024-04-26T20:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.164+0000 7f8c4504b740 10 client.26749 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.164+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2fda5e4b0 con 0x55a2fda4cde0 2024-04-26T20:28:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.164+0000 7f8c28c00640 1 -- 172.21.15.131:0/883072045 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a2fd9b8e00 con 0x55a2fda4cde0 2024-04-26T20:28:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.164+0000 7f8c28c00640 1 -- 172.21.15.131:0/883072045 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8c2c031ca0 con 0x55a2fda4cde0 2024-04-26T20:28:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.165+0000 7f8c4504b740 10 client.26749 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.165+0000 7f8c4504b740 10 client.26749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.165+0000 7f8c4504b740 20 client.26749 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.165+0000 7f8c4504b740 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fdac9290 0x55a2fdacb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.165+0000 7f8c4504b740 4 client.26749 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:28:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.165+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2fd9344b0 con 0x55a2fdac9290 2024-04-26T20:28:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.165+0000 7f8c33e00640 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fdac9290 0x55a2fdacb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.166+0000 7f8c33e00640 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fdac9290 0x55a2fdacb6b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d1def6e9c53c98 server_cookie=b32aef25c2cc57f5 in_seq=0 out_seq=0 2024-04-26T20:28:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.166+0000 7f8c28c00640 10 client.26749 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c28c00640 1 -- 172.21.15.131:0/883072045 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2fd9344b0 con 0x55a2fdac9290 2024-04-26T20:28:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c28c00640 10 client.26749 handle_command_reply: tid=0 2024-04-26T20:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 client.26749 shutdown 2024-04-26T20:28:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 msgr2=0x55a2fda5b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda5b830 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c02a910 tx=0x7f8c2c003e20 comp rx=0 tx=0).stop 2024-04-26T20:28:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 shutdown_connections 2024-04-26T20:28:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a2fdac9290 0x55a2fdacb6b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2fda52940 0x55a2fda5cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c28c00640 0 client.26749 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2fda4d7d0 0x55a2fda5bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 --2- 172.21.15.131:0/883072045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2fda4cde0 0x55a2fda5b830 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 >> 172.21.15.131:0/883072045 conn(0x55a2fda48450 msgr2=0x55a2fda566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 shutdown_connections 2024-04-26T20:28:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 1 -- 172.21.15.131:0/883072045 wait complete. 2024-04-26T20:28:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.193+0000 7f8c4504b740 20 client.26749 trim_cache size 0 max 16384 2024-04-26T20:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:28:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:28:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:28:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8c200a3b40 msgr2=0x7f8c200a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8c200a3b40 0x7f8c200a6000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8c3401b040 tx=0x7f8c34005fd0 comp rx=0 tx=0).stop 2024-04-26T20:28:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 msgr2=0x7f8c3c19b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 0x7f8c3c19b200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d770 tx=0x7f8c2400db70 comp rx=0 tx=0).stop 2024-04-26T20:28:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 shutdown_connections 2024-04-26T20:28:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f8c200a3b40 0x7f8c200a6000 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c3c104380 0x7f8c3c1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3c1039c0 0x7f8c3c19b200 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 --2- 172.21.15.131:0/2482951003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c3c1027d0 0x7f8c3c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 >> 172.21.15.131:0/2482951003 conn(0x7f8c3c0fdd40 msgr2=0x7f8c3c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 shutdown_connections 2024-04-26T20:28:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:18.194+0000 7f8c40c00640 1 -- 172.21.15.131:0/2482951003 wait complete. 2024-04-26T20:28:34.150 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-04-26T20:28:34.194 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-26T20:28:34.249 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-04-26T20:28:34.258 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-26T20:28:34.267 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-04-26T20:28:34.275 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-26T20:28:34.284 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-04-26T20:28:34.293 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-26T20:28:34.360 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-04-26T20:28:34.407 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-26T20:28:34.417 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-04-26T20:28:34.427 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-26T20:28:34.436 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-04-26T20:28:34.498 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-26T20:28:34.507 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-04-26T20:28:34.515 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-26T20:28:34.523 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-04-26T20:28:34.532 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-26T20:28:34.580 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-04-26T20:28:34.638 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-26T20:28:34.647 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-04-26T20:28:34.654 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-26T20:28:34.662 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-04-26T20:28:34.668 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-26T20:28:34.676 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-04-26T20:28:34.739 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-26T20:28:34.784 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-04-26T20:28:34.792 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-26T20:28:34.803 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-04-26T20:28:34.810 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-26T20:28:34.856 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-04-26T20:28:34.865 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-26T20:28:34.872 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-04-26T20:28:34.879 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-26T20:28:34.886 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-04-26T20:28:34.948 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-26T20:28:35.011 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-04-26T20:28:35.018 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-26T20:28:35.026 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-04-26T20:28:35.033 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-26T20:28:35.040 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-04-26T20:28:35.047 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-26T20:28:35.089 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-04-26T20:28:35.129 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-26T20:28:35.142 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-04-26T20:28:35.149 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-26T20:28:35.156 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-04-26T20:28:35.213 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-26T20:28:35.222 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-04-26T20:28:35.229 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-26T20:28:35.236 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-04-26T20:28:35.242 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-26T20:28:35.298 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-04-26T20:28:35.358 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-26T20:28:35.366 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-04-26T20:28:35.373 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-26T20:28:35.380 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-04-26T20:28:35.387 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-26T20:28:35.394 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-04-26T20:28:35.441 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-26T20:28:35.479 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-04-26T20:28:35.486 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-26T20:28:35.494 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-04-26T20:28:35.501 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-26T20:28:35.579 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-04-26T20:28:35.588 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-26T20:28:35.595 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-04-26T20:28:35.602 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-26T20:28:35.610 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-04-26T20:28:35.687 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-26T20:28:35.746 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-04-26T20:28:35.753 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-26T20:28:35.762 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-04-26T20:28:35.772 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-26T20:28:35.784 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-04-26T20:28:35.795 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-26T20:28:35.856 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-04-26T20:28:35.896 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-26T20:28:35.901 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-04-26T20:28:35.912 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-26T20:28:35.921 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-04-26T20:28:35.981 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-26T20:28:35.990 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-04-26T20:28:35.999 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-26T20:28:36.008 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-04-26T20:28:36.017 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-26T20:28:36.061 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-04-26T20:28:36.112 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-26T20:28:36.121 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-04-26T20:28:36.130 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-26T20:28:36.139 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-04-26T20:28:36.148 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-26T20:28:36.159 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-04-26T20:28:36.254 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-26T20:28:36.304 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-04-26T20:28:36.311 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-26T20:28:36.320 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-04-26T20:28:36.328 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-26T20:28:36.337 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-04-26T20:28:36.346 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-26T20:28:36.389 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-04-26T20:28:36.397 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-26T20:28:36.405 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-04-26T20:28:36.485 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-26T20:28:36.548 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-04-26T20:28:36.557 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-26T20:28:36.565 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-04-26T20:28:36.573 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-26T20:28:36.581 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-04-26T20:28:36.589 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-26T20:28:36.640 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-04-26T20:28:36.694 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-26T20:28:36.702 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-04-26T20:28:36.709 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-26T20:28:36.716 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-04-26T20:28:36.840 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-26T20:28:36.847 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-04-26T20:28:36.855 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-26T20:28:36.862 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-04-26T20:28:36.870 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-26T20:28:36.904 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-04-26T20:28:36.961 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-26T20:28:36.969 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-04-26T20:28:36.976 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-26T20:28:36.983 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-04-26T20:28:36.990 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-26T20:28:36.998 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-04-26T20:28:37.051 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-26T20:28:37.100 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-04-26T20:28:37.108 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-26T20:28:37.116 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-04-26T20:28:37.123 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-26T20:28:37.213 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-04-26T20:28:37.220 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-26T20:28:37.227 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-04-26T20:28:37.233 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-26T20:28:37.241 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-04-26T20:28:37.286 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-26T20:28:37.341 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-04-26T20:28:37.351 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-26T20:28:37.359 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-04-26T20:28:37.368 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-26T20:28:37.377 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-04-26T20:28:37.385 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-26T20:28:37.438 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-04-26T20:28:37.488 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-26T20:28:37.497 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-04-26T20:28:37.506 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-26T20:28:37.514 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-04-26T20:28:37.569 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-26T20:28:37.578 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-04-26T20:28:37.588 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-26T20:28:37.597 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-04-26T20:28:37.606 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-26T20:28:37.663 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-04-26T20:28:37.714 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-26T20:28:37.723 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-04-26T20:28:37.731 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-26T20:28:37.740 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-04-26T20:28:37.749 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-26T20:28:37.758 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-04-26T20:28:37.804 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-26T20:28:37.845 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-04-26T20:28:37.852 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-26T20:28:37.868 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-04-26T20:28:37.877 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-26T20:28:37.886 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-04-26T20:28:37.894 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-26T20:28:37.947 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-04-26T20:28:37.954 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-26T20:28:37.962 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-04-26T20:28:38.010 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-26T20:28:38.069 INFO:tasks.workunit.client.0.smithi131.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-04-26T20:28:38.075 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-26T20:28:38.082 INFO:tasks.workunit.client.0.smithi131.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-04-26T20:28:38.088 INFO:tasks.workunit.client.0.smithi131.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-26T20:28:38.096 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T20:28:38.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T20:28:38.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:19 smithi131 ceph-mon[27919]: pgmap v929: 129 pgs: 129 active+clean; 148 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 437 KiB/s wr, 205 op/s 2024-04-26T20:28:38.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:21 smithi131 ceph-mon[27919]: pgmap v930: 129 pgs: 129 active+clean; 148 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 407 KiB/s wr, 167 op/s 2024-04-26T20:28:38.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:23 smithi131 ceph-mon[27919]: pgmap v931: 129 pgs: 129 active+clean; 81 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 533 KiB/s wr, 174 op/s 2024-04-26T20:28:38.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:25 smithi131 ceph-mon[27919]: pgmap v932: 129 pgs: 129 active+clean; 49 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 411 KiB/s wr, 157 op/s 2024-04-26T20:28:38.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:27 smithi131 ceph-mon[27919]: pgmap v933: 129 pgs: 129 active+clean; 21 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 390 KiB/s wr, 143 op/s 2024-04-26T20:28:38.297 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:29 smithi131 ceph-mon[27919]: pgmap v934: 129 pgs: 129 active+clean; 5.0 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 236 KiB/s wr, 108 op/s 2024-04-26T20:28:38.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:31 smithi157 ceph-mon[32703]: pgmap v965: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:28:38.321 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:33 smithi157 ceph-mon[32703]: pgmap v966: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:38.328 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:35 smithi157 ceph-mon[32703]: pgmap v967: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:38.335 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:37 smithi157 ceph-mon[32703]: pgmap v968: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:28:38.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:31 smithi148 ceph-mon[32281]: pgmap v965: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:28:38.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:33 smithi148 ceph-mon[32281]: pgmap v966: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:38.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:35 smithi148 ceph-mon[32281]: pgmap v967: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:38.446 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:37 smithi148 ceph-mon[32281]: pgmap v968: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:28:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 -- 172.21.15.131:0/2262520789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54105db0 msgr2=0x7f9f54106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/2262520789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54105db0 0x7f9f54106230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009900 tx=0x7f9f3c02a410 comp rx=0 tx=0).stop 2024-04-26T20:28:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 -- 172.21.15.131:0/2262520789 shutdown_connections 2024-04-26T20:28:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/2262520789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f54106770 0x7f9f5410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/2262520789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54105db0 0x7f9f54106230 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/2262520789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54104bc0 0x7f9f54104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 -- 172.21.15.131:0/2262520789 >> 172.21.15.131:0/2262520789 conn(0x7f9f54100150 msgr2=0x7f9f54102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 -- 172.21.15.131:0/2262520789 shutdown_connections 2024-04-26T20:28:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.337+0000 7f9f5ac00640 1 -- 172.21.15.131:0/2262520789 wait complete. 2024-04-26T20:28:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.338+0000 7f9f5ac00640 1 Processor -- start 2024-04-26T20:28:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.338+0000 7f9f5ac00640 1 -- start start 2024-04-26T20:28:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f5ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f54104bc0 0x7f9f5419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f5ac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54105db0 0x7f9f5419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f5ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106770 0x7f9f541a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f5ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54110740 con 0x7f9f54106770 2024-04-26T20:28:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f5ac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541105c0 con 0x7f9f54105db0 2024-04-26T20:28:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f5ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541108c0 con 0x7f9f54104bc0 2024-04-26T20:28:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106770 0x7f9f541a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f54104bc0 0x7f9f5419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106770 0x7f9f541a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48590/0 (socket says 172.21.15.131:48590) 2024-04-26T20:28:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f53400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54105db0 0x7f9f5419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 -- 172.21.15.131:0/1027549079 learned_addr learned my addr 172.21.15.131:0/1027549079 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 -- 172.21.15.131:0/1027549079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f54104bc0 msgr2=0x7f9f5419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f54104bc0 0x7f9f5419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 -- 172.21.15.131:0/1027549079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54105db0 msgr2=0x7f9f5419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54105db0 0x7f9f5419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f58c00640 1 -- 172.21.15.131:0/1027549079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f541a2920 con 0x7f9f54106770 2024-04-26T20:28:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.339+0000 7f9f53e00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f54104bc0 0x7f9f5419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.340+0000 7f9f58c00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106770 0x7f9f541a2280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400e970 tx=0x7f9f4400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.340+0000 7f9f50c00640 1 -- 172.21.15.131:0/1027549079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400ccc0 con 0x7f9f54106770 2024-04-26T20:28:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.340+0000 7f9f50c00640 1 -- 172.21.15.131:0/1027549079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4400ce60 con 0x7f9f54106770 2024-04-26T20:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.340+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f541a2c10 con 0x7f9f54106770 2024-04-26T20:28:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.340+0000 7f9f50c00640 1 -- 172.21.15.131:0/1027549079 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44010430 con 0x7f9f54106770 2024-04-26T20:28:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.340+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f541a31b0 con 0x7f9f54106770 2024-04-26T20:28:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.343+0000 7f9f5da08740 1 Processor -- start 2024-04-26T20:28:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.343+0000 7f9f50c00640 1 -- 172.21.15.131:0/1027549079 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9f44004f90 con 0x7f9f54106770 2024-04-26T20:28:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f5da08740 1 -- start start 2024-04-26T20:28:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c90bd0 0x559913c90ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913c960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913c98bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f5da08740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559913bf5380 con 0x559913c90bd0 2024-04-26T20:28:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f5da08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559913b91c30 con 0x559913c96730 2024-04-26T20:28:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f5da08740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559913bf1340 con 0x559913c915c0 2024-04-26T20:28:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c90bd0 0x559913c90ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c90bd0 0x559913c90ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48606/0 (socket says 172.21.15.131:48606) 2024-04-26T20:28:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f53400640 1 -- 172.21.15.131:0/2841703352 learned_addr learned my addr 172.21.15.131:0/2841703352 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.344+0000 7f9f53e00640 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913c960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53400640 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913c98bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53e00640 1 -- 172.21.15.131:0/2841703352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 msgr2=0x559913c98bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53e00640 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913c98bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53e00640 1 -- 172.21.15.131:0/2841703352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c90bd0 msgr2=0x559913c90ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53e00640 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c90bd0 0x559913c90ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53e00640 1 -- 172.21.15.131:0/2841703352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559913bb3940 con 0x559913c915c0 2024-04-26T20:28:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53400640 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913c98bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:28:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f50c00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9f380a3d70 0x7f9f380a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53e00640 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913c960c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9f54105c10 tx=0x7f9f4c011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78fef2656fbdc64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f50c00640 1 -- 172.21.15.131:0/1027549079 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9f440d0d70 con 0x7f9f54106770 2024-04-26T20:28:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f53400640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9f380a3d70 0x7f9f380a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.345+0000 7f9f49200640 1 -- 172.21.15.131:0/2841703352 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c011e60 con 0x559913c915c0 2024-04-26T20:28:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f49200640 1 -- 172.21.15.131:0/2841703352 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c004d20 con 0x559913c915c0 2024-04-26T20:28:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f49200640 1 -- 172.21.15.131:0/2841703352 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c005960 con 0x559913c915c0 2024-04-26T20:28:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f5da08740 1 -- 172.21.15.131:0/2841703352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 msgr2=0x559913c960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f53400640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9f380a3d70 0x7f9f380a6230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x559913c91e20 tx=0x7f9f3c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f5da08740 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913c960c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9f54105c10 tx=0x7f9f4c011c70 comp rx=0 tx=0).stop 2024-04-26T20:28:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f5da08740 1 -- 172.21.15.131:0/2841703352 shutdown_connections 2024-04-26T20:28:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f5da08740 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913c98bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f5da08740 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913c960c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f5da08740 1 --2- 172.21.15.131:0/2841703352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c90bd0 0x559913c90ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.346+0000 7f9f5da08740 1 -- 172.21.15.131:0/2841703352 >> 172.21.15.131:0/2841703352 conn(0x559913c8c240 msgr2=0x559913c8c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.347+0000 7f9f5da08740 1 -- 172.21.15.131:0/2841703352 shutdown_connections 2024-04-26T20:28:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.347+0000 7f9f5da08740 1 -- 172.21.15.131:0/2841703352 wait complete. 2024-04-26T20:28:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.347+0000 7f9f5da08740 1 -- start start 2024-04-26T20:28:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.347+0000 7f9f5da08740 1 Processor -- start 2024-04-26T20:28:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c90bd0 0x559913c9ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913ca2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f5da08740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559913c996e0 con 0x559913c915c0 2024-04-26T20:28:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f5da08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559913b7cc60 con 0x559913c96730 2024-04-26T20:28:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f5da08740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559913c99860 con 0x559913c90bd0 2024-04-26T20:28:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f5da08740 10 client.0 _ll_register_callbacks cb 0x559913ba8e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c90bd0 0x559913c9ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c90bd0 0x559913c9ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38972/0 (socket says 172.21.15.131:38972) 2024-04-26T20:28:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 -- 172.21.15.131:0/1152309851 learned_addr learned my addr 172.21.15.131:0/1152309851 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53400640 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f58c00640 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913ca2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 -- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 msgr2=0x559913ca2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913ca2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 -- 172.21.15.131:0/1152309851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 msgr2=0x559913c9b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.348+0000 7f9f53e00640 1 -- 172.21.15.131:0/1152309851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559913b8b510 con 0x559913c90bd0 2024-04-26T20:28:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f53e00640 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c90bd0 0x559913c9ae40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c01b040 tx=0x7f9f4c004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31ca5f78735af8b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f41600640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:28:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f41600640 1 -- 172.21.15.131:0/1152309851 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c029020 con 0x559913c90bd0 2024-04-26T20:28:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c004d20 con 0x559913c90bd0 2024-04-26T20:28:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f41600640 1 -- 172.21.15.131:0/1152309851 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9f4c011040 con 0x559913c90bd0 2024-04-26T20:28:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f41600640 1 -- 172.21.15.131:0/1152309851 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c034050 con 0x559913c90bd0 2024-04-26T20:28:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f5da08740 10 client.26761 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.349+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559913907210 con 0x559913c90bd0 2024-04-26T20:28:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f41600640 1 -- 172.21.15.131:0/1152309851 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c004d20 con 0x559913c90bd0 2024-04-26T20:28:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f41600640 1 -- 172.21.15.131:0/1152309851 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9f4c025880 con 0x559913c90bd0 2024-04-26T20:28:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f5da08740 10 client.26761 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f5da08740 10 client.26761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f5da08740 20 client.26761 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f5da08740 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913ca6680 0x559913ca8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f5da08740 4 client.26761 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.350+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559913c02f90 con 0x559913ca6680 2024-04-26T20:28:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.351+0000 7f9f53400640 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913ca6680 0x559913ca8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.351+0000 7f9f53400640 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913ca6680 0x559913ca8a70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa71cd0aa0fb6b9 server_cookie=367c3f4c544f2ccd in_seq=0 out_seq=0 2024-04-26T20:28:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.351+0000 7f9f41600640 10 client.26761 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f41600640 1 -- 172.21.15.131:0/1152309851 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559913c02f90 con 0x559913ca6680 2024-04-26T20:28:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f41600640 10 client.26761 handle_command_reply: tid=0 2024-04-26T20:28:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 client.26761 shutdown 2024-04-26T20:28:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c90bd0 msgr2=0x559913c9ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c90bd0 0x559913c9ae40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c01b040 tx=0x7f9f4c004c70 comp rx=0 tx=0).stop 2024-04-26T20:28:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 shutdown_connections 2024-04-26T20:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913ca6680 0x559913ca8a70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c96730 0x559913ca2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 --2- 172.21.15.131:0/1152309851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c90bd0 0x559913c9ae40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f41600640 0 client.26761 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 >> 172.21.15.131:0/1152309851 conn(0x559913c8c240 msgr2=0x559913c8b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 shutdown_connections 2024-04-26T20:28:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 1 -- 172.21.15.131:0/1152309851 wait complete. 2024-04-26T20:28:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.352+0000 7f9f5da08740 20 client.26761 trim_cache size 0 max 16384 2024-04-26T20:28:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 Processor -- start 2024-04-26T20:28:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 -- start start 2024-04-26T20:28:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913c8f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913ca2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c96730 0x559913c8f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b0080 con 0x559913c96730 2024-04-26T20:28:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541ad950 con 0x559913c90bd0 2024-04-26T20:28:41.172 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.369+0000 7f9f5da08740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541ab7a0 con 0x559913c915c0 2024-04-26T20:28:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c96730 0x559913c8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913ca2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913ca2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38974/0 (socket says 172.21.15.131:38974) 2024-04-26T20:28:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 -- 172.21.15.131:0/977833329 learned_addr learned my addr 172.21.15.131:0/977833329 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913c8f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 -- 172.21.15.131:0/977833329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 msgr2=0x559913c8f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 --2- 172.21.15.131:0/977833329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913c8f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 -- 172.21.15.131:0/977833329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c96730 msgr2=0x559913c8f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 --2- 172.21.15.131:0/977833329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c96730 0x559913c8f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 -- 172.21.15.131:0/977833329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599139d3620 con 0x559913c915c0 2024-04-26T20:28:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f53400640 1 --2- 172.21.15.131:0/977833329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913ca2270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005ab0 tx=0x7f9f3c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570b96079720948d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.370+0000 7f9f43e00640 1 -- 172.21.15.131:0/977833329 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c05c070 con 0x559913c915c0 2024-04-26T20:28:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f43e00640 1 -- 172.21.15.131:0/977833329 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c003dc0 con 0x559913c915c0 2024-04-26T20:28:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f43e00640 1 -- 172.21.15.131:0/977833329 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c035050 con 0x559913c915c0 2024-04-26T20:28:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 -- 172.21.15.131:0/977833329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 msgr2=0x559913ca2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 --2- 172.21.15.131:0/977833329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913ca2270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005ab0 tx=0x7f9f3c009260 comp rx=0 tx=0).stop 2024-04-26T20:28:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 -- 172.21.15.131:0/977833329 shutdown_connections 2024-04-26T20:28:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 --2- 172.21.15.131:0/977833329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c96730 0x559913c8f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 --2- 172.21.15.131:0/977833329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c915c0 0x559913ca2270 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 --2- 172.21.15.131:0/977833329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913c8f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 -- 172.21.15.131:0/977833329 >> 172.21.15.131:0/977833329 conn(0x559913c8c240 msgr2=0x559913c8c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 -- 172.21.15.131:0/977833329 shutdown_connections 2024-04-26T20:28:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.371+0000 7f9f5da08740 1 -- 172.21.15.131:0/977833329 wait complete. 2024-04-26T20:28:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 -- start start 2024-04-26T20:28:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 Processor -- start 2024-04-26T20:28:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913ca20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c96730 0x559913c9f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541abe10 con 0x559913c915c0 2024-04-26T20:28:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541ad280 con 0x559913c90bd0 2024-04-26T20:28:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.372+0000 7f9f5da08740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541ac4e0 con 0x559913c96730 2024-04-26T20:28:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f5da08740 10 client.0 _ll_register_callbacks cb 0x7f9f44005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913ca20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c96730 0x559913c9f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913ca20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:53042/0 (socket says 172.21.15.131:53042) 2024-04-26T20:28:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 -- 172.21.15.131:0/597015704 learned_addr learned my addr 172.21.15.131:0/597015704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48648/0 (socket says 172.21.15.131:48648) 2024-04-26T20:28:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 -- 172.21.15.131:0/597015704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c96730 msgr2=0x559913c9f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c96730 0x559913c9f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 -- 172.21.15.131:0/597015704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 msgr2=0x559913c9ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 -- 172.21.15.131:0/597015704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559913bafcb0 con 0x559913c90bd0 2024-04-26T20:28:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f53e00640 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913ca20c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c006010 tx=0x7f9f4c026f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a55c01e1e3ae7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.373+0000 7f9f41600640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:28:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.374+0000 7f9f41600640 1 -- 172.21.15.131:0/597015704 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c03d060 con 0x559913c90bd0 2024-04-26T20:28:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.374+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3c003dc0 con 0x559913c90bd0 2024-04-26T20:28:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.374+0000 7f9f41600640 1 -- 172.21.15.131:0/597015704 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f9f4c004ce0 con 0x559913c90bd0 2024-04-26T20:28:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.374+0000 7f9f41600640 1 -- 172.21.15.131:0/597015704 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c044050 con 0x559913c90bd0 2024-04-26T20:28:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.374+0000 7f9f5da08740 10 client.26786 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.374+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559913d0baa0 con 0x559913c90bd0 2024-04-26T20:28:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.374+0000 7f9f41600640 1 -- 172.21.15.131:0/597015704 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c003dc0 con 0x559913c90bd0 2024-04-26T20:28:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f41600640 1 -- 172.21.15.131:0/597015704 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9f4c025470 con 0x559913c90bd0 2024-04-26T20:28:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f5da08740 10 client.26786 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f5da08740 10 client.26786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f5da08740 20 client.26786 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f5da08740 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913d0c390 0x559913d0e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f5da08740 4 client.26786 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:28:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559913bfce00 con 0x559913d0c390 2024-04-26T20:28:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.375+0000 7f9f53400640 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913d0c390 0x559913d0e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.376+0000 7f9f53400640 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913d0c390 0x559913d0e7b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c019808a94295d12 server_cookie=cd3537230c9c3c19 in_seq=0 out_seq=0 2024-04-26T20:28:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.376+0000 7f9f41600640 10 client.26786 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f41600640 1 -- 172.21.15.131:0/597015704 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559913bfce00 con 0x559913d0c390 2024-04-26T20:28:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f41600640 10 client.26786 handle_command_reply: tid=0 2024-04-26T20:28:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 client.26786 shutdown 2024-04-26T20:28:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 msgr2=0x559913ca20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913ca20c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c006010 tx=0x7f9f4c026f60 comp rx=0 tx=0).stop 2024-04-26T20:28:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 shutdown_connections 2024-04-26T20:28:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559913d0c390 0x559913d0e7b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559913c96730 0x559913c9f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559913c915c0 0x559913c9ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 --2- 172.21.15.131:0/597015704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559913c90bd0 0x559913ca20c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f41600640 0 client.26786 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 >> 172.21.15.131:0/597015704 conn(0x559913c8c240 msgr2=0x559913c8f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 shutdown_connections 2024-04-26T20:28:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 1 -- 172.21.15.131:0/597015704 wait complete. 2024-04-26T20:28:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.390+0000 7f9f5da08740 20 client.26786 trim_cache size 0 max 16384 2024-04-26T20:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:28:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:28:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:28:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:28:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9f380a3d70 msgr2=0x7f9f380a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9f380a3d70 0x7f9f380a6230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x559913c91e20 tx=0x7f9f3c000bc0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106770 msgr2=0x7f9f541a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106770 0x7f9f541a2280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400e970 tx=0x7f9f4400ed70 comp rx=0 tx=0).stop 2024-04-26T20:28:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 shutdown_connections 2024-04-26T20:28:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f54106770 0x7f9f541a2280 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54105db0 0x7f9f5419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f9f380a3d70 0x7f9f380a6230 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 --2- 172.21.15.131:0/1027549079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f54104bc0 0x7f9f5419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 >> 172.21.15.131:0/1027549079 conn(0x7f9f54100150 msgr2=0x7f9f54109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 shutdown_connections 2024-04-26T20:28:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:31.391+0000 7f9f5ac00640 1 -- 172.21.15.131:0/1027549079 wait complete. 2024-04-26T20:28:43.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:39 smithi157 ceph-mon[32703]: pgmap v969: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:43.310 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:41 smithi157 ceph-mon[32703]: pgmap v970: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T20:28:43.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:39 smithi148 ceph-mon[32281]: pgmap v969: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:43.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:41 smithi148 ceph-mon[32281]: pgmap v970: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T20:28:43.376 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:31 smithi131 ceph-mon[27919]: pgmap v935: 129 pgs: 129 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 188 KiB/s wr, 84 op/s 2024-04-26T20:28:43.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:33 smithi131 ceph-mon[27919]: pgmap v936: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 180 KiB/s wr, 72 op/s 2024-04-26T20:28:43.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:35 smithi131 ceph-mon[27919]: pgmap v937: 129 pgs: 129 active+clean; 5.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 36 KiB/s wr, 35 op/s 2024-04-26T20:28:43.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:37 smithi131 ceph-mon[27919]: pgmap v938: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.7 KiB/s wr, 17 op/s 2024-04-26T20:28:43.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:39 smithi131 ceph-mon[27919]: pgmap v939: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 14 op/s 2024-04-26T20:28:43.449 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:41 smithi131 ceph-mon[27919]: pgmap v940: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-04-26T20:28:43.473 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:43 smithi131 ceph-mon[27919]: pgmap v941: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-26T20:28:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.533+0000 7f16aa600640 1 -- 172.21.15.131:0/4038431497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 msgr2=0x7f16a4074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.533+0000 7f16aa600640 1 --2- 172.21.15.131:0/4038431497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 0x7f16a4074dc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1698009930 tx=0x7f169802a440 comp rx=0 tx=0).stop 2024-04-26T20:28:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.533+0000 7f16aa600640 1 -- 172.21.15.131:0/4038431497 shutdown_connections 2024-04-26T20:28:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.533+0000 7f16aa600640 1 --2- 172.21.15.131:0/4038431497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16a410eda0 0x7f16a4111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.533+0000 7f16aa600640 1 --2- 172.21.15.131:0/4038431497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a4075390 0x7f16a4075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.533+0000 7f16aa600640 1 --2- 172.21.15.131:0/4038431497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 0x7f16a4074dc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.533+0000 7f16aa600640 1 -- 172.21.15.131:0/4038431497 >> 172.21.15.131:0/4038431497 conn(0x7f16a40fdd40 msgr2=0x7f16a4100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.534+0000 7f16aa600640 1 -- 172.21.15.131:0/4038431497 shutdown_connections 2024-04-26T20:28:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.534+0000 7f16aa600640 1 -- 172.21.15.131:0/4038431497 wait complete. 2024-04-26T20:28:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 Processor -- start 2024-04-26T20:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 -- start start 2024-04-26T20:28:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16a4075390 0x7f16a419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 0x7f16a419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a410eda0 0x7f16a41a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4114940 con 0x7f16a410eda0 2024-04-26T20:28:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41147c0 con 0x7f16a4076970 2024-04-26T20:28:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.535+0000 7f16aa600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4114ac0 con 0x7f16a4075390 2024-04-26T20:28:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16a4075390 0x7f16a419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a410eda0 0x7f16a41a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16a4075390 0x7f16a419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59098/0 (socket says 172.21.15.131:59098) 2024-04-26T20:28:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a410eda0 0x7f16a41a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57412/0 (socket says 172.21.15.131:57412) 2024-04-26T20:28:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9200640 1 -- 172.21.15.131:0/2499709854 learned_addr learned my addr 172.21.15.131:0/2499709854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a3e00640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 0x7f16a419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 -- 172.21.15.131:0/2499709854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16a4075390 msgr2=0x7f16a419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16a4075390 0x7f16a419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 -- 172.21.15.131:0/2499709854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 msgr2=0x7f16a419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 0x7f16a419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 -- 172.21.15.131:0/2499709854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a41a6c60 con 0x7f16a410eda0 2024-04-26T20:28:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.536+0000 7f16a9c00640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a410eda0 0x7f16a41a65f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f169c009fd0 tx=0x7f169c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.537+0000 7f16a1600640 1 -- 172.21.15.131:0/2499709854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c00ce20 con 0x7f16a410eda0 2024-04-26T20:28:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.537+0000 7f16a1600640 1 -- 172.21.15.131:0/2499709854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f169c005020 con 0x7f16a410eda0 2024-04-26T20:28:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.537+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a41a6f50 con 0x7f16a410eda0 2024-04-26T20:28:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.537+0000 7f16a1600640 1 -- 172.21.15.131:0/2499709854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c010690 con 0x7f16a410eda0 2024-04-26T20:28:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.537+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a41a7410 con 0x7f16a410eda0 2024-04-26T20:28:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.540+0000 7f16aea0c740 1 Processor -- start 2024-04-26T20:28:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.540+0000 7f16a1600640 1 -- 172.21.15.131:0/2499709854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f169c0054d0 con 0x7f16a410eda0 2024-04-26T20:28:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.540+0000 7f16aea0c740 1 -- start start 2024-04-26T20:28:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e76200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e767d0 0x55f3c7e7b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16aea0c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7dda380 con 0x55f3c7e7b940 2024-04-26T20:28:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16aea0c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7d76c30 con 0x55f3c7e767d0 2024-04-26T20:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16aea0c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7dd6340 con 0x55f3c7e75de0 2024-04-26T20:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e76200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e76200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59110/0 (socket says 172.21.15.131:59110) 2024-04-26T20:28:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16a3e00640 1 -- 172.21.15.131:0/3471504427 learned_addr learned my addr 172.21.15.131:0/3471504427 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16a9200640 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e767d0 0x55f3c7e7b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.541+0000 7f16a3e00640 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a3e00640 1 -- 172.21.15.131:0/3471504427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e767d0 msgr2=0x55f3c7e7b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a3e00640 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e767d0 0x55f3c7e7b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a3e00640 1 -- 172.21.15.131:0/3471504427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 msgr2=0x55f3c7e7ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a3e00640 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e7ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a3e00640 1 -- 172.21.15.131:0/3471504427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c7d98940 con 0x55f3c7e75de0 2024-04-26T20:28:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a3e00640 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e76200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f16900058c0 tx=0x7f1690005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb65bfb9bc88672e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a1600640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f168c0a3c30 0x7f168c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f1696000640 1 -- 172.21.15.131:0/3471504427 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690004030 con 0x55f3c7e75de0 2024-04-26T20:28:44.619 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f1696000640 1 -- 172.21.15.131:0/3471504427 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1690005020 con 0x55f3c7e75de0 2024-04-26T20:28:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f1696000640 1 -- 172.21.15.131:0/3471504427 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690004ba0 con 0x55f3c7e75de0 2024-04-26T20:28:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a9200640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f168c0a3c30 0x7f168c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.542+0000 7f16a1600640 1 -- 172.21.15.131:0/2499709854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f169c0d1070 con 0x7f16a410eda0 2024-04-26T20:28:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 -- 172.21.15.131:0/3471504427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 msgr2=0x55f3c7e76200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16a9200640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f168c0a3c30 0x7f168c0a60f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1698000c00 tx=0x7f1698005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e76200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f16900058c0 tx=0x7f1690005850 comp rx=0 tx=0).stop 2024-04-26T20:28:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 -- 172.21.15.131:0/3471504427 shutdown_connections 2024-04-26T20:28:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e7ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e767d0 0x55f3c7e7b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3471504427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e76200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 -- 172.21.15.131:0/3471504427 >> 172.21.15.131:0/3471504427 conn(0x55f3c7e71450 msgr2=0x55f3c7e71850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 -- 172.21.15.131:0/3471504427 shutdown_connections 2024-04-26T20:28:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.543+0000 7f16aea0c740 1 -- 172.21.15.131:0/3471504427 wait complete. 2024-04-26T20:28:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.544+0000 7f16aea0c740 1 -- start start 2024-04-26T20:28:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.544+0000 7f16aea0c740 1 Processor -- start 2024-04-26T20:28:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.544+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e80030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.544+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e80570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e87540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16aea0c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7e7e8f0 con 0x55f3c7e7b940 2024-04-26T20:28:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16aea0c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7d61c60 con 0x55f3c7e75de0 2024-04-26T20:28:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16aea0c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7e7ea70 con 0x55f3c7e767d0 2024-04-26T20:28:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16aea0c740 10 client.0 _ll_register_callbacks cb 0x55f3c7d8de50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e87540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e80030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e80030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:38428/0 (socket says 172.21.15.131:38428) 2024-04-26T20:28:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 -- 172.21.15.131:0/42899492 learned_addr learned my addr 172.21.15.131:0/42899492 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a3e00640 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e80570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 -- 172.21.15.131:0/42899492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 msgr2=0x55f3c7e80570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e80570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 -- 172.21.15.131:0/42899492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 msgr2=0x55f3c7e87540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e87540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9200640 1 -- 172.21.15.131:0/42899492 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c7d70510 con 0x55f3c7e75de0 2024-04-26T20:28:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.545+0000 7f16a9c00640 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e87540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f16a9200640 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e80030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1698009900 tx=0x7f1698009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88425762d5e70148 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f168aa00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:28:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f168aa00640 1 -- 172.21.15.131:0/42899492 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698035040 con 0x55f3c7e75de0 2024-04-26T20:28:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f168aa00640 1 -- 172.21.15.131:0/42899492 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f16980319b0 con 0x55f3c7e75de0 2024-04-26T20:28:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1690005020 con 0x55f3c7e75de0 2024-04-26T20:28:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f168aa00640 1 -- 172.21.15.131:0/42899492 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1698003e70 con 0x55f3c7e75de0 2024-04-26T20:28:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f16aea0c740 10 client.26798 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3c7aec210 con 0x55f3c7e75de0 2024-04-26T20:28:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.546+0000 7f168aa00640 1 -- 172.21.15.131:0/42899492 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1690005020 con 0x55f3c7e75de0 2024-04-26T20:28:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f168aa00640 1 -- 172.21.15.131:0/42899492 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1698044030 con 0x55f3c7e75de0 2024-04-26T20:28:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f16aea0c740 10 client.26798 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f16aea0c740 10 client.26798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f16aea0c740 20 client.26798 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f16aea0c740 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7e8ba30 0x55f3c7e8dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f16aea0c740 4 client.26798 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c7de7f90 con 0x55f3c7e8ba30 2024-04-26T20:28:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.547+0000 7f16a3e00640 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7e8ba30 0x55f3c7e8dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.548+0000 7f16a3e00640 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7e8ba30 0x55f3c7e8dea0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f178bd7e95f4c2a server_cookie=5b6f942d07154065 in_seq=0 out_seq=0 2024-04-26T20:28:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.548+0000 7f168aa00640 10 client.26798 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.548+0000 7f168aa00640 1 -- 172.21.15.131:0/42899492 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3c7de7f90 con 0x55f3c7e8ba30 2024-04-26T20:28:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.548+0000 7f168aa00640 10 client.26798 handle_command_reply: tid=0 2024-04-26T20:28:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 client.26798 shutdown 2024-04-26T20:28:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 msgr2=0x55f3c7e80030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e80030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1698009900 tx=0x7f1698009260 comp rx=0 tx=0).stop 2024-04-26T20:28:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 shutdown_connections 2024-04-26T20:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7e8ba30 0x55f3c7e8dea0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e87540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e80570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 --2- 172.21.15.131:0/42899492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e80030 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f168aa00640 0 client.26798 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 >> 172.21.15.131:0/42899492 conn(0x55f3c7e71450 msgr2=0x55f3c7e70650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 shutdown_connections 2024-04-26T20:28:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 1 -- 172.21.15.131:0/42899492 wait complete. 2024-04-26T20:28:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.549+0000 7f16aea0c740 20 client.26798 trim_cache size 0 max 16384 2024-04-26T20:28:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.565+0000 7f16aea0c740 1 Processor -- start 2024-04-26T20:28:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16aea0c740 1 -- start start 2024-04-26T20:28:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e74d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16aea0c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41b43f0 con 0x55f3c7e7b940 2024-04-26T20:28:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16aea0c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41b1cc0 con 0x55f3c7e75de0 2024-04-26T20:28:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16aea0c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41afb10 con 0x55f3c7e767d0 2024-04-26T20:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16a9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16a9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e80350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57422/0 (socket says 172.21.15.131:57422) 2024-04-26T20:28:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16a9c00640 1 -- 172.21.15.131:0/1972700028 learned_addr learned my addr 172.21.15.131:0/1972700028 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16a3e00640 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.566+0000 7f16a9200640 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e74d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f16a3e00640 1 -- 172.21.15.131:0/1972700028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 msgr2=0x55f3c7e74d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f16a3e00640 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e74d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f16a3e00640 1 -- 172.21.15.131:0/1972700028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 msgr2=0x55f3c7e80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f16a3e00640 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f16a3e00640 1 -- 172.21.15.131:0/1972700028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c7bb8620 con 0x55f3c7e767d0 2024-04-26T20:28:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f16a9c00640 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f16a3e00640 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e874d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1690009800 tx=0x7f1690009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ea492fad2dc698 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f1695600640 1 -- 172.21.15.131:0/1972700028 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690011050 con 0x55f3c7e767d0 2024-04-26T20:28:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f1695600640 1 -- 172.21.15.131:0/1972700028 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16900044f0 con 0x55f3c7e767d0 2024-04-26T20:28:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.567+0000 7f1695600640 1 -- 172.21.15.131:0/1972700028 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169000fcb0 con 0x55f3c7e767d0 2024-04-26T20:28:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 -- 172.21.15.131:0/1972700028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 msgr2=0x55f3c7e874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e874d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1690009800 tx=0x7f1690009fd0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 -- 172.21.15.131:0/1972700028 shutdown_connections 2024-04-26T20:28:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e7b940 0x55f3c7e80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e767d0 0x55f3c7e874d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 --2- 172.21.15.131:0/1972700028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e75de0 0x55f3c7e74d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 -- 172.21.15.131:0/1972700028 >> 172.21.15.131:0/1972700028 conn(0x55f3c7e71450 msgr2=0x55f3c7e7fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 -- 172.21.15.131:0/1972700028 shutdown_connections 2024-04-26T20:28:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.568+0000 7f16aea0c740 1 -- 172.21.15.131:0/1972700028 wait complete. 2024-04-26T20:28:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 -- start start 2024-04-26T20:28:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 Processor -- start 2024-04-26T20:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e767d0 0x55f3c7e85020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:46.693 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e7b940 0x55f3c7e86030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41b0180 con 0x55f3c7e767d0 2024-04-26T20:28:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41b15f0 con 0x55f3c7e7b940 2024-04-26T20:28:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41b0850 con 0x55f3c7e75de0 2024-04-26T20:28:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16aea0c740 10 client.0 _ll_register_callbacks cb 0x7f1698009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.569+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e767d0 0x55f3c7e85020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e767d0 0x55f3c7e85020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57438/0 (socket says 172.21.15.131:57438) 2024-04-26T20:28:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 -- 172.21.15.131:0/3591916710 learned_addr learned my addr 172.21.15.131:0/3591916710 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a9200640 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a9c00640 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e7b940 0x55f3c7e86030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 -- 172.21.15.131:0/3591916710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 msgr2=0x55f3c7e84ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e84ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 -- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e7b940 msgr2=0x55f3c7e86030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e7b940 0x55f3c7e86030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 -- 172.21.15.131:0/3591916710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c7d94cb0 con 0x55f3c7e767d0 2024-04-26T20:28:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f16a3e00640 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e767d0 0x55f3c7e85020 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1690009c00 tx=0x7f1690009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62456f92f02be4e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.570+0000 7f168aa00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:28:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f168aa00640 1 -- 172.21.15.131:0/3591916710 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690005690 con 0x55f3c7e767d0 2024-04-26T20:28:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16900044f0 con 0x55f3c7e767d0 2024-04-26T20:28:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f168aa00640 1 -- 172.21.15.131:0/3591916710 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f169000f9f0 con 0x55f3c7e767d0 2024-04-26T20:28:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f168aa00640 1 -- 172.21.15.131:0/3591916710 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f169000fbf0 con 0x55f3c7e767d0 2024-04-26T20:28:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f16aea0c740 10 client.16686 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3c7e87760 con 0x55f3c7e767d0 2024-04-26T20:28:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f168aa00640 1 -- 172.21.15.131:0/3591916710 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16900044f0 con 0x55f3c7e767d0 2024-04-26T20:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.571+0000 7f168aa00640 1 -- 172.21.15.131:0/3591916710 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f169002d5c0 con 0x55f3c7e767d0 2024-04-26T20:28:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16aea0c740 10 client.16686 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16aea0c740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16aea0c740 20 client.16686 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7ef2600 0x55f3c7ef4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16aea0c740 4 client.16686 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:28:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3c7de1e00 con 0x55f3c7ef2600 2024-04-26T20:28:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16a9200640 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7ef2600 0x55f3c7ef4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f16a9200640 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7ef2600 0x55f3c7ef4a20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84de7d6c42b2574 server_cookie=7b4ca360dbc3a3c2 in_seq=0 out_seq=0 2024-04-26T20:28:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.572+0000 7f168aa00640 10 client.16686 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.583+0000 7f168aa00640 1 -- 172.21.15.131:0/3591916710 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3c7de1e00 con 0x55f3c7ef2600 2024-04-26T20:28:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.583+0000 7f168aa00640 10 client.16686 handle_command_reply: tid=0 2024-04-26T20:28:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.583+0000 7f16aea0c740 1 client.16686 shutdown 2024-04-26T20:28:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.583+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e767d0 msgr2=0x55f3c7e85020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e767d0 0x55f3c7e85020 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1690009c00 tx=0x7f1690009cf0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 shutdown_connections 2024-04-26T20:28:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55f3c7ef2600 0x55f3c7ef4a20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f3c7e7b940 0x55f3c7e86030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3c7e767d0 0x55f3c7e85020 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 --2- 172.21.15.131:0/3591916710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3c7e75de0 0x55f3c7e84ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f168aa00640 0 client.16686 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 >> 172.21.15.131:0/3591916710 conn(0x55f3c7e71450 msgr2=0x55f3c7e6fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 shutdown_connections 2024-04-26T20:28:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 1 -- 172.21.15.131:0/3591916710 wait complete. 2024-04-26T20:28:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.584+0000 7f16aea0c740 20 client.16686 trim_cache size 0 max 16384 2024-04-26T20:28:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:28:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:28:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:28:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:28:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f168c0a3c30 msgr2=0x7f168c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f168c0a3c30 0x7f168c0a60f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1698000c00 tx=0x7f1698005a20 comp rx=0 tx=0).stop 2024-04-26T20:28:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a410eda0 msgr2=0x7f16a41a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a410eda0 0x7f16a41a65f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f169c009fd0 tx=0x7f169c00ef10 comp rx=0 tx=0).stop 2024-04-26T20:28:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 shutdown_connections 2024-04-26T20:28:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16a410eda0 0x7f16a41a65f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a4076970 0x7f16a419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f168c0a3c30 0x7f168c0a60f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 --2- 172.21.15.131:0/2499709854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16a4075390 0x7f16a419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 >> 172.21.15.131:0/2499709854 conn(0x7f16a40fdd40 msgr2=0x7f16a4102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 shutdown_connections 2024-04-26T20:28:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:44.585+0000 7f16aa600640 1 -- 172.21.15.131:0/2499709854 wait complete. 2024-04-26T20:28:47.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:43 smithi157 ceph-mon[32703]: pgmap v971: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:47.856 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:44 smithi157 ceph-mon[32703]: scrub queued for path: / 2024-04-26T20:28:47.865 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:44 smithi157 ceph-mon[32703]: scrub summary: idle+waiting paths [/] 2024-04-26T20:28:47.874 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:44 smithi157 ceph-mon[32703]: scrub summary: active paths [/] 2024-04-26T20:28:47.882 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:44 smithi157 ceph-mon[32703]: scrub summary: idle+waiting paths [/] 2024-04-26T20:28:47.891 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:45 smithi157 ceph-mon[32703]: pgmap v972: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:28:47.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:43 smithi148 ceph-mon[32281]: pgmap v971: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:47.981 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:44 smithi148 ceph-mon[32281]: scrub queued for path: / 2024-04-26T20:28:47.990 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:44 smithi148 ceph-mon[32281]: scrub summary: idle+waiting paths [/] 2024-04-26T20:28:47.999 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:44 smithi148 ceph-mon[32281]: scrub summary: active paths [/] 2024-04-26T20:28:48.007 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:44 smithi148 ceph-mon[32281]: scrub summary: idle+waiting paths [/] 2024-04-26T20:28:48.034 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:45 smithi148 ceph-mon[32281]: pgmap v972: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:28:48.075 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:45 smithi131 ceph-mon[27919]: pgmap v942: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T20:28:48.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:47 smithi131 ceph-mon[27919]: pgmap v943: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-04-26T20:28:48.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:49 smithi131 ceph-mon[27919]: pgmap v944: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T20:28:48.102 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:51 smithi131 ceph-mon[27919]: pgmap v945: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T20:28:48.111 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:53 smithi131 ceph-mon[27919]: pgmap v946: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T20:28:48.121 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:55 smithi131 ceph-mon[27919]: pgmap v947: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T20:28:48.169 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:47 smithi157 ceph-mon[32703]: pgmap v973: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:48.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:47 smithi157 ceph-mon[32703]: scrub summary: idle 2024-04-26T20:28:48.187 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:47 smithi148 ceph-mon[32281]: pgmap v973: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:48.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:47 smithi148 ceph-mon[32281]: scrub summary: idle 2024-04-26T20:28:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 -- 172.21.15.131:0/363854452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 msgr2=0x7f473410a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 --2- 172.21.15.131:0/363854452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 0x7f473410a450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4724009860 tx=0x7f472402a340 comp rx=0 tx=0).stop 2024-04-26T20:28:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 -- 172.21.15.131:0/363854452 shutdown_connections 2024-04-26T20:28:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 --2- 172.21.15.131:0/363854452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473410a990 0x7f4734111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 --2- 172.21.15.131:0/363854452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 0x7f473410a450 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 --2- 172.21.15.131:0/363854452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4734108de0 0x7f4734109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 -- 172.21.15.131:0/363854452 >> 172.21.15.131:0/363854452 conn(0x7f4734077dc0 msgr2=0x7f473407a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 -- 172.21.15.131:0/363854452 shutdown_connections 2024-04-26T20:28:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.727+0000 7f4733200640 1 -- 172.21.15.131:0/363854452 wait complete. 2024-04-26T20:28:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.728+0000 7f4733200640 1 Processor -- start 2024-04-26T20:28:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.728+0000 7f4733200640 1 -- start start 2024-04-26T20:28:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4733200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4734108de0 0x7f473419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4733200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 0x7f473419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4733200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f473410a990 0x7f47341a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4733200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4734114910 con 0x7f4734109fd0 2024-04-26T20:28:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4733200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4734114790 con 0x7f473410a990 2024-04-26T20:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4733200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4734114a90 con 0x7f4734108de0 2024-04-26T20:28:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4731e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4734108de0 0x7f473419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4732800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f473410a990 0x7f47341a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4731e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4734108de0 0x7f473419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40258/0 (socket says 172.21.15.131:40258) 2024-04-26T20:28:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4731e00640 1 -- 172.21.15.131:0/473842636 learned_addr learned my addr 172.21.15.131:0/473842636 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4731400640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 0x7f473419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4732800640 1 -- 172.21.15.131:0/473842636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4734108de0 msgr2=0x7f473419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4732800640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4734108de0 0x7f473419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4732800640 1 -- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 msgr2=0x7f473419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4732800640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 0x7f473419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4732800640 1 -- 172.21.15.131:0/473842636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47341a6be0 con 0x7f473410a990 2024-04-26T20:28:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.729+0000 7f4731e00640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4734108de0 0x7f473419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.730+0000 7f4732800640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f473410a990 0x7f47341a6570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.730+0000 7f472aa00640 1 -- 172.21.15.131:0/473842636 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c00ce20 con 0x7f473410a990 2024-04-26T20:28:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.730+0000 7f472aa00640 1 -- 172.21.15.131:0/473842636 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f472c005020 con 0x7f473410a990 2024-04-26T20:28:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.730+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47341a6ed0 con 0x7f473410a990 2024-04-26T20:28:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.730+0000 7f472aa00640 1 -- 172.21.15.131:0/473842636 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c010690 con 0x7f473410a990 2024-04-26T20:28:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.730+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47341a7390 con 0x7f473410a990 2024-04-26T20:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.733+0000 7f473bb9b740 1 Processor -- start 2024-04-26T20:28:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.733+0000 7f472aa00640 1 -- 172.21.15.131:0/473842636 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f472c0054d0 con 0x7f473410a990 2024-04-26T20:28:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.733+0000 7f473bb9b740 1 -- start start 2024-04-26T20:28:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfac7d0 0x56228cfb12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfb3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f473bb9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56228cf10380 con 0x56228cfb1940 2024-04-26T20:28:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f473bb9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56228ceacc30 con 0x56228cfabde0 2024-04-26T20:28:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f473bb9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56228cf0c340 con 0x56228cfac7d0 2024-04-26T20:28:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f4731e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfac7d0 0x56228cfb12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f4731400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfb3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f4731e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfac7d0 0x56228cfb12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40272/0 (socket says 172.21.15.131:40272) 2024-04-26T20:28:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f4731e00640 1 -- 172.21.15.131:0/1830578989 learned_addr learned my addr 172.21.15.131:0/1830578989 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.734+0000 7f4731400640 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4731e00640 1 -- 172.21.15.131:0/1830578989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 msgr2=0x56228cfac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4731e00640 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4731e00640 1 -- 172.21.15.131:0/1830578989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 msgr2=0x56228cfb3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4731e00640 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfb3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4731e00640 1 -- 172.21.15.131:0/1830578989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56228cece940 con 0x56228cfac7d0 2024-04-26T20:28:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4731e00640 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfac7d0 0x56228cfb12d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f471c009fd0 tx=0x7f471c011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c022806331b43c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f472aa00640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47180a3c30 0x7f47180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4723000640 1 -- 172.21.15.131:0/1830578989 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c01e070 con 0x56228cfac7d0 2024-04-26T20:28:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4723000640 1 -- 172.21.15.131:0/1830578989 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471c00faf0 con 0x56228cfac7d0 2024-04-26T20:28:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4723000640 1 -- 172.21.15.131:0/1830578989 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c019b40 con 0x56228cfac7d0 2024-04-26T20:28:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f472aa00640 1 -- 172.21.15.131:0/473842636 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f472c0d1070 con 0x7f473410a990 2024-04-26T20:28:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.735+0000 7f4731400640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47180a3c30 0x7f47180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 -- 172.21.15.131:0/1830578989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfac7d0 msgr2=0x56228cfb12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfac7d0 0x56228cfb12d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f471c009fd0 tx=0x7f471c011a50 comp rx=0 tx=0).stop 2024-04-26T20:28:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 -- 172.21.15.131:0/1830578989 shutdown_connections 2024-04-26T20:28:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfb3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfac7d0 0x56228cfb12d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 --2- 172.21.15.131:0/1830578989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 -- 172.21.15.131:0/1830578989 >> 172.21.15.131:0/1830578989 conn(0x56228cfa7450 msgr2=0x56228cfa7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 -- 172.21.15.131:0/1830578989 shutdown_connections 2024-04-26T20:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f4731400640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47180a3c30 0x7f47180a60f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f472402a930 tx=0x7f472402a340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.736+0000 7f473bb9b740 1 -- 172.21.15.131:0/1830578989 wait complete. 2024-04-26T20:28:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 -- start start 2024-04-26T20:28:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 Processor -- start 2024-04-26T20:28:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfabde0 0x56228cfb5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfb6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfbd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56228cfb48f0 con 0x56228cfb1940 2024-04-26T20:28:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56228ce97c60 con 0x56228cfac7d0 2024-04-26T20:28:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56228cfb4a70 con 0x56228cfabde0 2024-04-26T20:28:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.737+0000 7f473bb9b740 10 client.0 _ll_register_callbacks cb 0x56228cec3e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfabde0 0x56228cfb5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfabde0 0x56228cfb5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40286/0 (socket says 172.21.15.131:40286) 2024-04-26T20:28:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4732800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfbd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 -- 172.21.15.131:0/2166331209 learned_addr learned my addr 172.21.15.131:0/2166331209 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731400640 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfb6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 -- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 msgr2=0x56228cfb6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfb6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 -- 172.21.15.131:0/2166331209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 msgr2=0x56228cfbd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfbd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 -- 172.21.15.131:0/2166331209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56228cea6510 con 0x56228cfabde0 2024-04-26T20:28:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4731e00640 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfabde0 0x56228cfb5f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f471c000c00 tx=0x7f471c009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7211342f46af217b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.738+0000 7f4717400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:28:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f4717400640 1 -- 172.21.15.131:0/2166331209 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c01e030 con 0x56228cfabde0 2024-04-26T20:28:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f4717400640 1 -- 172.21.15.131:0/2166331209 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f471c004040 con 0x56228cfabde0 2024-04-26T20:28:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f471c00faf0 con 0x56228cfabde0 2024-04-26T20:28:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f4717400640 1 -- 172.21.15.131:0/2166331209 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f471c031050 con 0x56228cfabde0 2024-04-26T20:28:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f473bb9b740 10 client.26797 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56228cc22210 con 0x56228cfabde0 2024-04-26T20:28:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f4717400640 1 -- 172.21.15.131:0/2166331209 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471c00faf0 con 0x56228cfabde0 2024-04-26T20:28:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.739+0000 7f4717400640 1 -- 172.21.15.131:0/2166331209 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f471c0051e0 con 0x56228cfabde0 2024-04-26T20:28:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f473bb9b740 10 client.26797 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f473bb9b740 10 client.26797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f473bb9b740 20 client.26797 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228cfc1700 0x56228cfc3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f473bb9b740 4 client.26797 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56228cf1df90 con 0x56228cfc1700 2024-04-26T20:28:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f4731400640 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228cfc1700 0x56228cfc3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f4731400640 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228cfc1700 0x56228cfc3af0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1832d791030532e server_cookie=7b0841f0ca23291c in_seq=0 out_seq=0 2024-04-26T20:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.740+0000 7f4717400640 10 client.26797 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f4717400640 1 -- 172.21.15.131:0/2166331209 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56228cf1df90 con 0x56228cfc1700 2024-04-26T20:28:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f4717400640 10 client.26797 handle_command_reply: tid=0 2024-04-26T20:28:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 client.26797 shutdown 2024-04-26T20:28:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfabde0 msgr2=0x56228cfb5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfabde0 0x56228cfb5f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f471c000c00 tx=0x7f471c009eb0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 shutdown_connections 2024-04-26T20:28:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228cfc1700 0x56228cfc3af0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfb1940 0x56228cfbd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfb6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f4717400640 0 client.26797 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2166331209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfabde0 0x56228cfb5f00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.741+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 >> 172.21.15.131:0/2166331209 conn(0x56228cfa7450 msgr2=0x56228cfa6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.742+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 shutdown_connections 2024-04-26T20:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.742+0000 7f473bb9b740 1 -- 172.21.15.131:0/2166331209 wait complete. 2024-04-26T20:28:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.742+0000 7f473bb9b740 20 client.26797 trim_cache size 0 max 16384 2024-04-26T20:28:50.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:57 smithi131 ceph-mon[27919]: pgmap v948: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T20:28:50.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:49 smithi148 ceph-mon[32281]: pgmap v974: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:28:50.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:49 smithi157 ceph-mon[32703]: pgmap v974: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:28:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 Processor -- start 2024-04-26T20:28:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 -- start start 2024-04-26T20:28:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfaa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfac7d0 0x56228cfbd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228cfa5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47341b4370 con 0x56228cfac7d0 2024-04-26T20:28:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47341b1c40 con 0x56228cfabde0 2024-04-26T20:28:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.758+0000 7f473bb9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47341afa90 con 0x56228cfb1940 2024-04-26T20:28:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfaa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfac7d0 0x56228cfbd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4732800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228cfa5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfaa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:44404/0 (socket says 172.21.15.131:44404) 2024-04-26T20:28:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 -- 172.21.15.131:0/3604025110 learned_addr learned my addr 172.21.15.131:0/3604025110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 -- 172.21.15.131:0/3604025110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 msgr2=0x56228cfa5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 --2- 172.21.15.131:0/3604025110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228cfa5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 -- 172.21.15.131:0/3604025110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfac7d0 msgr2=0x56228cfbd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 --2- 172.21.15.131:0/3604025110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfac7d0 0x56228cfbd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 -- 172.21.15.131:0/3604025110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56228ccee620 con 0x56228cfabde0 2024-04-26T20:28:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4731e00640 1 --2- 172.21.15.131:0/3604025110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfaa8e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f471c011510 tx=0x7f471c011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca3a36a23f03f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.759+0000 7f4722600640 1 -- 172.21.15.131:0/3604025110 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c01e030 con 0x56228cfabde0 2024-04-26T20:28:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f4722600640 1 -- 172.21.15.131:0/3604025110 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471c016020 con 0x56228cfabde0 2024-04-26T20:28:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f4722600640 1 -- 172.21.15.131:0/3604025110 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0051b0 con 0x56228cfabde0 2024-04-26T20:28:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 -- 172.21.15.131:0/3604025110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 msgr2=0x56228cfaa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 --2- 172.21.15.131:0/3604025110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfaa8e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f471c011510 tx=0x7f471c011a50 comp rx=0 tx=0).stop 2024-04-26T20:28:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 -- 172.21.15.131:0/3604025110 shutdown_connections 2024-04-26T20:28:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 --2- 172.21.15.131:0/3604025110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228cfa5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 --2- 172.21.15.131:0/3604025110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfac7d0 0x56228cfbd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 --2- 172.21.15.131:0/3604025110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfabde0 0x56228cfaa8e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 -- 172.21.15.131:0/3604025110 >> 172.21.15.131:0/3604025110 conn(0x56228cfa7450 msgr2=0x56228cfaa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 -- 172.21.15.131:0/3604025110 shutdown_connections 2024-04-26T20:28:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.760+0000 7f473bb9b740 1 -- 172.21.15.131:0/3604025110 wait complete. 2024-04-26T20:28:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 -- start start 2024-04-26T20:28:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 Processor -- start 2024-04-26T20:28:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfabde0 0x56228cfba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228d025c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47341b0100 con 0x56228cfabde0 2024-04-26T20:28:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47341b1570 con 0x56228cfac7d0 2024-04-26T20:28:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47341b07d0 con 0x56228cfb1940 2024-04-26T20:28:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.761+0000 7f473bb9b740 10 client.0 _ll_register_callbacks cb 0x7f471c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfabde0 0x56228cfba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfabde0 0x56228cfba260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57680/0 (socket says 172.21.15.131:57680) 2024-04-26T20:28:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 -- 172.21.15.131:0/2202421536 learned_addr learned my addr 172.21.15.131:0/2202421536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731400640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4732800640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228d025c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 -- 172.21.15.131:0/2202421536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 msgr2=0x56228d025c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228d025c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 -- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 msgr2=0x56228cfba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 -- 172.21.15.131:0/2202421536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56228cecacb0 con 0x56228cfabde0 2024-04-26T20:28:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4732800640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228d025c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:28:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4731e00640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfabde0 0x56228cfba260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f471c0057c0 tx=0x7f471c004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337219ff4249fa6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4717400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:28:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.762+0000 7f4717400640 1 -- 172.21.15.131:0/2202421536 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c01e070 con 0x56228cfabde0 2024-04-26T20:28:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f4717400640 1 -- 172.21.15.131:0/2202421536 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f471c005100 con 0x56228cfabde0 2024-04-26T20:28:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f471c016020 con 0x56228cfabde0 2024-04-26T20:28:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f4717400640 1 -- 172.21.15.131:0/2202421536 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f471c043050 con 0x56228cfabde0 2024-04-26T20:28:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 10 client.16704 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56228d027180 con 0x56228cfabde0 2024-04-26T20:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f4717400640 1 -- 172.21.15.131:0/2202421536 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471c016020 con 0x56228cfabde0 2024-04-26T20:28:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f4717400640 1 -- 172.21.15.131:0/2202421536 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f471c023440 con 0x56228cfabde0 2024-04-26T20:28:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 10 client.16704 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 20 client.16704 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228d027ac0 0x56228d029ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 4 client.16704 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:28:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.763+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56228cf17e00 con 0x56228d027ac0 2024-04-26T20:28:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.764+0000 7f4731400640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228d027ac0 0x56228d029ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.764+0000 7f4731400640 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228d027ac0 0x56228d029ee0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b709b76be1ae1229 server_cookie=a856becb8cb80330 in_seq=0 out_seq=0 2024-04-26T20:28:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.764+0000 7f4717400640 10 client.16704 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f4717400640 1 -- 172.21.15.131:0/2202421536 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56228cf17e00 con 0x56228d027ac0 2024-04-26T20:28:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f4717400640 10 client.16704 handle_command_reply: tid=0 2024-04-26T20:28:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 client.16704 shutdown 2024-04-26T20:28:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfabde0 msgr2=0x56228cfba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfabde0 0x56228cfba260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f471c0057c0 tx=0x7f471c004940 comp rx=0 tx=0).stop 2024-04-26T20:28:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 shutdown_connections 2024-04-26T20:28:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x56228d027ac0 0x56228d029ee0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56228cfb1940 0x56228d025c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56228cfac7d0 0x56228cfba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 --2- 172.21.15.131:0/2202421536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56228cfabde0 0x56228cfba260 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f4717400640 0 client.16704 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 >> 172.21.15.131:0/2202421536 conn(0x56228cfa7450 msgr2=0x56228cfa5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 shutdown_connections 2024-04-26T20:28:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.773+0000 7f473bb9b740 1 -- 172.21.15.131:0/2202421536 wait complete. 2024-04-26T20:28:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f473bb9b740 20 client.16704 trim_cache size 0 max 16384 2024-04-26T20:28:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:28:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:28:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:28:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:28:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47180a3c30 msgr2=0x7f47180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47180a3c30 0x7f47180a60f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f472402a930 tx=0x7f472402a340 comp rx=0 tx=0).stop 2024-04-26T20:28:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f473410a990 msgr2=0x7f47341a6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f473410a990 0x7f47341a6570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c00ef10 comp rx=0 tx=0).stop 2024-04-26T20:28:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 shutdown_connections 2024-04-26T20:28:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f473410a990 0x7f47341a6570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4734109fd0 0x7f473419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f47180a3c30 0x7f47180a60f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 --2- 172.21.15.131:0/473842636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4734108de0 0x7f473419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 >> 172.21.15.131:0/473842636 conn(0x7f4734077dc0 msgr2=0x7f473410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.774+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 shutdown_connections 2024-04-26T20:28:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:27:57.775+0000 7f4733200640 1 -- 172.21.15.131:0/473842636 wait complete. 2024-04-26T20:28:52.347 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:27:59 smithi131 ceph-mon[27919]: pgmap v949: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T20:28:52.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:01 smithi131 ceph-mon[27919]: pgmap v950: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T20:28:52.414 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:01 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:28:52.423 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:28:52.431 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:28:52.438 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:28:52.445 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:03 smithi131 ceph-mon[27919]: pgmap v951: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:28:52.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:05 smithi131 ceph-mon[27919]: pgmap v952: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T20:28:52.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:07 smithi131 ceph-mon[27919]: pgmap v953: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:28:52.533 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:09 smithi131 ceph-mon[27919]: pgmap v954: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T20:28:52.543 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:51 smithi148 ceph-mon[32281]: pgmap v975: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:28:52.551 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:51 smithi157 ceph-mon[32703]: pgmap v975: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:28:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 -- 172.21.15.131:0/3634385557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5008103290 msgr2=0x7f5008109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 --2- 172.21.15.131:0/3634385557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5008103290 0x7f5008109b20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5000009ca0 tx=0x7f500002a8c0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 -- 172.21.15.131:0/3634385557 shutdown_connections 2024-04-26T20:28:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 --2- 172.21.15.131:0/3634385557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5008103290 0x7f5008109b20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 --2- 172.21.15.131:0/3634385557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 0x7f5008102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 --2- 172.21.15.131:0/3634385557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50081016e0 0x7f5008101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 -- 172.21.15.131:0/3634385557 >> 172.21.15.131:0/3634385557 conn(0x7f50080fcc70 msgr2=0x7f50080ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 -- 172.21.15.131:0/3634385557 shutdown_connections 2024-04-26T20:28:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.918+0000 7f500e400640 1 -- 172.21.15.131:0/3634385557 wait complete. 2024-04-26T20:28:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.919+0000 7f500e400640 1 Processor -- start 2024-04-26T20:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.919+0000 7f500e400640 1 -- start start 2024-04-26T20:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.919+0000 7f500e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50081016e0 0x7f500819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500e400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 0x7f500819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5008103290 0x7f50081a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f500810e390 con 0x7f5008103290 2024-04-26T20:28:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500e400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f500810e210 con 0x7f50081028d0 2024-04-26T20:28:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f500810e510 con 0x7f50081016e0 2024-04-26T20:28:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5008103290 0x7f50081a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 0x7f500819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50081016e0 0x7f500819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 0x7f500819b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:56874/0 (socket says 172.21.15.131:56874) 2024-04-26T20:28:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f500da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5008103290 0x7f50081a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41896/0 (socket says 172.21.15.131:41896) 2024-04-26T20:28:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 -- 172.21.15.131:0/2720324168 learned_addr learned my addr 172.21.15.131:0/2720324168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 -- 172.21.15.131:0/2720324168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50081016e0 msgr2=0x7f500819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50081016e0 0x7f500819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 -- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5008103290 msgr2=0x7f50081a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5008103290 0x7f50081a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.920+0000 7f5007e00640 1 -- 172.21.15.131:0/2720324168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50081a27a0 con 0x7f50081028d0 2024-04-26T20:28:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.921+0000 7f5007e00640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 0x7f500819b0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00ea60 tx=0x7f4ffc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:53.208 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.921+0000 7f5005600640 1 -- 172.21.15.131:0/2720324168 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00cde0 con 0x7f50081028d0 2024-04-26T20:28:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.921+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50081a2a90 con 0x7f50081028d0 2024-04-26T20:28:53.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.921+0000 7f5005600640 1 -- 172.21.15.131:0/2720324168 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ffc0035f0 con 0x7f50081028d0 2024-04-26T20:28:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.921+0000 7f5005600640 1 -- 172.21.15.131:0/2720324168 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc010690 con 0x7f50081028d0 2024-04-26T20:28:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.921+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50081a2fe0 con 0x7f50081028d0 2024-04-26T20:28:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.925+0000 7f50127dd740 1 Processor -- start 2024-04-26T20:28:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.925+0000 7f50127dd740 1 -- start start 2024-04-26T20:28:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f50127dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf6de0 0x55a90acf7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f50127dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf77d0 0x55a90acfc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f50127dd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90acfedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f50127dd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a90ac5b380 con 0x55a90acfc940 2024-04-26T20:28:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f50127dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a90abf7c30 con 0x55a90acf6de0 2024-04-26T20:28:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f50127dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a90ac57340 con 0x55a90acf77d0 2024-04-26T20:28:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f500d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf77d0 0x55a90acfc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f500d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf77d0 0x55a90acfc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56406/0 (socket says 172.21.15.131:56406) 2024-04-26T20:28:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f500d000640 1 -- 172.21.15.131:0/247619830 learned_addr learned my addr 172.21.15.131:0/247619830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f5005600640 1 -- 172.21.15.131:0/2720324168 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4ffc0037a0 con 0x7f50081028d0 2024-04-26T20:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.926+0000 7f500da00640 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf6de0 0x55a90acf7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f500d000640 1 -- 172.21.15.131:0/247619830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf6de0 msgr2=0x55a90acf7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f500d000640 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf6de0 0x55a90acf7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f5007e00640 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90acfedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f500d000640 1 -- 172.21.15.131:0/247619830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 msgr2=0x55a90acfedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f500d000640 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90acfedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f500d000640 1 -- 172.21.15.131:0/247619830 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ac19940 con 0x55a90acf77d0 2024-04-26T20:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f5007e00640 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90acfedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f500d000640 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf77d0 0x55a90acfc2d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009ca0 tx=0x7f4ff8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b656b0209132531e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f4ff6000640 1 -- 172.21.15.131:0/247619830 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8004220 con 0x55a90acf77d0 2024-04-26T20:28:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f4ff6000640 1 -- 172.21.15.131:0/247619830 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff80043c0 con 0x55a90acf77d0 2024-04-26T20:28:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.927+0000 7f4ff6000640 1 -- 172.21.15.131:0/247619830 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80197d0 con 0x55a90acf77d0 2024-04-26T20:28:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f5005600640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4ff00a3c30 0x7f4ff00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 -- 172.21.15.131:0/247619830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf77d0 msgr2=0x55a90acfc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf77d0 0x55a90acfc2d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009ca0 tx=0x7f4ff8011810 comp rx=0 tx=0).stop 2024-04-26T20:28:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f5005600640 1 -- 172.21.15.131:0/2720324168 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4ffc0d0c50 con 0x7f50081028d0 2024-04-26T20:28:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 -- 172.21.15.131:0/247619830 shutdown_connections 2024-04-26T20:28:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90acfedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf77d0 0x55a90acfc2d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 --2- 172.21.15.131:0/247619830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf6de0 0x55a90acf7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 -- 172.21.15.131:0/247619830 >> 172.21.15.131:0/247619830 conn(0x55a90acf2450 msgr2=0x55a90acf2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 -- 172.21.15.131:0/247619830 shutdown_connections 2024-04-26T20:28:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f500da00640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4ff00a3c30 0x7f4ff00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.928+0000 7f50127dd740 1 -- 172.21.15.131:0/247619830 wait complete. 2024-04-26T20:28:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.929+0000 7f50127dd740 1 -- start start 2024-04-26T20:28:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.929+0000 7f500da00640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4ff00a3c30 0x7f4ff00a60f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5000005e30 tx=0x7f5000031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.929+0000 7f50127dd740 1 Processor -- start 2024-04-26T20:28:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f50127dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad00fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f50127dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf77d0 0x55a90ad01500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f50127dd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90ad08540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f50127dd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a90acff8f0 con 0x55a90acfc940 2024-04-26T20:28:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f50127dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a90abe2c60 con 0x55a90acf77d0 2024-04-26T20:28:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f50127dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a90acffa70 con 0x55a90acf6de0 2024-04-26T20:28:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f50127dd740 10 client.0 _ll_register_callbacks cb 0x55a90ac0ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f500da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90ad08540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f500da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90ad08540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41912/0 (socket says 172.21.15.131:41912) 2024-04-26T20:28:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f500da00640 1 -- 172.21.15.131:0/1204668658 learned_addr learned my addr 172.21.15.131:0/1204668658 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f5007e00640 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf77d0 0x55a90ad01500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f500d000640 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad00fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f5007e00640 1 -- 172.21.15.131:0/1204668658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 msgr2=0x55a90ad00fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f5007e00640 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad00fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f5007e00640 1 -- 172.21.15.131:0/1204668658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 msgr2=0x55a90ad08540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f5007e00640 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90ad08540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.930+0000 7f5007e00640 1 -- 172.21.15.131:0/1204668658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90abf1510 con 0x55a90acf77d0 2024-04-26T20:28:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f5007e00640 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf77d0 0x55a90ad01500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0ce1b0 tx=0x7f4ffc09c5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36a18bcd753d4d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f4feaa00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:28:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f4feaa00640 1 -- 172.21.15.131:0/1204668658 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0a0d30 con 0x55a90acf77d0 2024-04-26T20:28:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f4feaa00640 1 -- 172.21.15.131:0/1204668658 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4ffc0ce400 con 0x55a90acf77d0 2024-04-26T20:28:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff80043c0 con 0x55a90acf77d0 2024-04-26T20:28:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f4feaa00640 1 -- 172.21.15.131:0/1204668658 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ffc0a55f0 con 0x55a90acf77d0 2024-04-26T20:28:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f50127dd740 10 client.26834 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a90a96d210 con 0x55a90acf77d0 2024-04-26T20:28:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f4feaa00640 1 -- 172.21.15.131:0/1204668658 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff80043c0 con 0x55a90acf77d0 2024-04-26T20:28:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.931+0000 7f4feaa00640 1 -- 172.21.15.131:0/1204668658 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4ffc09bdd0 con 0x55a90acf77d0 2024-04-26T20:28:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.932+0000 7f50127dd740 10 client.26834 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.932+0000 7f50127dd740 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.932+0000 7f50127dd740 20 client.26834 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.932+0000 7f50127dd740 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad0c9f0 0x55a90ad0ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.932+0000 7f50127dd740 4 client.26834 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.932+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a90ac68f90 con 0x55a90ad0c9f0 2024-04-26T20:28:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.932+0000 7f500d000640 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad0c9f0 0x55a90ad0ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:54.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.933+0000 7f500d000640 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad0c9f0 0x55a90ad0ee00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c806bc0544502cf server_cookie=ef42464f9326c909 in_seq=0 out_seq=0 2024-04-26T20:28:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.933+0000 7f4feaa00640 10 client.26834 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.933+0000 7f4feaa00640 1 -- 172.21.15.131:0/1204668658 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a90ac68f90 con 0x55a90ad0c9f0 2024-04-26T20:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.933+0000 7f4feaa00640 10 client.26834 handle_command_reply: tid=0 2024-04-26T20:28:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.933+0000 7f50127dd740 1 client.26834 shutdown 2024-04-26T20:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf77d0 msgr2=0x55a90ad01500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf77d0 0x55a90ad01500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0ce1b0 tx=0x7f4ffc09c5b0 comp rx=0 tx=0).stop 2024-04-26T20:28:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 shutdown_connections 2024-04-26T20:28:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad0c9f0 0x55a90ad0ee00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acfc940 0x55a90ad08540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acf77d0 0x55a90ad01500 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 --2- 172.21.15.131:0/1204668658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad00fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 >> 172.21.15.131:0/1204668658 conn(0x55a90acf2450 msgr2=0x55a90acf1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f4feaa00640 0 client.26834 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 shutdown_connections 2024-04-26T20:28:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 1 -- 172.21.15.131:0/1204668658 wait complete. 2024-04-26T20:28:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.934+0000 7f50127dd740 20 client.26834 trim_cache size 0 max 16384 2024-04-26T20:28:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 Processor -- start 2024-04-26T20:28:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 -- start start 2024-04-26T20:28:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad08500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50081b0010 con 0x55a90acf77d0 2024-04-26T20:28:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ad8e0 con 0x55a90acfc940 2024-04-26T20:28:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.950+0000 7f50127dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ab730 con 0x55a90acf6de0 2024-04-26T20:28:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f5007e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad08500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad08500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:56898/0 (socket says 172.21.15.131:56898) 2024-04-26T20:28:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f5007e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad14c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41918/0 (socket says 172.21.15.131:41918) 2024-04-26T20:28:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 -- 172.21.15.131:0/3673792564 learned_addr learned my addr 172.21.15.131:0/3673792564 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 -- 172.21.15.131:0/3673792564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 msgr2=0x55a90ad14710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad14710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 -- 172.21.15.131:0/3673792564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 msgr2=0x55a90ad14c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad14c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f500da00640 1 -- 172.21.15.131:0/3673792564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90aa39620 con 0x55a90acfc940 2024-04-26T20:28:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.951+0000 7f5007e00640 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad14c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.952+0000 7f500da00640 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad08500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5000006010 tx=0x7f5000041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb3d721331367d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.952+0000 7f4ff5600640 1 -- 172.21.15.131:0/3673792564 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000003970 con 0x55a90acfc940 2024-04-26T20:28:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.952+0000 7f4ff5600640 1 -- 172.21.15.131:0/3673792564 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5000036050 con 0x55a90acfc940 2024-04-26T20:28:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.952+0000 7f4ff5600640 1 -- 172.21.15.131:0/3673792564 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500002f610 con 0x55a90acfc940 2024-04-26T20:28:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 -- 172.21.15.131:0/3673792564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 msgr2=0x55a90ad08500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad08500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5000006010 tx=0x7f5000041000 comp rx=0 tx=0).stop 2024-04-26T20:28:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 -- 172.21.15.131:0/3673792564 shutdown_connections 2024-04-26T20:28:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad08500 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad14c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 --2- 172.21.15.131:0/3673792564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 -- 172.21.15.131:0/3673792564 >> 172.21.15.131:0/3673792564 conn(0x55a90acf2450 msgr2=0x55a90acf2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 -- 172.21.15.131:0/3673792564 shutdown_connections 2024-04-26T20:28:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 -- 172.21.15.131:0/3673792564 wait complete. 2024-04-26T20:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 -- start start 2024-04-26T20:28:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.953+0000 7f50127dd740 1 Processor -- start 2024-04-26T20:28:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f50127dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad05780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f50127dd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad05cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f50127dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad06ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f50127dd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50081abda0 con 0x55a90acf77d0 2024-04-26T20:28:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f50127dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ad210 con 0x55a90acfc940 2024-04-26T20:28:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f50127dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ac470 con 0x55a90acf6de0 2024-04-26T20:28:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f50127dd740 10 client.0 _ll_register_callbacks cb 0x7f4ffc004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f5007e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad05cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f5007e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad05cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41928/0 (socket says 172.21.15.131:41928) 2024-04-26T20:28:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f500d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad05780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f5007e00640 1 -- 172.21.15.131:0/1620154095 learned_addr learned my addr 172.21.15.131:0/1620154095 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f500da00640 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad06ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f500d000640 1 -- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 msgr2=0x55a90ad06ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.954+0000 7f500d000640 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad06ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f500d000640 1 -- 172.21.15.131:0/1620154095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 msgr2=0x55a90ad05cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f500d000640 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad05cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f500d000640 1 -- 172.21.15.131:0/1620154095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90ac15cb0 con 0x55a90acf6de0 2024-04-26T20:28:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f5007e00640 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad05cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f500d000640 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad05780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55a90acf33c0 tx=0x7f4ff8009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be26860eadc2d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f4feaa00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:28:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f4feaa00640 1 -- 172.21.15.131:0/1620154095 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff802bd60 con 0x55a90acf6de0 2024-04-26T20:28:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5000036050 con 0x55a90acf6de0 2024-04-26T20:28:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f4feaa00640 1 -- 172.21.15.131:0/1620154095 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4ff80050f0 con 0x55a90acf6de0 2024-04-26T20:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f4feaa00640 1 -- 172.21.15.131:0/1620154095 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ff803a050 con 0x55a90acf6de0 2024-04-26T20:28:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f50127dd740 10 client.26821 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.955+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a90ad081f0 con 0x55a90acf6de0 2024-04-26T20:28:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f4feaa00640 1 -- 172.21.15.131:0/1620154095 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5000036050 con 0x55a90acf6de0 2024-04-26T20:28:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f4feaa00640 1 -- 172.21.15.131:0/1620154095 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4ff802b230 con 0x55a90acf6de0 2024-04-26T20:28:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f50127dd740 10 client.26821 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f50127dd740 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f50127dd740 20 client.26821 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f50127dd740 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad7b790 0x55a90ad7db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f50127dd740 4 client.26821 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:28:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a90ac62e00 con 0x55a90ad7b790 2024-04-26T20:28:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.956+0000 7f5007e00640 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad7b790 0x55a90ad7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.957+0000 7f4feaa00640 10 client.26821 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.957+0000 7f5007e00640 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad7b790 0x55a90ad7db80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7496fafbf61540ae server_cookie=29ab9de95d6ad00a in_seq=0 out_seq=0 2024-04-26T20:28:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.963+0000 7f4feaa00640 1 -- 172.21.15.131:0/1620154095 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a90ac62e00 con 0x55a90ad7b790 2024-04-26T20:28:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.963+0000 7f4feaa00640 10 client.26821 handle_command_reply: tid=0 2024-04-26T20:28:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.963+0000 7f50127dd740 1 client.26821 shutdown 2024-04-26T20:28:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 msgr2=0x55a90ad05780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad05780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55a90acf33c0 tx=0x7f4ff8009f10 comp rx=0 tx=0).stop 2024-04-26T20:28:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 shutdown_connections 2024-04-26T20:28:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55a90ad7b790 0x55a90ad7db80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a90acfc940 0x55a90ad06ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f4feaa00640 0 client.26821 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90acf77d0 0x55a90ad05cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 --2- 172.21.15.131:0/1620154095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90acf6de0 0x55a90ad05780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 >> 172.21.15.131:0/1620154095 conn(0x55a90acf2450 msgr2=0x55a90ad00f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 shutdown_connections 2024-04-26T20:28:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 1 -- 172.21.15.131:0/1620154095 wait complete. 2024-04-26T20:28:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.964+0000 7f50127dd740 20 client.26821 trim_cache size 0 max 16384 2024-04-26T20:28:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:28:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:28:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:28:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:28:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4ff00a3c30 msgr2=0x7f4ff00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4ff00a3c30 0x7f4ff00a60f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5000005e30 tx=0x7f5000031af0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 msgr2=0x7f500819b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 0x7f500819b0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00ea60 tx=0x7f4ffc00ef20 comp rx=0 tx=0).stop 2024-04-26T20:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 shutdown_connections 2024-04-26T20:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5008103290 0x7f50081a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50081028d0 0x7f500819b0c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4ff00a3c30 0x7f4ff00a60f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 --2- 172.21.15.131:0/2720324168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50081016e0 0x7f500819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 >> 172.21.15.131:0/2720324168 conn(0x7f50080fcc70 msgr2=0x7f500810a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 shutdown_connections 2024-04-26T20:28:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:10.965+0000 7f500e400640 1 -- 172.21.15.131:0/2720324168 wait complete. 2024-04-26T20:28:56.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:11 smithi131 ceph-mon[27919]: pgmap v955: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:28:56.949 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:13 smithi131 ceph-mon[27919]: pgmap v956: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T20:28:56.957 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:15 smithi131 ceph-mon[27919]: pgmap v957: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:28:56.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:17 smithi131 ceph-mon[27919]: pgmap v958: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:56.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:19 smithi131 ceph-mon[27919]: pgmap v959: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:28:56.984 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:21 smithi131 ceph-mon[27919]: pgmap v960: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:28:57.042 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:23 smithi131 ceph-mon[27919]: pgmap v961: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:28:57.053 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:53 smithi148 ceph-mon[32281]: pgmap v976: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T20:28:57.062 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:55 smithi148 ceph-mon[32281]: pgmap v977: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:28:57.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:53 smithi157 ceph-mon[32703]: pgmap v976: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T20:28:57.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:55 smithi157 ceph-mon[32703]: pgmap v977: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:28:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.107+0000 7f7764c00640 1 -- 172.21.15.131:0/2637717444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 msgr2=0x7f77601058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.107+0000 7f7764c00640 1 --2- 172.21.15.131:0/2637717444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f77601058a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7750009900 tx=0x7f775002a410 comp rx=0 tx=0).stop 2024-04-26T20:28:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.107+0000 7f7764c00640 1 -- 172.21.15.131:0/2637717444 shutdown_connections 2024-04-26T20:28:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.107+0000 7f7764c00640 1 --2- 172.21.15.131:0/2637717444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7760108710 0x7f776010ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.107+0000 7f7764c00640 1 --2- 172.21.15.131:0/2637717444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7760105de0 0x7f77601081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.107+0000 7f7764c00640 1 --2- 172.21.15.131:0/2637717444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f77601058a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.107+0000 7f7764c00640 1 -- 172.21.15.131:0/2637717444 >> 172.21.15.131:0/2637717444 conn(0x7f77600fce00 msgr2=0x7f77600ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.108+0000 7f7764c00640 1 -- 172.21.15.131:0/2637717444 shutdown_connections 2024-04-26T20:28:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.108+0000 7f7764c00640 1 -- 172.21.15.131:0/2637717444 wait complete. 2024-04-26T20:28:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 Processor -- start 2024-04-26T20:28:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 -- start start 2024-04-26T20:28:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f776019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7760105de0 0x7f776019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7760108710 0x7f77601a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f776010e4e0 con 0x7f77601034b0 2024-04-26T20:28:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f776010e360 con 0x7f7760108710 2024-04-26T20:28:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f7764c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f776010e660 con 0x7f7760105de0 2024-04-26T20:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f775de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f776019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f775de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f776019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33854/0 (socket says 172.21.15.131:33854) 2024-04-26T20:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f775de00640 1 -- 172.21.15.131:0/247962341 learned_addr learned my addr 172.21.15.131:0/247962341 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f775d400640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7760105de0 0x7f776019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.109+0000 7f775e800640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7760108710 0x7f77601a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f775de00640 1 -- 172.21.15.131:0/247962341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7760105de0 msgr2=0x7f776019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f775de00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7760105de0 0x7f776019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f775de00640 1 -- 172.21.15.131:0/247962341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7760108710 msgr2=0x7f77601a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f775de00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7760108710 0x7f77601a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f775de00640 1 -- 172.21.15.131:0/247962341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77601a2990 con 0x7f77601034b0 2024-04-26T20:28:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f775de00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f776019ad70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7750005940 tx=0x7f77500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f7756a00640 1 -- 172.21.15.131:0/247962341 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775003f030 con 0x7f77601034b0 2024-04-26T20:28:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f7756a00640 1 -- 172.21.15.131:0/247962341 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7750003cb0 con 0x7f77601034b0 2024-04-26T20:28:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.110+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77601a2c20 con 0x7f77601034b0 2024-04-26T20:28:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.111+0000 7f7756a00640 1 -- 172.21.15.131:0/247962341 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750035070 con 0x7f77601034b0 2024-04-26T20:28:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.111+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77601a3110 con 0x7f77601034b0 2024-04-26T20:28:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f7756a00640 1 -- 172.21.15.131:0/247962341 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7750042030 con 0x7f77601034b0 2024-04-26T20:28:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 Processor -- start 2024-04-26T20:28:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 -- start start 2024-04-26T20:28:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5c99ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5c9f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9f730 0x55ffd5ca1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ffd5bfe380 con 0x55ffd5c99bd0 2024-04-26T20:28:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ffd5b9ac30 con 0x55ffd5c9f730 2024-04-26T20:28:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.114+0000 7f77679cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ffd5bfa340 con 0x55ffd5c9a5c0 2024-04-26T20:28:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5c99ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5c99ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33860/0 (socket says 172.21.15.131:33860) 2024-04-26T20:28:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775e800640 1 -- 172.21.15.131:0/526783712 learned_addr learned my addr 172.21.15.131:0/526783712 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775de00640 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5c9f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775d400640 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9f730 0x55ffd5ca1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775de00640 1 -- 172.21.15.131:0/526783712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9f730 msgr2=0x55ffd5ca1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775de00640 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9f730 0x55ffd5ca1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775de00640 1 -- 172.21.15.131:0/526783712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 msgr2=0x55ffd5c99ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775de00640 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5c99ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.115+0000 7f775de00640 1 -- 172.21.15.131:0/526783712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd5bbc940 con 0x55ffd5c9a5c0 2024-04-26T20:28:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.116+0000 7f775de00640 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5c9f0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f77500ee2d0 tx=0x7f77500f4f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd510a0bd1c527d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.116+0000 7f7756a00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77440a3cf0 0x7f77440a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.116+0000 7f774b000640 1 -- 172.21.15.131:0/526783712 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77500bbfd0 con 0x55ffd5c9a5c0 2024-04-26T20:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.116+0000 7f774b000640 1 -- 172.21.15.131:0/526783712 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7750003e60 con 0x55ffd5c9a5c0 2024-04-26T20:28:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.116+0000 7f774b000640 1 -- 172.21.15.131:0/526783712 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77500bc300 con 0x55ffd5c9a5c0 2024-04-26T20:28:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.116+0000 7f775e800640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77440a3cf0 0x7f77440a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.116+0000 7f7756a00640 1 -- 172.21.15.131:0/247962341 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f77500f0da0 con 0x7f77601034b0 2024-04-26T20:28:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 -- 172.21.15.131:0/526783712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9a5c0 msgr2=0x55ffd5c9f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5c9f0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f77500ee2d0 tx=0x7f77500f4f50 comp rx=0 tx=0).stop 2024-04-26T20:28:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f775e800640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77440a3cf0 0x7f77440a61b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55ffd5c9ae20 tx=0x7f7758009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 -- 172.21.15.131:0/526783712 shutdown_connections 2024-04-26T20:28:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9f730 0x55ffd5ca1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5c9f0c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 --2- 172.21.15.131:0/526783712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5c99ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 -- 172.21.15.131:0/526783712 >> 172.21.15.131:0/526783712 conn(0x55ffd5c95240 msgr2=0x55ffd5c95640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 -- 172.21.15.131:0/526783712 shutdown_connections 2024-04-26T20:28:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 -- 172.21.15.131:0/526783712 wait complete. 2024-04-26T20:28:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.117+0000 7f77679cc740 1 -- start start 2024-04-26T20:28:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 1 Processor -- start 2024-04-26T20:28:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5caaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ffd5ca24a0 con 0x55ffd5c9f730 2024-04-26T20:28:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ffd5b85c60 con 0x55ffd5c9a5c0 2024-04-26T20:28:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ffd5ca2620 con 0x55ffd5c99bd0 2024-04-26T20:28:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f77679cc740 10 client.0 _ll_register_callbacks cb 0x55ffd5bb1e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.118+0000 7f775e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5caaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5caaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33870/0 (socket says 172.21.15.131:33870) 2024-04-26T20:28:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 -- 172.21.15.131:0/1979217454 learned_addr learned my addr 172.21.15.131:0/1979217454 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775d400640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775de00640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 -- 172.21.15.131:0/1979217454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 msgr2=0x55ffd5ca39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 -- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 msgr2=0x55ffd5ca3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 -- 172.21.15.131:0/1979217454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd5b94510 con 0x55ffd5c9f730 2024-04-26T20:28:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775d400640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775de00640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:28:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f775e800640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5caaf60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7758009810 tx=0x7f7758004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b080a73cdac6cda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.119+0000 7f773f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:28:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f773f400640 1 -- 172.21.15.131:0/1979217454 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758003970 con 0x55ffd5c9f730 2024-04-26T20:28:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f773f400640 1 -- 172.21.15.131:0/1979217454 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7758017030 con 0x55ffd5c9f730 2024-04-26T20:28:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7750003e60 con 0x55ffd5c9f730 2024-04-26T20:28:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f773f400640 1 -- 172.21.15.131:0/1979217454 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7758032050 con 0x55ffd5c9f730 2024-04-26T20:28:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f77679cc740 10 client.16731 fetch_fsmap learned FSMap version 27 2024-04-26T20:28:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffd5910210 con 0x55ffd5c9f730 2024-04-26T20:28:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f773f400640 1 -- 172.21.15.131:0/1979217454 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7750003e60 con 0x55ffd5c9f730 2024-04-26T20:28:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.120+0000 7f773f400640 1 -- 172.21.15.131:0/1979217454 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f77580040c0 con 0x55ffd5c9f730 2024-04-26T20:28:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.121+0000 7f77679cc740 10 client.16731 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:28:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.121+0000 7f77679cc740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:28:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.121+0000 7f77679cc740 20 client.16731 populate_metadata read hostname 'smithi131' 2024-04-26T20:28:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.121+0000 7f77679cc740 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5caf520 0x55ffd5cb1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.121+0000 7f77679cc740 4 client.16731 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.121+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd5c0bf90 con 0x55ffd5caf520 2024-04-26T20:28:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.121+0000 7f775de00640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5caf520 0x55ffd5cb1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.122+0000 7f775de00640 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5caf520 0x55ffd5cb1970 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c853251b74092a6c server_cookie=12b06bd72e2e85f8 in_seq=0 out_seq=0 2024-04-26T20:28:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.122+0000 7f773f400640 10 client.16731 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.122+0000 7f773f400640 1 -- 172.21.15.131:0/1979217454 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffd5c0bf90 con 0x55ffd5caf520 2024-04-26T20:28:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.122+0000 7f773f400640 10 client.16731 handle_command_reply: tid=0 2024-04-26T20:28:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 client.16731 shutdown 2024-04-26T20:28:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 msgr2=0x55ffd5caaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5caaf60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7758009810 tx=0x7f7758004ab0 comp rx=0 tx=0).stop 2024-04-26T20:28:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 shutdown_connections 2024-04-26T20:28:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5caf520 0x55ffd5cb1970 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5caaf60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 --2- 172.21.15.131:0/1979217454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f773f400640 0 client.16731 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:28:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 >> 172.21.15.131:0/1979217454 conn(0x55ffd5c95240 msgr2=0x55ffd5c94440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 shutdown_connections 2024-04-26T20:28:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 1 -- 172.21.15.131:0/1979217454 wait complete. 2024-04-26T20:28:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.123+0000 7f77679cc740 20 client.16731 trim_cache size 0 max 16384 2024-04-26T20:28:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 Processor -- start 2024-04-26T20:28:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 -- start start 2024-04-26T20:28:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5caaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9f730 0x55ffd5c9b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77601b0120 con 0x55ffd5c99bd0 2024-04-26T20:28:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77601ad9f0 con 0x55ffd5c9a5c0 2024-04-26T20:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.139+0000 7f77679cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77601ab840 con 0x55ffd5c9f730 2024-04-26T20:28:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33880/0 (socket says 172.21.15.131:33880) 2024-04-26T20:28:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 -- 172.21.15.131:0/3248053375 learned_addr learned my addr 172.21.15.131:0/3248053375 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:28:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775d400640 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5caaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775e800640 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9f730 0x55ffd5c9b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 -- 172.21.15.131:0/3248053375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9f730 msgr2=0x55ffd5c9b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9f730 0x55ffd5c9b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 -- 172.21.15.131:0/3248053375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 msgr2=0x55ffd5caaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5caaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 -- 172.21.15.131:0/3248053375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd59dc620 con 0x55ffd5c99bd0 2024-04-26T20:29:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.140+0000 7f775de00640 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca3000 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f77500dd010 tx=0x7f7750009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673298ccb8d282b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f774a600640 1 -- 172.21.15.131:0/3248053375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750049070 con 0x55ffd5c99bd0 2024-04-26T20:29:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f774a600640 1 -- 172.21.15.131:0/3248053375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7750003e60 con 0x55ffd5c99bd0 2024-04-26T20:29:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f774a600640 1 -- 172.21.15.131:0/3248053375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77500bc4e0 con 0x55ffd5c99bd0 2024-04-26T20:29:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 -- 172.21.15.131:0/3248053375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 msgr2=0x55ffd5ca3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca3000 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f77500dd010 tx=0x7f7750009680 comp rx=0 tx=0).stop 2024-04-26T20:29:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 -- 172.21.15.131:0/3248053375 shutdown_connections 2024-04-26T20:29:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c9f730 0x55ffd5c9b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5caaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 --2- 172.21.15.131:0/3248053375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca3000 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 -- 172.21.15.131:0/3248053375 >> 172.21.15.131:0/3248053375 conn(0x55ffd5c95240 msgr2=0x55ffd5ca39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 -- 172.21.15.131:0/3248053375 shutdown_connections 2024-04-26T20:29:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.141+0000 7f77679cc740 1 -- 172.21.15.131:0/3248053375 wait complete. 2024-04-26T20:29:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 -- start start 2024-04-26T20:29:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 Processor -- start 2024-04-26T20:29:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5d13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77601abeb0 con 0x55ffd5c9f730 2024-04-26T20:29:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77601ad320 con 0x55ffd5c9a5c0 2024-04-26T20:29:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77601ac580 con 0x55ffd5c99bd0 2024-04-26T20:29:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.142+0000 7f77679cc740 10 client.0 _ll_register_callbacks cb 0x7f7758009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5d13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5d13ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33894/0 (socket says 172.21.15.131:33894) 2024-04-26T20:29:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775e800640 1 -- 172.21.15.131:0/371118804 learned_addr learned my addr 172.21.15.131:0/371118804 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775de00640 1 -- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 msgr2=0x55ffd5ca8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775de00640 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775de00640 1 -- 172.21.15.131:0/371118804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 msgr2=0x55ffd5d13ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775de00640 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5d13ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775de00640 1 -- 172.21.15.131:0/371118804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd5bb8cb0 con 0x55ffd5c99bd0 2024-04-26T20:29:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775d400640 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775e800640 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5d13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f775de00640 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca7bc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77500f4e60 tx=0x7f775003b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a3fe7e84a364d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f773f400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:29:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.143+0000 7f773f400640 1 -- 172.21.15.131:0/371118804 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750049040 con 0x55ffd5c99bd0 2024-04-26T20:29:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f773f400640 1 -- 172.21.15.131:0/371118804 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f77500fd040 con 0x55ffd5c99bd0 2024-04-26T20:29:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7750003e60 con 0x55ffd5c99bd0 2024-04-26T20:29:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f773f400640 1 -- 172.21.15.131:0/371118804 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77500bc6c0 con 0x55ffd5c99bd0 2024-04-26T20:29:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 10 client.26839 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffd5d14eb0 con 0x55ffd5c99bd0 2024-04-26T20:29:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f773f400640 1 -- 172.21.15.131:0/371118804 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7750003e60 con 0x55ffd5c99bd0 2024-04-26T20:29:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f773f400640 1 -- 172.21.15.131:0/371118804 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f77500bc170 con 0x55ffd5c99bd0 2024-04-26T20:29:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 10 client.26839 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 10 client.26839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 20 client.26839 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5d15470 0x55ffd5d17860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 4 client.26839 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:29:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.144+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffd5c05e00 con 0x55ffd5d15470 2024-04-26T20:29:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.145+0000 7f775d400640 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5d15470 0x55ffd5d17860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.145+0000 7f773f400640 10 client.26839 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.145+0000 7f775d400640 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5d15470 0x55ffd5d17860 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53223c830cdde2bd server_cookie=d52bd5fd8d2ffeed in_seq=0 out_seq=0 2024-04-26T20:29:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f773f400640 1 -- 172.21.15.131:0/371118804 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffd5c05e00 con 0x55ffd5d15470 2024-04-26T20:29:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f773f400640 10 client.26839 handle_command_reply: tid=0 2024-04-26T20:29:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 client.26839 shutdown 2024-04-26T20:29:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 msgr2=0x55ffd5ca7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca7bc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77500f4e60 tx=0x7f775003b6b0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 shutdown_connections 2024-04-26T20:29:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ffd5d15470 0x55ffd5d17860 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffd5c9f730 0x55ffd5d13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffd5c9a5c0 0x55ffd5ca8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 --2- 172.21.15.131:0/371118804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffd5c99bd0 0x55ffd5ca7bc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f773f400640 0 client.26839 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.152+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 >> 172.21.15.131:0/371118804 conn(0x55ffd5c95240 msgr2=0x55ffd5ca39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 shutdown_connections 2024-04-26T20:29:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f77679cc740 1 -- 172.21.15.131:0/371118804 wait complete. 2024-04-26T20:29:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f77679cc740 20 client.26839 trim_cache size 0 max 16384 2024-04-26T20:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:29:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:29:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:29:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:29:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77440a3cf0 msgr2=0x7f77440a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77440a3cf0 0x7f77440a61b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55ffd5c9ae20 tx=0x7f7758009500 comp rx=0 tx=0).stop 2024-04-26T20:29:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 msgr2=0x7f776019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f776019ad70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7750005940 tx=0x7f77500042f0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 shutdown_connections 2024-04-26T20:29:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7760108710 0x7f77601a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f77440a3cf0 0x7f77440a61b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7760105de0 0x7f776019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 --2- 172.21.15.131:0/247962341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77601034b0 0x7f776019ad70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.153+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 >> 172.21.15.131:0/247962341 conn(0x7f77600fce00 msgr2=0x7f7760102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.154+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 shutdown_connections 2024-04-26T20:29:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:24.154+0000 7f7764c00640 1 -- 172.21.15.131:0/247962341 wait complete. 2024-04-26T20:29:01.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:25 smithi131 ceph-mon[27919]: pgmap v962: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:29:01.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:27 smithi131 ceph-mon[27919]: pgmap v963: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:01.762 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:29 smithi131 ceph-mon[27919]: pgmap v964: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:29:01.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:31 smithi131 ceph-mon[27919]: pgmap v965: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:29:01.776 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:33 smithi131 ceph-mon[27919]: pgmap v966: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:29:01.782 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:35 smithi131 ceph-mon[27919]: pgmap v967: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:29:01.791 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:57 smithi148 ceph-mon[32281]: pgmap v978: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:01.847 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:28:59 smithi148 ceph-mon[32281]: pgmap v979: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:29:01.855 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:57 smithi157 ceph-mon[32703]: pgmap v978: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:01.863 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:28:59 smithi157 ceph-mon[32703]: pgmap v979: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:29:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.291+0000 7f0d47e00640 1 -- 172.21.15.131:0/778624634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d48108de0 msgr2=0x7f0d48109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.291+0000 7f0d47e00640 1 --2- 172.21.15.131:0/778624634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d48108de0 0x7f0d48109200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009860 tx=0x7f0d3802a340 comp rx=0 tx=0).stop 2024-04-26T20:29:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.292+0000 7f0d47e00640 1 -- 172.21.15.131:0/778624634 shutdown_connections 2024-04-26T20:29:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.292+0000 7f0d47e00640 1 --2- 172.21.15.131:0/778624634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4810a990 0x7f0d48111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.292+0000 7f0d47e00640 1 --2- 172.21.15.131:0/778624634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d48109fd0 0x7f0d4810a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.292+0000 7f0d47e00640 1 --2- 172.21.15.131:0/778624634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d48108de0 0x7f0d48109200 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.292+0000 7f0d47e00640 1 -- 172.21.15.131:0/778624634 >> 172.21.15.131:0/778624634 conn(0x7f0d48077dc0 msgr2=0x7f0d4807a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.292+0000 7f0d47e00640 1 -- 172.21.15.131:0/778624634 shutdown_connections 2024-04-26T20:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.292+0000 7f0d47e00640 1 -- 172.21.15.131:0/778624634 wait complete. 2024-04-26T20:29:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 Processor -- start 2024-04-26T20:29:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 -- start start 2024-04-26T20:29:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d48108de0 0x7f0d4819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d48109fd0 0x7f0d4819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4810a990 0x7f0d481a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48114910 con 0x7f0d48109fd0 2024-04-26T20:29:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48114790 con 0x7f0d48108de0 2024-04-26T20:29:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48114a90 con 0x7f0d4810a990 2024-04-26T20:29:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d3e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d48109fd0 0x7f0d4819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d47400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4810a990 0x7f0d481a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d3e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d48109fd0 0x7f0d4819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39064/0 (socket says 172.21.15.131:39064) 2024-04-26T20:29:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.293+0000 7f0d3e000640 1 -- 172.21.15.131:0/4003092729 learned_addr learned my addr 172.21.15.131:0/4003092729 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d46a00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d48108de0 0x7f0d4819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d47400640 1 -- 172.21.15.131:0/4003092729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d48108de0 msgr2=0x7f0d4819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d47400640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d48108de0 0x7f0d4819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d47400640 1 -- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d48109fd0 msgr2=0x7f0d4819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d47400640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d48109fd0 0x7f0d4819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d47400640 1 -- 172.21.15.131:0/4003092729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d481a6c10 con 0x7f0d4810a990 2024-04-26T20:29:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d46a00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d48108de0 0x7f0d4819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d47400640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4810a990 0x7f0d481a65a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009fd0 tx=0x7f0d4000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d3fe00640 1 -- 172.21.15.131:0/4003092729 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4000ce20 con 0x7f0d4810a990 2024-04-26T20:29:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d3fe00640 1 -- 172.21.15.131:0/4003092729 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d40005020 con 0x7f0d4810a990 2024-04-26T20:29:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d3fe00640 1 -- 172.21.15.131:0/4003092729 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40010690 con 0x7f0d4810a990 2024-04-26T20:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.294+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d481a6f00 con 0x7f0d4810a990 2024-04-26T20:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.295+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a73c0 con 0x7f0d4810a990 2024-04-26T20:29:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.298+0000 7f0d507ff740 1 Processor -- start 2024-04-26T20:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.298+0000 7f0d3fe00640 1 -- 172.21.15.131:0/4003092729 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0d400054d0 con 0x7f0d4810a990 2024-04-26T20:29:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.298+0000 7f0d507ff740 1 -- start start 2024-04-26T20:29:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f651422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f65144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d507ff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555f650a1380 con 0x555f65142940 2024-04-26T20:29:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d507ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555f6503dc30 con 0x555f6513cde0 2024-04-26T20:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d507ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f6509d340 con 0x555f6513d7d0 2024-04-26T20:29:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f651422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f651422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40738/0 (socket says 172.21.15.131:40738) 2024-04-26T20:29:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 -- 172.21.15.131:0/1487093685 learned_addr learned my addr 172.21.15.131:0/1487093685 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d3e000640 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d3e000640 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f65144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 -- 172.21.15.131:0/1487093685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 msgr2=0x555f6513d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 -- 172.21.15.131:0/1487093685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 msgr2=0x555f65144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f65144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.299+0000 7f0d46a00640 1 -- 172.21.15.131:0/1487093685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6505f940 con 0x555f6513d7d0 2024-04-26T20:29:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.300+0000 7f0d46a00640 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f651422d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d38030b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e879a43ff239fb19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.300+0000 7f0d37e00640 1 -- 172.21.15.131:0/1487093685 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38041020 con 0x555f6513d7d0 2024-04-26T20:29:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.300+0000 7f0d37e00640 1 -- 172.21.15.131:0/1487093685 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d38004040 con 0x555f6513d7d0 2024-04-26T20:29:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.300+0000 7f0d37e00640 1 -- 172.21.15.131:0/1487093685 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38037040 con 0x555f6513d7d0 2024-04-26T20:29:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.300+0000 7f0d3fe00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0d280a3c30 0x7f0d280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.300+0000 7f0d3fe00640 1 -- 172.21.15.131:0/4003092729 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0d400d1120 con 0x7f0d4810a990 2024-04-26T20:29:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.300+0000 7f0d3e000640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0d280a3c30 0x7f0d280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 -- 172.21.15.131:0/1487093685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 msgr2=0x555f651422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f651422d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d38030b50 comp rx=0 tx=0).stop 2024-04-26T20:29:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 -- 172.21.15.131:0/1487093685 shutdown_connections 2024-04-26T20:29:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f65144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f651422d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1487093685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 -- 172.21.15.131:0/1487093685 >> 172.21.15.131:0/1487093685 conn(0x555f65138450 msgr2=0x555f65138850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 -- 172.21.15.131:0/1487093685 shutdown_connections 2024-04-26T20:29:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 -- 172.21.15.131:0/1487093685 wait complete. 2024-04-26T20:29:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d3e000640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0d280a3c30 0x7f0d280a60f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x555f6513e030 tx=0x7f0d30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.301+0000 7f0d507ff740 1 -- start start 2024-04-26T20:29:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 1 Processor -- start 2024-04-26T20:29:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f65146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f65147500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f6514e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555f65145890 con 0x555f65142940 2024-04-26T20:29:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555f65028c60 con 0x555f6513cde0 2024-04-26T20:29:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555f65145a10 con 0x555f6513d7d0 2024-04-26T20:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d507ff740 10 client.0 _ll_register_callbacks cb 0x555f65054e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d46a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f65146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d47400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f6514e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.302+0000 7f0d3e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f65147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d3e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f65147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40754/0 (socket says 172.21.15.131:40754) 2024-04-26T20:29:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d3e000640 1 -- 172.21.15.131:0/928939213 learned_addr learned my addr 172.21.15.131:0/928939213 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d46a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f65146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:38900/0 (socket says 172.21.15.131:38900) 2024-04-26T20:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d46a00640 1 -- 172.21.15.131:0/928939213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 msgr2=0x555f65147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d46a00640 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f65147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d46a00640 1 -- 172.21.15.131:0/928939213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 msgr2=0x555f6514e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d46a00640 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f6514e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d46a00640 1 -- 172.21.15.131:0/928939213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f65037510 con 0x555f6513cde0 2024-04-26T20:29:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d3e000640 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f65147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d46a00640 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f65146fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x555f651393c0 tx=0x7f0d38005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b739444944a95f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.303+0000 7f0d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:29:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d34c00640 1 -- 172.21.15.131:0/928939213 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38037050 con 0x555f6513cde0 2024-04-26T20:29:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d34c00640 1 -- 172.21.15.131:0/928939213 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0d38030550 con 0x555f6513cde0 2024-04-26T20:29:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d38004040 con 0x555f6513cde0 2024-04-26T20:29:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d34c00640 1 -- 172.21.15.131:0/928939213 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d38059050 con 0x555f6513cde0 2024-04-26T20:29:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d507ff740 10 client.26870 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f64db3210 con 0x555f6513cde0 2024-04-26T20:29:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d34c00640 1 -- 172.21.15.131:0/928939213 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d38004040 con 0x555f6513cde0 2024-04-26T20:29:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d34c00640 1 -- 172.21.15.131:0/928939213 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0d380046c0 con 0x555f6513cde0 2024-04-26T20:29:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d507ff740 10 client.26870 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.304+0000 7f0d507ff740 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.305+0000 7f0d507ff740 20 client.26870 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.305+0000 7f0d507ff740 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651526a0 0x555f65154a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.305+0000 7f0d507ff740 4 client.26870 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.305+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f650aef90 con 0x555f651526a0 2024-04-26T20:29:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.305+0000 7f0d3e000640 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651526a0 0x555f65154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.305+0000 7f0d3e000640 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651526a0 0x555f65154a90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b537fd8a0916a99 server_cookie=32ce5b28484cca9f in_seq=0 out_seq=0 2024-04-26T20:29:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.305+0000 7f0d34c00640 10 client.26870 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.306+0000 7f0d34c00640 1 -- 172.21.15.131:0/928939213 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f650aef90 con 0x555f651526a0 2024-04-26T20:29:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.306+0000 7f0d34c00640 10 client.26870 handle_command_reply: tid=0 2024-04-26T20:29:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 client.26870 shutdown 2024-04-26T20:29:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 msgr2=0x555f65146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f65146fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x555f651393c0 tx=0x7f0d38005f10 comp rx=0 tx=0).stop 2024-04-26T20:29:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 shutdown_connections 2024-04-26T20:29:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651526a0 0x555f65154a90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f65142940 0x555f6514e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f6513d7d0 0x555f65147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 --2- 172.21.15.131:0/928939213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f65146fc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 >> 172.21.15.131:0/928939213 conn(0x555f65138450 msgr2=0x555f65137650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d34c00640 0 client.26870 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 shutdown_connections 2024-04-26T20:29:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 1 -- 172.21.15.131:0/928939213 wait complete. 2024-04-26T20:29:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.307+0000 7f0d507ff740 20 client.26870 trim_cache size 0 max 16384 2024-04-26T20:29:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 Processor -- start 2024-04-26T20:29:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 -- start start 2024-04-26T20:29:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f651465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f65146b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481b4330 con 0x555f6513d7d0 2024-04-26T20:29:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481b1c00 con 0x555f6513cde0 2024-04-26T20:29:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.323+0000 7f0d507ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481afa50 con 0x555f65142940 2024-04-26T20:29:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d3e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f651465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:38916/0 (socket says 172.21.15.131:38916) 2024-04-26T20:29:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 -- 172.21.15.131:0/2301350074 learned_addr learned my addr 172.21.15.131:0/2301350074 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d47400640 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f65146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 -- 172.21.15.131:0/2301350074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 msgr2=0x555f65146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f65146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 -- 172.21.15.131:0/2301350074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 msgr2=0x555f651465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f651465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 -- 172.21.15.131:0/2301350074 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f64e7f620 con 0x555f6513cde0 2024-04-26T20:29:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d47400640 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f65146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.324+0000 7f0d46a00640 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513bb10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0d38005e00 tx=0x7f0d38030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284e3131a9c73973 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d37400640 1 -- 172.21.15.131:0/2301350074 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38037070 con 0x555f6513cde0 2024-04-26T20:29:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d37400640 1 -- 172.21.15.131:0/2301350074 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d380046c0 con 0x555f6513cde0 2024-04-26T20:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d37400640 1 -- 172.21.15.131:0/2301350074 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38041040 con 0x555f6513cde0 2024-04-26T20:29:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 -- 172.21.15.131:0/2301350074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 msgr2=0x555f6513bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513bb10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0d38005e00 tx=0x7f0d38030b50 comp rx=0 tx=0).stop 2024-04-26T20:29:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 -- 172.21.15.131:0/2301350074 shutdown_connections 2024-04-26T20:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f65146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f651465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 --2- 172.21.15.131:0/2301350074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6513bb10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 -- 172.21.15.131:0/2301350074 >> 172.21.15.131:0/2301350074 conn(0x555f65138450 msgr2=0x555f6513b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 -- 172.21.15.131:0/2301350074 shutdown_connections 2024-04-26T20:29:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.325+0000 7f0d507ff740 1 -- 172.21.15.131:0/2301350074 wait complete. 2024-04-26T20:29:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 -- start start 2024-04-26T20:29:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 Processor -- start 2024-04-26T20:29:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6514a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f6514aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f651b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481b00c0 con 0x555f6513d7d0 2024-04-26T20:29:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481b1530 con 0x555f6513cde0 2024-04-26T20:29:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.326+0000 7f0d507ff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481b0790 con 0x555f65142940 2024-04-26T20:29:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d507ff740 10 client.0 _ll_register_callbacks cb 0x7f0d30005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f6514aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d46a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6514a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f6514aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39106/0 (socket says 172.21.15.131:39106) 2024-04-26T20:29:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 -- 172.21.15.131:0/1223763004 learned_addr learned my addr 172.21.15.131:0/1223763004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d47400640 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f651b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 -- 172.21.15.131:0/1223763004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 msgr2=0x555f651b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f651b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 -- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 msgr2=0x555f6514a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6514a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 -- 172.21.15.131:0/1223763004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6505bcb0 con 0x555f6513d7d0 2024-04-26T20:29:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d46a00640 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6514a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d3e000640 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f6514aee0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009dd0 tx=0x7f0d30009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d624a5f6278888e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.327+0000 7f0d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d34c00640 1 -- 172.21.15.131:0/1223763004 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30030020 con 0x555f6513d7d0 2024-04-26T20:29:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d34c00640 1 -- 172.21.15.131:0/1223763004 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f0d3002c610 con 0x555f6513d7d0 2024-04-26T20:29:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d380046c0 con 0x555f6513d7d0 2024-04-26T20:29:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d34c00640 1 -- 172.21.15.131:0/1223763004 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d3003b050 con 0x555f6513d7d0 2024-04-26T20:29:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d507ff740 10 client.16755 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f651b81e0 con 0x555f6513d7d0 2024-04-26T20:29:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d34c00640 1 -- 172.21.15.131:0/1223763004 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d380046c0 con 0x555f6513d7d0 2024-04-26T20:29:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.328+0000 7f0d34c00640 1 -- 172.21.15.131:0/1223763004 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0d3001e550 con 0x555f6513d7d0 2024-04-26T20:29:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d507ff740 10 client.16755 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d507ff740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d507ff740 20 client.16755 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651b8b70 0x555f651baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d507ff740 4 client.16755 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:29:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f650a8e00 con 0x555f651b8b70 2024-04-26T20:29:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d46a00640 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651b8b70 0x555f651baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d46a00640 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651b8b70 0x555f651baf90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d8beb2123c8969 server_cookie=d73954280ca638d1 in_seq=0 out_seq=0 2024-04-26T20:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.329+0000 7f0d34c00640 10 client.16755 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d34c00640 1 -- 172.21.15.131:0/1223763004 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f650a8e00 con 0x555f651b8b70 2024-04-26T20:29:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d34c00640 10 client.16755 handle_command_reply: tid=0 2024-04-26T20:29:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 client.16755 shutdown 2024-04-26T20:29:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 msgr2=0x555f6514aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f6514aee0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009dd0 tx=0x7f0d30009800 comp rx=0 tx=0).stop 2024-04-26T20:29:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 shutdown_connections 2024-04-26T20:29:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x555f651b8b70 0x555f651baf90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f65142940 0x555f651b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f6513d7d0 0x555f6514aee0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 --2- 172.21.15.131:0/1223763004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555f6513cde0 0x555f6514a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 >> 172.21.15.131:0/1223763004 conn(0x555f65138450 msgr2=0x555f65136dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d34c00640 0 client.16755 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 shutdown_connections 2024-04-26T20:29:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 1 -- 172.21.15.131:0/1223763004 wait complete. 2024-04-26T20:29:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.334+0000 7f0d507ff740 20 client.16755 trim_cache size 0 max 16384 2024-04-26T20:29:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:29:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:29:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:29:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.335+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0d280a3c30 msgr2=0x7f0d280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.335+0000 7f0d47e00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0d280a3c30 0x7f0d280a60f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x555f6513e030 tx=0x7f0d30009260 comp rx=0 tx=0).stop 2024-04-26T20:29:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.335+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4810a990 msgr2=0x7f0d481a65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.335+0000 7f0d47e00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4810a990 0x7f0d481a65a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009fd0 tx=0x7f0d4000ef10 comp rx=0 tx=0).stop 2024-04-26T20:29:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 shutdown_connections 2024-04-26T20:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f0d280a3c30 0x7f0d280a60f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4810a990 0x7f0d481a65a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d48109fd0 0x7f0d4819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 --2- 172.21.15.131:0/4003092729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d48108de0 0x7f0d4819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 >> 172.21.15.131:0/4003092729 conn(0x7f0d48077dc0 msgr2=0x7f0d4810d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 shutdown_connections 2024-04-26T20:29:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:37.336+0000 7f0d47e00640 1 -- 172.21.15.131:0/4003092729 wait complete. 2024-04-26T20:29:06.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:01 smithi148 ceph-mon[32281]: pgmap v980: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:29:06.571 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:02 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:29:06.593 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:03 smithi148 ceph-mon[32281]: pgmap v981: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:06.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:29:06.607 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:29:06.612 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:29:06.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:05 smithi148 ceph-mon[32281]: pgmap v982: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T20:29:06.628 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:01 smithi157 ceph-mon[32703]: pgmap v980: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:29:06.676 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:02 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:29:06.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:03 smithi157 ceph-mon[32703]: pgmap v981: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:06.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:29:06.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:29:06.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:29:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:05 smithi157 ceph-mon[32703]: pgmap v982: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T20:29:06.797 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:37 smithi131 ceph-mon[27919]: pgmap v968: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T20:29:06.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:39 smithi131 ceph-mon[27919]: pgmap v969: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T20:29:06.811 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:41 smithi131 ceph-mon[27919]: pgmap v970: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T20:29:06.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 -- 172.21.15.131:0/1309646422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474103410 msgr2=0x7f5474105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 --2- 172.21.15.131:0/1309646422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474103410 0x7f5474105800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5464009860 tx=0x7f546402a340 comp rx=0 tx=0).stop 2024-04-26T20:29:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 -- 172.21.15.131:0/1309646422 shutdown_connections 2024-04-26T20:29:06.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 --2- 172.21.15.131:0/1309646422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474108670 0x7f547410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 --2- 172.21.15.131:0/1309646422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5474105d40 0x7f5474108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 --2- 172.21.15.131:0/1309646422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474103410 0x7f5474105800 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:06.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 -- 172.21.15.131:0/1309646422 >> 172.21.15.131:0/1309646422 conn(0x7f54740fce20 msgr2=0x7f54740ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:06.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 -- 172.21.15.131:0/1309646422 shutdown_connections 2024-04-26T20:29:07.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.521+0000 7f5479600640 1 -- 172.21.15.131:0/1309646422 wait complete. 2024-04-26T20:29:07.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.522+0000 7f5479600640 1 Processor -- start 2024-04-26T20:29:07.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.522+0000 7f5479600640 1 -- start start 2024-04-26T20:29:07.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5479600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474103410 0x7f5474198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:07.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5479600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474105d40 0x7f5474199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:07.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5479600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5474108670 0x7f54741a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:07.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5479600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f547410e4c0 con 0x7f5474103410 2024-04-26T20:29:07.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5479600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f547410e340 con 0x7f5474105d40 2024-04-26T20:29:07.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5479600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f547410e640 con 0x7f5474108670 2024-04-26T20:29:07.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5472a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474103410 0x7f5474198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5472a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474103410 0x7f5474198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39120/0 (socket says 172.21.15.131:39120) 2024-04-26T20:29:07.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5472a00640 1 -- 172.21.15.131:0/264678475 learned_addr learned my addr 172.21.15.131:0/264678475 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:07.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f5473400640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5474108670 0x7f54741a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.523+0000 7f546be00640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474105d40 0x7f5474199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:07.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.524+0000 7f5472a00640 1 -- 172.21.15.131:0/264678475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5474108670 msgr2=0x7f54741a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.524+0000 7f5472a00640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5474108670 0x7f54741a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:07.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.524+0000 7f5472a00640 1 -- 172.21.15.131:0/264678475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474105d40 msgr2=0x7f5474199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:07.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.524+0000 7f5472a00640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474105d40 0x7f5474199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.524+0000 7f5472a00640 1 -- 172.21.15.131:0/264678475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54741a0860 con 0x7f5474103410 2024-04-26T20:29:07.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.524+0000 7f5473400640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5474108670 0x7f54741a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:07.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.524+0000 7f5472a00640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474103410 0x7f5474198c50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f54640057c0 tx=0x7f5464032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:07.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.525+0000 7f546b400640 1 -- 172.21.15.131:0/264678475 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464037040 con 0x7f5474103410 2024-04-26T20:29:07.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.525+0000 7f546b400640 1 -- 172.21.15.131:0/264678475 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5464030bd0 con 0x7f5474103410 2024-04-26T20:29:07.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.525+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54741a0af0 con 0x7f5474103410 2024-04-26T20:29:07.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.525+0000 7f546b400640 1 -- 172.21.15.131:0/264678475 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464005280 con 0x7f5474103410 2024-04-26T20:29:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.525+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a0fd0 con 0x7f5474103410 2024-04-26T20:29:07.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 Processor -- start 2024-04-26T20:29:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f546b400640 1 -- 172.21.15.131:0/264678475 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5464043020 con 0x7f5474103410 2024-04-26T20:29:07.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 -- start start 2024-04-26T20:29:07.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacbfdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:07.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc02d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc05850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:07.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563dacb62980 con 0x563dacbfe250 2024-04-26T20:29:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563dacaff220 con 0x563dacc033c0 2024-04-26T20:29:07.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.528+0000 7f547c5b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563dacb5e940 con 0x563dacbfd860 2024-04-26T20:29:07.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f5472a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc02d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:07.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f546be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc05850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:07.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f5472a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc02d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39132/0 (socket says 172.21.15.131:39132) 2024-04-26T20:29:07.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f5473400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacbfdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f5472a00640 1 -- 172.21.15.131:0/2851232605 learned_addr learned my addr 172.21.15.131:0/2851232605 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:07.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f546be00640 1 -- 172.21.15.131:0/2851232605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 msgr2=0x563dacbfdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f546be00640 1 --2- 172.21.15.131:0/2851232605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacbfdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:07.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f546be00640 1 -- 172.21.15.131:0/2851232605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 msgr2=0x563dacc02d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:07.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f546be00640 1 --2- 172.21.15.131:0/2851232605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc02d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:07.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.529+0000 7f546be00640 1 -- 172.21.15.131:0/2851232605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dac93c7e0 con 0x563dacc033c0 2024-04-26T20:29:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.530+0000 7f546be00640 1 --2- 172.21.15.131:0/2851232605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc05850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f545c009bf0 tx=0x7f545c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c490fb9702ac660 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.530+0000 7f546b400640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f54580a3cc0 0x7f54580a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.530+0000 7f546b400640 1 -- 172.21.15.131:0/264678475 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f54640efef0 con 0x7f5474103410 2024-04-26T20:29:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.530+0000 7f5463400640 1 -- 172.21.15.131:0/2851232605 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c004050 con 0x563dacc033c0 2024-04-26T20:29:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.530+0000 7f5473400640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f54580a3cc0 0x7f54580a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.530+0000 7f5463400640 1 -- 172.21.15.131:0/2851232605 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f545c0054d0 con 0x563dacc033c0 2024-04-26T20:29:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.530+0000 7f5463400640 1 -- 172.21.15.131:0/2851232605 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c004cd0 con 0x563dacc033c0 2024-04-26T20:29:07.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 -- 172.21.15.131:0/2851232605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 msgr2=0x563dacc05850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:07.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2851232605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc05850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f545c009bf0 tx=0x7f545c011810 comp rx=0 tx=0).stop 2024-04-26T20:29:07.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f5473400640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f54580a3cc0 0x7f54580a6180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x563dacbfeab0 tx=0x7f546c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:07.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 -- 172.21.15.131:0/2851232605 shutdown_connections 2024-04-26T20:29:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2851232605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc05850 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2851232605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc02d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2851232605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacbfdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 -- 172.21.15.131:0/2851232605 >> 172.21.15.131:0/2851232605 conn(0x563dacbf8ed0 msgr2=0x563dacbf92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 -- 172.21.15.131:0/2851232605 shutdown_connections 2024-04-26T20:29:08.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 -- 172.21.15.131:0/2851232605 wait complete. 2024-04-26T20:29:08.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.531+0000 7f547c5b1740 1 -- start start 2024-04-26T20:29:08.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 1 Processor -- start 2024-04-26T20:29:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:08.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfe250 0x563dacc0c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:08.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacc033c0 0x563dacc13620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:08.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563dacc06370 con 0x563dacc033c0 2024-04-26T20:29:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563dacaea170 con 0x563dacbfd860 2024-04-26T20:29:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563dacc064f0 con 0x563dacbfe250 2024-04-26T20:29:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f547c5b1740 10 client.0 _ll_register_callbacks cb 0x563dacb59770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:08.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f5473400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacc033c0 0x563dacc13620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:08.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f546be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfe250 0x563dacc0c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:08.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f5472a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:08.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f5473400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacc033c0 0x563dacc13620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39142/0 (socket says 172.21.15.131:39142) 2024-04-26T20:29:08.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f5472a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:38970/0 (socket says 172.21.15.131:38970) 2024-04-26T20:29:08.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f5473400640 1 -- 172.21.15.131:0/1284100063 learned_addr learned my addr 172.21.15.131:0/1284100063 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:08.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.532+0000 7f546be00640 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfe250 0x563dacc0c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40816/0 (socket says 172.21.15.131:40816) 2024-04-26T20:29:08.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5472a00640 1 -- 172.21.15.131:0/1284100063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfe250 msgr2=0x563dacc0c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5472a00640 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfe250 0x563dacc0c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:08.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5472a00640 1 -- 172.21.15.131:0/1284100063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacc033c0 msgr2=0x563dacc13620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5472a00640 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacc033c0 0x563dacc13620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:08.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5472a00640 1 -- 172.21.15.131:0/1284100063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dac99cf70 con 0x563dacbfd860 2024-04-26T20:29:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5472a00640 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5464005e00 tx=0x7f5464032f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=381734bb16b1abe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:08.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5457e00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:29:08.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5457e00640 1 -- 172.21.15.131:0/1284100063 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640bb870 con 0x563dacbfd860 2024-04-26T20:29:08.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5457e00640 1 -- 172.21.15.131:0/1284100063 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f54640c4510 con 0x563dacbfd860 2024-04-26T20:29:08.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f545c0054d0 con 0x563dacbfd860 2024-04-26T20:29:08.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.533+0000 7f5457e00640 1 -- 172.21.15.131:0/1284100063 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54640b9e70 con 0x563dacbfd860 2024-04-26T20:29:08.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 10 client.26888 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:08.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563dac834170 con 0x563dacbfd860 2024-04-26T20:29:08.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f5457e00640 1 -- 172.21.15.131:0/1284100063 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f545c0054d0 con 0x563dacbfd860 2024-04-26T20:29:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f5457e00640 1 -- 172.21.15.131:0/1284100063 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54640d5080 con 0x563dacbfd860 2024-04-26T20:29:08.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 10 client.26888 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:08.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 10 client.26888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:08.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 20 client.26888 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:08.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc17a70 0x563dacc19f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 4 client.26888 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dacb70590 con 0x563dacc17a70 2024-04-26T20:29:08.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.534+0000 7f546be00640 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc17a70 0x563dacc19f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:08.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.535+0000 7f546be00640 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc17a70 0x563dacc19f00 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9855126791387d4 server_cookie=a2f16bdb23fcf2a2 in_seq=0 out_seq=0 2024-04-26T20:29:08.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.535+0000 7f5457e00640 10 client.26888 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:08.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f5457e00640 1 -- 172.21.15.131:0/1284100063 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563dacb70590 con 0x563dacc17a70 2024-04-26T20:29:08.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f5457e00640 10 client.26888 handle_command_reply: tid=0 2024-04-26T20:29:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 client.26888 shutdown 2024-04-26T20:29:09.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 msgr2=0x563dacc0c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:09.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5464005e00 tx=0x7f5464032f10 comp rx=0 tx=0).stop 2024-04-26T20:29:09.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 shutdown_connections 2024-04-26T20:29:09.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc17a70 0x563dacc19f00 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacc033c0 0x563dacc13620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfe250 0x563dacc0c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f5457e00640 0 client.26888 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:09.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 --2- 172.21.15.131:0/1284100063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c0a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 >> 172.21.15.131:0/1284100063 conn(0x563dacbf8ed0 msgr2=0x563dacbf7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:09.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 shutdown_connections 2024-04-26T20:29:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.536+0000 7f547c5b1740 1 -- 172.21.15.131:0/1284100063 wait complete. 2024-04-26T20:29:09.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.537+0000 7f547c5b1740 20 client.26888 trim_cache size 0 max 16384 2024-04-26T20:29:09.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 Processor -- start 2024-04-26T20:29:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 -- start start 2024-04-26T20:29:09.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:09.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:09.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacc033c0 0x563dacc1e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:09.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54741abd50 con 0x563dacbfe250 2024-04-26T20:29:09.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54741a9620 con 0x563dacbfd860 2024-04-26T20:29:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f547c5b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54741a7470 con 0x563dacc033c0 2024-04-26T20:29:09.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f546be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:09.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f546be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc135e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39144/0 (socket says 172.21.15.131:39144) 2024-04-26T20:29:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f546be00640 1 -- 172.21.15.131:0/2219058358 learned_addr learned my addr 172.21.15.131:0/2219058358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.556+0000 7f5472a00640 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f5473400640 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacc033c0 0x563dacc1e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:09.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f546be00640 1 -- 172.21.15.131:0/2219058358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacc033c0 msgr2=0x563dacc1e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:09.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f546be00640 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacc033c0 0x563dacc1e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f546be00640 1 -- 172.21.15.131:0/2219058358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 msgr2=0x563dacc0c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:09.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f546be00640 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f546be00640 1 -- 172.21.15.131:0/2219058358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dacbf8a70 con 0x563dacbfe250 2024-04-26T20:29:09.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f546be00640 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc135e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x563dacbf9e40 tx=0x7f545c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6b696b0e009582 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:09.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f5462a00640 1 -- 172.21.15.131:0/2219058358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c033070 con 0x563dacbfe250 2024-04-26T20:29:09.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f5462a00640 1 -- 172.21.15.131:0/2219058358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f545c02edd0 con 0x563dacbfe250 2024-04-26T20:29:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.557+0000 7f5462a00640 1 -- 172.21.15.131:0/2219058358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c0035f0 con 0x563dacbfe250 2024-04-26T20:29:09.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 -- 172.21.15.131:0/2219058358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 msgr2=0x563dacc135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:09.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc135e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x563dacbf9e40 tx=0x7f545c005d50 comp rx=0 tx=0).stop 2024-04-26T20:29:09.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 -- 172.21.15.131:0/2219058358 shutdown_connections 2024-04-26T20:29:09.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacc033c0 0x563dacc1e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc135e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 --2- 172.21.15.131:0/2219058358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacbfd860 0x563dacc0c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:09.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 -- 172.21.15.131:0/2219058358 >> 172.21.15.131:0/2219058358 conn(0x563dacbf8ed0 msgr2=0x563dacbf7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:09.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 -- 172.21.15.131:0/2219058358 shutdown_connections 2024-04-26T20:29:09.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 -- 172.21.15.131:0/2219058358 wait complete. 2024-04-26T20:29:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.558+0000 7f547c5b1740 1 -- start start 2024-04-26T20:29:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 1 Processor -- start 2024-04-26T20:29:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacc131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:09.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc11840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:09.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc11d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54741a7ae0 con 0x563dacbfe250 2024-04-26T20:29:09.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54741a8f50 con 0x563dacc033c0 2024-04-26T20:29:09.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54741a81b0 con 0x563dacbfd860 2024-04-26T20:29:09.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f547c5b1740 10 client.0 _ll_register_callbacks cb 0x7f54640f3690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:09.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f546be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc11840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:09.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f5472a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacc131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f5472a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacc131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40844/0 (socket says 172.21.15.131:40844) 2024-04-26T20:29:09.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f546be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc11840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39160/0 (socket says 172.21.15.131:39160) 2024-04-26T20:29:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.559+0000 7f5472a00640 1 -- 172.21.15.131:0/4268546549 learned_addr learned my addr 172.21.15.131:0/4268546549 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5473400640 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc11d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:10.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5472a00640 1 -- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 msgr2=0x563dacc11d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5472a00640 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc11d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5472a00640 1 -- 172.21.15.131:0/4268546549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 msgr2=0x563dacc11840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:10.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5472a00640 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc11840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5472a00640 1 -- 172.21.15.131:0/4268546549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dacc122c0 con 0x563dacbfd860 2024-04-26T20:29:10.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5473400640 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc11d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:10.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5472a00640 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacc131e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f54640dc010 tx=0x7f54640d52b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aec1b8a6257f126e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:10.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5457e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:29:10.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.560+0000 7f5457e00640 1 -- 172.21.15.131:0/4268546549 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640bb690 con 0x563dacbfd860 2024-04-26T20:29:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.561+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f545c02edd0 con 0x563dacbfd860 2024-04-26T20:29:10.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.561+0000 7f5457e00640 1 -- 172.21.15.131:0/4268546549 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f54640c4510 con 0x563dacbfd860 2024-04-26T20:29:10.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f5457e00640 1 -- 172.21.15.131:0/4268546549 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54640bb870 con 0x563dacbfd860 2024-04-26T20:29:10.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 10 client.26875 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:10.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563dacc7ece0 con 0x563dacbfd860 2024-04-26T20:29:10.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f5457e00640 1 -- 172.21.15.131:0/4268546549 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f545c02edd0 con 0x563dacbfd860 2024-04-26T20:29:10.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f5457e00640 1 -- 172.21.15.131:0/4268546549 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5464030550 con 0x563dacbfd860 2024-04-26T20:29:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 10 client.26875 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:10.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 10 client.26875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:10.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 20 client.26875 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:10.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc7f2d0 0x563dacc816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:10.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 4 client.26875 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T20:29:10.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.565+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563dacb6a400 con 0x563dacc7f2d0 2024-04-26T20:29:10.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.566+0000 7f546be00640 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc7f2d0 0x563dacc816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:10.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.566+0000 7f546be00640 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc7f2d0 0x563dacc816f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9087b69139dd676 server_cookie=aa6261f9989702e in_seq=0 out_seq=0 2024-04-26T20:29:10.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.566+0000 7f5457e00640 10 client.26875 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f5457e00640 1 -- 172.21.15.131:0/4268546549 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563dacb6a400 con 0x563dacc7f2d0 2024-04-26T20:29:10.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f5457e00640 10 client.26875 handle_command_reply: tid=0 2024-04-26T20:29:10.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 client.26875 shutdown 2024-04-26T20:29:10.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 msgr2=0x563dacc131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacc131e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f54640dc010 tx=0x7f54640d52b0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 shutdown_connections 2024-04-26T20:29:10.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x563dacc7f2d0 0x563dacc816f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563dacc033c0 0x563dacc11d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563dacbfe250 0x563dacc11840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 --2- 172.21.15.131:0/4268546549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563dacbfd860 0x563dacc131e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f5457e00640 0 client.26875 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:10.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 >> 172.21.15.131:0/4268546549 conn(0x563dacbf8ed0 msgr2=0x563dacbf7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 shutdown_connections 2024-04-26T20:29:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 1 -- 172.21.15.131:0/4268546549 wait complete. 2024-04-26T20:29:10.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.568+0000 7f547c5b1740 20 client.26875 trim_cache size 0 max 16384 2024-04-26T20:29:10.756 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:29:10.813 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-04-26T20:29:10.823 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "1ea4f764-0557-4f4e-a037-794ade89a1aa", 2024-04-26T20:29:10.834 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-04-26T20:29:10.843 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:29:10.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f54580a3cc0 msgr2=0x7f54580a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:10.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f54580a3cc0 0x7f54580a6180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x563dacbfeab0 tx=0x7f546c0095c0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474103410 msgr2=0x7f5474198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:10.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474103410 0x7f5474198c50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f54640057c0 tx=0x7f5464032da0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 shutdown_connections 2024-04-26T20:29:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f54580a3cc0 0x7f54580a6180 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5474108670 0x7f54741a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474105d40 0x7f5474199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 --2- 172.21.15.131:0/264678475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5474103410 0x7f5474198c50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.569+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 >> 172.21.15.131:0/264678475 conn(0x7f54740fce20 msgr2=0x7f5474103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:11.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.570+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 shutdown_connections 2024-04-26T20:29:11.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:28:43.570+0000 7f5479600640 1 -- 172.21.15.131:0/264678475 wait complete. 2024-04-26T20:29:11.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T20:29:11.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T20:29:11.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:43 smithi131 ceph-mon[27919]: pgmap v971: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:11.118 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:44 smithi131 ceph-mon[27919]: scrub queued for path: / 2024-04-26T20:29:11.127 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:44 smithi131 ceph-mon[27919]: scrub summary: idle+waiting paths [/] 2024-04-26T20:29:11.136 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:44 smithi131 ceph-mon[27919]: scrub summary: active paths [/] 2024-04-26T20:29:11.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:44 smithi131 ceph-mon[27919]: scrub summary: idle+waiting paths [/] 2024-04-26T20:29:11.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:45 smithi131 ceph-mon[27919]: pgmap v972: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:29:11.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:47 smithi131 ceph-mon[27919]: pgmap v973: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:11.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:47 smithi131 ceph-mon[27919]: scrub summary: idle 2024-04-26T20:29:11.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:49 smithi131 ceph-mon[27919]: pgmap v974: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:29:11.311 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:07 smithi148 ceph-mon[32281]: pgmap v983: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:11.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:09 smithi148 ceph-mon[32281]: pgmap v984: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:29:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:07 smithi157 ceph-mon[32703]: pgmap v983: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:09 smithi157 ceph-mon[32703]: pgmap v984: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 -- 172.21.15.131:0/1967656273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 msgr2=0x7f2770106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 --2- 172.21.15.131:0/1967656273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 0x7f2770106230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f275c009860 tx=0x7f275c02a340 comp rx=0 tx=0).stop 2024-04-26T20:29:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 -- 172.21.15.131:0/1967656273 shutdown_connections 2024-04-26T20:29:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 --2- 172.21.15.131:0/1967656273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770106770 0x7f277010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 --2- 172.21.15.131:0/1967656273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 0x7f2770106230 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 --2- 172.21.15.131:0/1967656273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 0x7f2770104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 -- 172.21.15.131:0/1967656273 >> 172.21.15.131:0/1967656273 conn(0x7f2770100150 msgr2=0x7f2770102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 -- 172.21.15.131:0/1967656273 shutdown_connections 2024-04-26T20:29:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.480+0000 7f2778200640 1 -- 172.21.15.131:0/1967656273 wait complete. 2024-04-26T20:29:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.481+0000 7f2778200640 1 Processor -- start 2024-04-26T20:29:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.481+0000 7f2778200640 1 -- start start 2024-04-26T20:29:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2778200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 0x7f277019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2778200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 0x7f277019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2778200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770106770 0x7f27701a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2778200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27701106d0 con 0x7f2770104bc0 2024-04-26T20:29:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2778200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2770110550 con 0x7f2770106770 2024-04-26T20:29:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2778200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2770110850 con 0x7f2770105db0 2024-04-26T20:29:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2775800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 0x7f277019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2775800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 0x7f277019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55480/0 (socket says 172.21.15.131:55480) 2024-04-26T20:29:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2775800640 1 -- 172.21.15.131:0/3474698594 learned_addr learned my addr 172.21.15.131:0/3474698594 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2774e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 0x7f277019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.482+0000 7f2776200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770106770 0x7f27701a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2774e00640 1 -- 172.21.15.131:0/3474698594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770106770 msgr2=0x7f27701a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2774e00640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770106770 0x7f27701a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2774e00640 1 -- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 msgr2=0x7f277019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2774e00640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 0x7f277019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2774e00640 1 -- 172.21.15.131:0/3474698594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27701a2950 con 0x7f2770105db0 2024-04-26T20:29:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2775800640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 0x7f277019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2774e00640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 0x7f277019b280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f275c02a860 tx=0x7f275c032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f276e000640 1 -- 172.21.15.131:0/3474698594 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c040020 con 0x7f2770105db0 2024-04-26T20:29:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f276e000640 1 -- 172.21.15.131:0/3474698594 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275c030d50 con 0x7f2770105db0 2024-04-26T20:29:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f276e000640 1 -- 172.21.15.131:0/3474698594 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c037040 con 0x7f2770105db0 2024-04-26T20:29:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.483+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27701a2be0 con 0x7f2770105db0 2024-04-26T20:29:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.484+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27701a3120 con 0x7f2770105db0 2024-04-26T20:29:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.487+0000 7f276e000640 1 -- 172.21.15.131:0/3474698594 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f275c043020 con 0x7f2770105db0 2024-04-26T20:29:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.487+0000 7f277afcb740 1 Processor -- start 2024-04-26T20:29:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.487+0000 7f277afcb740 1 -- start start 2024-04-26T20:29:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.487+0000 7f277afcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f277afcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f277afcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bb9b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f277afcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5634bb913380 con 0x5634bb9b4730 2024-04-26T20:29:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f277afcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5634bb8afc30 con 0x5634bb9af5c0 2024-04-26T20:29:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f277afcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5634bb90f340 con 0x5634bb9aebd0 2024-04-26T20:29:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:41658/0 (socket says 172.21.15.131:41658) 2024-04-26T20:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 -- 172.21.15.131:0/659684944 learned_addr learned my addr 172.21.15.131:0/659684944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2776200640 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2774e00640 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bb9b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 -- 172.21.15.131:0/659684944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 msgr2=0x5634bb9aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 -- 172.21.15.131:0/659684944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 msgr2=0x5634bb9b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bb9b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.488+0000 7f2775800640 1 -- 172.21.15.131:0/659684944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634bb8d1940 con 0x5634bb9af5c0 2024-04-26T20:29:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.489+0000 7f276e000640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27580a3d70 0x7f27580a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.489+0000 7f2775800640 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9b40c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f27640115f0 tx=0x7f2764011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240a4cacb03717a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.489+0000 7f276e000640 1 -- 172.21.15.131:0/3474698594 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f275c0f04f0 con 0x7f2770105db0 2024-04-26T20:29:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.489+0000 7f2762a00640 1 -- 172.21.15.131:0/659684944 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276401e070 con 0x5634bb9af5c0 2024-04-26T20:29:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.489+0000 7f2776200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27580a3d70 0x7f27580a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.489+0000 7f2762a00640 1 -- 172.21.15.131:0/659684944 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f276400fbb0 con 0x5634bb9af5c0 2024-04-26T20:29:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.489+0000 7f2762a00640 1 -- 172.21.15.131:0/659684944 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764019b90 con 0x5634bb9af5c0 2024-04-26T20:29:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 -- 172.21.15.131:0/659684944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 msgr2=0x5634bb9b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f2776200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27580a3d70 0x7f27580a6230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5634bb9afe20 tx=0x7f2768009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9b40c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f27640115f0 tx=0x7f2764011ab0 comp rx=0 tx=0).stop 2024-04-26T20:29:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 -- 172.21.15.131:0/659684944 shutdown_connections 2024-04-26T20:29:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bb9b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9b40c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 --2- 172.21.15.131:0/659684944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 -- 172.21.15.131:0/659684944 >> 172.21.15.131:0/659684944 conn(0x5634bb9aa240 msgr2=0x5634bb9aa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 -- 172.21.15.131:0/659684944 shutdown_connections 2024-04-26T20:29:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 -- 172.21.15.131:0/659684944 wait complete. 2024-04-26T20:29:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.490+0000 7f277afcb740 1 -- start start 2024-04-26T20:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 1 Processor -- start 2024-04-26T20:29:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9b4730 0x5634bb9c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5634bb9b76e0 con 0x5634bb9af5c0 2024-04-26T20:29:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5634bb89ac60 con 0x5634bb9b4730 2024-04-26T20:29:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5634bb9b7860 con 0x5634bb9aebd0 2024-04-26T20:29:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f277afcb740 10 client.0 _ll_register_callbacks cb 0x5634bb8c6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f2774e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f2774e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9b9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55496/0 (socket says 172.21.15.131:55496) 2024-04-26T20:29:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f2774e00640 1 -- 172.21.15.131:0/101457331 learned_addr learned my addr 172.21.15.131:0/101457331 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f2775800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.491+0000 7f2776200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9b4730 0x5634bb9c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2774e00640 1 -- 172.21.15.131:0/101457331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 msgr2=0x5634bb9b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2774e00640 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2774e00640 1 -- 172.21.15.131:0/101457331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9b4730 msgr2=0x5634bb9c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2774e00640 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9b4730 0x5634bb9c0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2774e00640 1 -- 172.21.15.131:0/101457331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634bb8a9510 con 0x5634bb9af5c0 2024-04-26T20:29:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2774e00640 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9b9350 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f275c035040 tx=0x7f275c0edf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951e02a6f60a9dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2757400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2757400640 1 -- 172.21.15.131:0/101457331 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0b8540 con 0x5634bb9af5c0 2024-04-26T20:29:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f2757400640 1 -- 172.21.15.131:0/101457331 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f275c0c4b10 con 0x5634bb9af5c0 2024-04-26T20:29:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.492+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f276400fbb0 con 0x5634bb9af5c0 2024-04-26T20:29:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f2757400640 1 -- 172.21.15.131:0/101457331 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f275c0ba470 con 0x5634bb9af5c0 2024-04-26T20:29:13.179 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 10 client.16785 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5634bb625210 con 0x5634bb9af5c0 2024-04-26T20:29:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f2757400640 1 -- 172.21.15.131:0/101457331 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f276400fbb0 con 0x5634bb9af5c0 2024-04-26T20:29:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f2757400640 1 -- 172.21.15.131:0/101457331 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f275c0b7ff0 con 0x5634bb9af5c0 2024-04-26T20:29:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 10 client.16785 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 20 client.16785 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bb9c4950 0x5634bb9c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 4 client.16785 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.493+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634bb920f90 con 0x5634bb9c4950 2024-04-26T20:29:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.494+0000 7f2775800640 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bb9c4950 0x5634bb9c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.494+0000 7f2775800640 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bb9c4950 0x5634bb9c6dc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93266a17b16713c6 server_cookie=315fb5a23f9d5e6b in_seq=0 out_seq=0 2024-04-26T20:29:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.494+0000 7f2757400640 10 client.16785 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f2757400640 1 -- 172.21.15.131:0/101457331 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634bb920f90 con 0x5634bb9c4950 2024-04-26T20:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f2757400640 10 client.16785 handle_command_reply: tid=0 2024-04-26T20:29:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 client.16785 shutdown 2024-04-26T20:29:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 msgr2=0x5634bb9b9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9b9350 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f275c035040 tx=0x7f275c0edf90 comp rx=0 tx=0).stop 2024-04-26T20:29:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 shutdown_connections 2024-04-26T20:29:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bb9c4950 0x5634bb9c6dc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9b4730 0x5634bb9c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9b9350 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 --2- 172.21.15.131:0/101457331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 >> 172.21.15.131:0/101457331 conn(0x5634bb9aa240 msgr2=0x5634bb9a9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.495+0000 7f2757400640 0 client.16785 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.496+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 shutdown_connections 2024-04-26T20:29:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.496+0000 7f277afcb740 1 -- 172.21.15.131:0/101457331 wait complete. 2024-04-26T20:29:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.496+0000 7f277afcb740 20 client.16785 trim_cache size 0 max 16384 2024-04-26T20:29:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 Processor -- start 2024-04-26T20:29:13.692 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 -- start start 2024-04-26T20:29:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9aebd0 0x5634bb9cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9b4730 0x5634bb9c0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27701b0050 con 0x5634bb9af5c0 2024-04-26T20:29:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27701ad920 con 0x5634bb9aebd0 2024-04-26T20:29:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f277afcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27701ab770 con 0x5634bb9b4730 2024-04-26T20:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f2774e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f2774e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55502/0 (socket says 172.21.15.131:55502) 2024-04-26T20:29:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f2774e00640 1 -- 172.21.15.131:0/3700651007 learned_addr learned my addr 172.21.15.131:0/3700651007 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.512+0000 7f2776200640 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9b4730 0x5634bb9c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2775800640 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9aebd0 0x5634bb9cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2774e00640 1 -- 172.21.15.131:0/3700651007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9b4730 msgr2=0x5634bb9c0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2774e00640 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9b4730 0x5634bb9c0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2774e00640 1 -- 172.21.15.131:0/3700651007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9aebd0 msgr2=0x5634bb9cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2774e00640 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9aebd0 0x5634bb9cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2774e00640 1 -- 172.21.15.131:0/3700651007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634bb6f1620 con 0x5634bb9af5c0 2024-04-26T20:29:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2775800640 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9aebd0 0x5634bb9cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2774e00640 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9ccbe0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f275c0b8450 tx=0x7f275c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245cd31082d71cde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2762000640 1 -- 172.21.15.131:0/3700651007 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0bbc90 con 0x5634bb9af5c0 2024-04-26T20:29:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2762000640 1 -- 172.21.15.131:0/3700651007 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275c0bca30 con 0x5634bb9af5c0 2024-04-26T20:29:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.513+0000 7f2762000640 1 -- 172.21.15.131:0/3700651007 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0bbe70 con 0x5634bb9af5c0 2024-04-26T20:29:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 -- 172.21.15.131:0/3700651007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 msgr2=0x5634bb9ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9ccbe0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f275c0b8450 tx=0x7f275c02a340 comp rx=0 tx=0).stop 2024-04-26T20:29:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 -- 172.21.15.131:0/3700651007 shutdown_connections 2024-04-26T20:29:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9b4730 0x5634bb9c0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9af5c0 0x5634bb9ccbe0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 --2- 172.21.15.131:0/3700651007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9aebd0 0x5634bb9cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 -- 172.21.15.131:0/3700651007 >> 172.21.15.131:0/3700651007 conn(0x5634bb9aa240 msgr2=0x5634bb9b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 -- 172.21.15.131:0/3700651007 shutdown_connections 2024-04-26T20:29:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 -- 172.21.15.131:0/3700651007 wait complete. 2024-04-26T20:29:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.514+0000 7f277afcb740 1 -- start start 2024-04-26T20:29:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 1 Processor -- start 2024-04-26T20:29:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9bd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bba28f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27701abde0 con 0x5634bb9b4730 2024-04-26T20:29:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27701ad250 con 0x5634bb9af5c0 2024-04-26T20:29:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27701ac4b0 con 0x5634bb9aebd0 2024-04-26T20:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f277afcb740 10 client.0 _ll_register_callbacks cb 0x7f275c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f2776200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bba28f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f2775800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f2776200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bba28f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55512/0 (socket says 172.21.15.131:55512) 2024-04-26T20:29:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f2775800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9bcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59374/0 (socket says 172.21.15.131:59374) 2024-04-26T20:29:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f2776200640 1 -- 172.21.15.131:0/3120697206 learned_addr learned my addr 172.21.15.131:0/3120697206 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.515+0000 7f2774e00640 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9bd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2775800640 1 -- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 msgr2=0x5634bb9bd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2775800640 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9bd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2775800640 1 -- 172.21.15.131:0/3120697206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 msgr2=0x5634bba28f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2775800640 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bba28f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2775800640 1 -- 172.21.15.131:0/3120697206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634bb8cdcb0 con 0x5634bb9aebd0 2024-04-26T20:29:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2775800640 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9bcf20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5634bb9ab1b0 tx=0x7f2764005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e4574414acd81e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2757400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:29:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2757400640 1 -- 172.21.15.131:0/3120697206 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276401e040 con 0x5634bb9aebd0 2024-04-26T20:29:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2757400640 1 -- 172.21.15.131:0/3120697206 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2764009cd0 con 0x5634bb9aebd0 2024-04-26T20:29:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f275c0bca30 con 0x5634bb9aebd0 2024-04-26T20:29:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f2757400640 1 -- 172.21.15.131:0/3120697206 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2764037050 con 0x5634bb9aebd0 2024-04-26T20:29:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f277afcb740 10 client.26893 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.516+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5634bba2a220 con 0x5634bb9aebd0 2024-04-26T20:29:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f2757400640 1 -- 172.21.15.131:0/3120697206 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275c0bca30 con 0x5634bb9aebd0 2024-04-26T20:29:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f2757400640 1 -- 172.21.15.131:0/3120697206 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2764005420 con 0x5634bb9aebd0 2024-04-26T20:29:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f277afcb740 10 client.26893 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f277afcb740 10 client.26893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f277afcb740 20 client.26893 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f277afcb740 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bba2aa80 0x5634bba2cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f277afcb740 4 client.26893 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:29:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634bb91ae00 con 0x5634bba2aa80 2024-04-26T20:29:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.517+0000 7f2774e00640 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bba2aa80 0x5634bba2cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.518+0000 7f2774e00640 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bba2aa80 0x5634bba2cea0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a10d5208623e66 server_cookie=8a0ba36ddc00e0f1 in_seq=0 out_seq=0 2024-04-26T20:29:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.518+0000 7f2757400640 10 client.26893 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.522+0000 7f2757400640 1 -- 172.21.15.131:0/3120697206 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634bb91ae00 con 0x5634bba2aa80 2024-04-26T20:29:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.522+0000 7f2757400640 10 client.26893 handle_command_reply: tid=0 2024-04-26T20:29:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.522+0000 7f277afcb740 1 client.26893 shutdown 2024-04-26T20:29:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.522+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 msgr2=0x5634bb9bcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.522+0000 7f277afcb740 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9bcf20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5634bb9ab1b0 tx=0x7f2764005df0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 shutdown_connections 2024-04-26T20:29:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5634bba2aa80 0x5634bba2cea0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634bb9b4730 0x5634bba28f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634bb9af5c0 0x5634bb9bd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 --2- 172.21.15.131:0/3120697206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634bb9aebd0 0x5634bb9bcf20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f2757400640 0 client.26893 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 >> 172.21.15.131:0/3120697206 conn(0x5634bb9aa240 msgr2=0x5634bb9b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 shutdown_connections 2024-04-26T20:29:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 1 -- 172.21.15.131:0/3120697206 wait complete. 2024-04-26T20:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.523+0000 7f277afcb740 20 client.26893 trim_cache size 0 max 16384 2024-04-26T20:29:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:29:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:29:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:29:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:29:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27580a3d70 msgr2=0x7f27580a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27580a3d70 0x7f27580a6230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5634bb9afe20 tx=0x7f2768009500 comp rx=0 tx=0).stop 2024-04-26T20:29:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 msgr2=0x7f277019b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 0x7f277019b280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f275c02a860 tx=0x7f275c032d50 comp rx=0 tx=0).stop 2024-04-26T20:29:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 shutdown_connections 2024-04-26T20:29:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770106770 0x7f27701a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27580a3d70 0x7f27580a6230 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2770105db0 0x7f277019b280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 --2- 172.21.15.131:0/3474698594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2770104bc0 0x7f277019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 >> 172.21.15.131:0/3474698594 conn(0x7f2770100150 msgr2=0x7f27701090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 shutdown_connections 2024-04-26T20:29:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:28:50.524+0000 7f2778200640 1 -- 172.21.15.131:0/3474698594 wait complete. 2024-04-26T20:29:15.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:51 smithi131 ceph-mon[27919]: pgmap v975: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:29:15.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:53 smithi131 ceph-mon[27919]: pgmap v976: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T20:29:15.719 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:55 smithi131 ceph-mon[27919]: pgmap v977: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:29:15.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:57 smithi131 ceph-mon[27919]: pgmap v978: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:15.801 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:28:59 smithi131 ceph-mon[27919]: pgmap v979: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T20:29:15.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:01 smithi131 ceph-mon[27919]: pgmap v980: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:29:15.817 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:02 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:29:15.826 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:11 smithi148 ceph-mon[32281]: pgmap v985: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:29:15.835 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:13 smithi148 ceph-mon[32281]: pgmap v986: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T20:29:15.881 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:11 smithi157 ceph-mon[32703]: pgmap v985: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:29:15.932 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:13 smithi157 ceph-mon[32703]: pgmap v986: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T20:29:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 -- 172.21.15.131:0/1249909527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754104bb0 msgr2=0x7fd754104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 --2- 172.21.15.131:0/1249909527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754104bb0 0x7fd754104fd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd744009ca0 tx=0x7fd74402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 -- 172.21.15.131:0/1249909527 shutdown_connections 2024-04-26T20:29:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 --2- 172.21.15.131:0/1249909527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd754106760 0x7fd75410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 --2- 172.21.15.131:0/1249909527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd754105da0 0x7fd754106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 --2- 172.21.15.131:0/1249909527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754104bb0 0x7fd754104fd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 -- 172.21.15.131:0/1249909527 >> 172.21.15.131:0/1249909527 conn(0x7fd754100120 msgr2=0x7fd754102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 -- 172.21.15.131:0/1249909527 shutdown_connections 2024-04-26T20:29:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.661+0000 7fd759800640 1 -- 172.21.15.131:0/1249909527 wait complete. 2024-04-26T20:29:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.662+0000 7fd759800640 1 Processor -- start 2024-04-26T20:29:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd759800640 1 -- start start 2024-04-26T20:29:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd759800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd754104bb0 0x7fd75419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd759800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd754105da0 0x7fd75419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd759800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754106760 0x7fd7541a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd759800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541106e0 con 0x7fd754105da0 2024-04-26T20:29:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd759800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd754110560 con 0x7fd754106760 2024-04-26T20:29:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd759800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd754110860 con 0x7fd754104bb0 2024-04-26T20:29:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd752a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd754104bb0 0x7fd75419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd752a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd754104bb0 0x7fd75419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37692/0 (socket says 172.21.15.131:37692) 2024-04-26T20:29:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd752a00640 1 -- 172.21.15.131:0/1694293840 learned_addr learned my addr 172.21.15.131:0/1694293840 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd752000640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd754105da0 0x7fd75419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.663+0000 7fd753400640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754106760 0x7fd7541a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd753400640 1 -- 172.21.15.131:0/1694293840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd754104bb0 msgr2=0x7fd75419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd753400640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd754104bb0 0x7fd75419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd753400640 1 -- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd754105da0 msgr2=0x7fd75419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd753400640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd754105da0 0x7fd75419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd753400640 1 -- 172.21.15.131:0/1694293840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7541a2950 con 0x7fd754106760 2024-04-26T20:29:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd752000640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd754105da0 0x7fd75419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd753400640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754106760 0x7fd7541a22b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00e970 tx=0x7fd74c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd74b400640 1 -- 172.21.15.131:0/1694293840 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00ccc0 con 0x7fd754106760 2024-04-26T20:29:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.664+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7541a2c40 con 0x7fd754106760 2024-04-26T20:29:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.665+0000 7fd74b400640 1 -- 172.21.15.131:0/1694293840 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c005020 con 0x7fd754106760 2024-04-26T20:29:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.665+0000 7fd74b400640 1 -- 172.21.15.131:0/1694293840 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c004a00 con 0x7fd754106760 2024-04-26T20:29:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.665+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7541a3190 con 0x7fd754106760 2024-04-26T20:29:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.668+0000 7fd75c582740 1 Processor -- start 2024-04-26T20:29:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd74b400640 1 -- 172.21.15.131:0/1694293840 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd74c004ba0 con 0x7fd754106760 2024-04-26T20:29:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd75c582740 1 -- start start 2024-04-26T20:29:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd75c582740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd75c582740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e1386910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd75c582740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e138691730 0x55e138693bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd75c582740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e1385f0380 con 0x55e138691730 2024-04-26T20:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd75c582740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e13858cc30 con 0x55e13868c5c0 2024-04-26T20:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.670+0000 7fd75c582740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e1385ec340 con 0x55e13868bbd0 2024-04-26T20:29:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e1386910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e1386910c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57394/0 (socket says 172.21.15.131:57394) 2024-04-26T20:29:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 -- 172.21.15.131:0/2839605460 learned_addr learned my addr 172.21.15.131:0/2839605460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752000640 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e138691730 0x55e138693bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 -- 172.21.15.131:0/2839605460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 msgr2=0x55e13868bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 -- 172.21.15.131:0/2839605460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e138691730 msgr2=0x55e138693bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e138691730 0x55e138693bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752a00640 1 -- 172.21.15.131:0/2839605460 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1385ae940 con 0x55e13868c5c0 2024-04-26T20:29:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd752000640 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e138691730 0x55e138693bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:29:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.671+0000 7fd74b400640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7380a3cf0 0x7fd7380a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.672+0000 7fd752a00640 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e1386910c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd7440093e0 tx=0x7fd744005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a6279fd052fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.672+0000 7fd74b400640 1 -- 172.21.15.131:0/1694293840 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd74c0d0950 con 0x7fd754106760 2024-04-26T20:29:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.672+0000 7fd752000640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7380a3cf0 0x7fd7380a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.673+0000 7fd752000640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7380a3cf0 0x7fd7380a61b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55e13868ce20 tx=0x7fd73c013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.677+0000 7fd743e00640 1 -- 172.21.15.131:0/2839605460 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744031650 con 0x55e13868c5c0 2024-04-26T20:29:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.677+0000 7fd743e00640 1 -- 172.21.15.131:0/2839605460 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd744036030 con 0x55e13868c5c0 2024-04-26T20:29:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.677+0000 7fd743e00640 1 -- 172.21.15.131:0/2839605460 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7440046f0 con 0x55e13868c5c0 2024-04-26T20:29:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 -- 172.21.15.131:0/2839605460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 msgr2=0x55e1386910c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e1386910c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd7440093e0 tx=0x7fd744005ef0 comp rx=0 tx=0).stop 2024-04-26T20:29:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 -- 172.21.15.131:0/2839605460 shutdown_connections 2024-04-26T20:29:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e138691730 0x55e138693bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.050 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e1386910c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 --2- 172.21.15.131:0/2839605460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 -- 172.21.15.131:0/2839605460 >> 172.21.15.131:0/2839605460 conn(0x55e138687240 msgr2=0x55e138687640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 -- 172.21.15.131:0/2839605460 shutdown_connections 2024-04-26T20:29:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 -- 172.21.15.131:0/2839605460 wait complete. 2024-04-26T20:29:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.678+0000 7fd75c582740 1 -- start start 2024-04-26T20:29:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 1 Processor -- start 2024-04-26T20:29:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868bbd0 0x55e138695c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e138696170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e138691730 0x55e13869d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e138694500 con 0x55e13868bbd0 2024-04-26T20:29:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e138577c60 con 0x55e13868c5c0 2024-04-26T20:29:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e138694680 con 0x55e138691730 2024-04-26T20:29:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd75c582740 10 client.0 _ll_register_callbacks cb 0x55e1385a3e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd752a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868bbd0 0x55e138695c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.679+0000 7fd752000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e138696170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868bbd0 0x55e138695c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45754/0 (socket says 172.21.15.131:45754) 2024-04-26T20:29:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752a00640 1 -- 172.21.15.131:0/377947901 learned_addr learned my addr 172.21.15.131:0/377947901 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd753400640 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e138691730 0x55e13869d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752000640 1 -- 172.21.15.131:0/377947901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e138691730 msgr2=0x55e13869d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752000640 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e138691730 0x55e13869d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752000640 1 -- 172.21.15.131:0/377947901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868bbd0 msgr2=0x55e138695c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752000640 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868bbd0 0x55e138695c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752000640 1 -- 172.21.15.131:0/377947901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e138586510 con 0x55e13868c5c0 2024-04-26T20:29:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd752000640 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e138696170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00f790 tx=0x7fd73c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=495fb0a32d0dc9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.680+0000 7fd740c00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:29:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd740c00640 1 -- 172.21.15.131:0/377947901 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c02d060 con 0x55e13868c5c0 2024-04-26T20:29:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd744036030 con 0x55e13868c5c0 2024-04-26T20:29:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd740c00640 1 -- 172.21.15.131:0/377947901 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd73c003d00 con 0x55e13868c5c0 2024-04-26T20:29:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd740c00640 1 -- 172.21.15.131:0/377947901 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd73c034050 con 0x55e13868c5c0 2024-04-26T20:29:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd75c582740 10 client.26924 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e138302210 con 0x55e13868c5c0 2024-04-26T20:29:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd740c00640 1 -- 172.21.15.131:0/377947901 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd744036030 con 0x55e13868c5c0 2024-04-26T20:29:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.681+0000 7fd740c00640 1 -- 172.21.15.131:0/377947901 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd73c01d410 con 0x55e13868c5c0 2024-04-26T20:29:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd75c582740 10 client.26924 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd75c582740 10 client.26924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd75c582740 20 client.26924 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd75c582740 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e1386a16c0 0x55e1386a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd75c582740 4 client.26924 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1385fdf90 con 0x55e1386a16c0 2024-04-26T20:29:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd752a00640 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e1386a16c0 0x55e1386a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.682+0000 7fd752a00640 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e1386a16c0 0x55e1386a3b50 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73a7c2ac4745c82 server_cookie=a41f1ed3ddc7d92c in_seq=0 out_seq=0 2024-04-26T20:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.683+0000 7fd740c00640 10 client.26924 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.683+0000 7fd740c00640 1 -- 172.21.15.131:0/377947901 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1385fdf90 con 0x55e1386a16c0 2024-04-26T20:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.683+0000 7fd740c00640 10 client.26924 handle_command_reply: tid=0 2024-04-26T20:29:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.683+0000 7fd75c582740 1 client.26924 shutdown 2024-04-26T20:29:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 msgr2=0x55e138696170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e138696170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00f790 tx=0x7fd73c005f90 comp rx=0 tx=0).stop 2024-04-26T20:29:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 shutdown_connections 2024-04-26T20:29:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e1386a16c0 0x55e1386a3b50 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e138691730 0x55e13869d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd740c00640 0 client.26924 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e13868c5c0 0x55e138696170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 --2- 172.21.15.131:0/377947901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868bbd0 0x55e138695c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 >> 172.21.15.131:0/377947901 conn(0x55e138687240 msgr2=0x55e138686440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 shutdown_connections 2024-04-26T20:29:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 1 -- 172.21.15.131:0/377947901 wait complete. 2024-04-26T20:29:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.684+0000 7fd75c582740 20 client.26924 trim_cache size 0 max 16384 2024-04-26T20:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 Processor -- start 2024-04-26T20:29:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 -- start start 2024-04-26T20:29:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13868dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541b0020 con 0x55e13868c5c0 2024-04-26T20:29:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541ad8f0 con 0x55e138691730 2024-04-26T20:29:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.700+0000 7fd75c582740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541ab740 con 0x55e13868bbd0 2024-04-26T20:29:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd753400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13868dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37728/0 (socket says 172.21.15.131:37728) 2024-04-26T20:29:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd753400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:57408/0 (socket says 172.21.15.131:57408) 2024-04-26T20:29:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 -- 172.21.15.131:0/3715178743 learned_addr learned my addr 172.21.15.131:0/3715178743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 -- 172.21.15.131:0/3715178743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 msgr2=0x55e13869d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 -- 172.21.15.131:0/3715178743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 msgr2=0x55e13868dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13868dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 -- 172.21.15.131:0/3715178743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1383ce620 con 0x55e13868bbd0 2024-04-26T20:29:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd753400640 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd752a00640 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868d7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd744006010 tx=0x7fd744005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6083dfc0619f98be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd743400640 1 -- 172.21.15.131:0/3715178743 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74405c070 con 0x55e13868bbd0 2024-04-26T20:29:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.701+0000 7fd743400640 1 -- 172.21.15.131:0/3715178743 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd744036040 con 0x55e13868bbd0 2024-04-26T20:29:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd743400640 1 -- 172.21.15.131:0/3715178743 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74404a040 con 0x55e13868bbd0 2024-04-26T20:29:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 -- 172.21.15.131:0/3715178743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 msgr2=0x55e13868d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868d7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd744006010 tx=0x7fd744005c50 comp rx=0 tx=0).stop 2024-04-26T20:29:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 -- 172.21.15.131:0/3715178743 shutdown_connections 2024-04-26T20:29:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13868dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 --2- 172.21.15.131:0/3715178743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13868d7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 -- 172.21.15.131:0/3715178743 >> 172.21.15.131:0/3715178743 conn(0x55e138687240 msgr2=0x55e138695c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 -- 172.21.15.131:0/3715178743 shutdown_connections 2024-04-26T20:29:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.702+0000 7fd75c582740 1 -- 172.21.15.131:0/3715178743 wait complete. 2024-04-26T20:29:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 -- start start 2024-04-26T20:29:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 Processor -- start 2024-04-26T20:29:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13869a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13869a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541abdb0 con 0x55e13868c5c0 2024-04-26T20:29:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541ad220 con 0x55e138691730 2024-04-26T20:29:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.703+0000 7fd75c582740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541ac480 con 0x55e13868bbd0 2024-04-26T20:29:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd75c582740 10 client.0 _ll_register_callbacks cb 0x7fd73c011d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13869a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13869a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13869a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45764/0 (socket says 172.21.15.131:45764) 2024-04-26T20:29:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752000640 1 -- 172.21.15.131:0/1680564253 learned_addr learned my addr 172.21.15.131:0/1680564253 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd753400640 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752a00640 1 -- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 msgr2=0x55e13869b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752a00640 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752a00640 1 -- 172.21.15.131:0/1680564253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 msgr2=0x55e13869a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752a00640 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13869a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752a00640 1 -- 172.21.15.131:0/1680564253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1385aacb0 con 0x55e13868bbd0 2024-04-26T20:29:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752000640 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13869a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd752a00640 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13869a460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd744009430 tx=0x7fd744057340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83b213c831d61bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.704+0000 7fd740c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:29:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd740c00640 1 -- 172.21.15.131:0/1680564253 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74405c070 con 0x55e13868bbd0 2024-04-26T20:29:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd744036040 con 0x55e13868bbd0 2024-04-26T20:29:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd740c00640 1 -- 172.21.15.131:0/1680564253 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd744057570 con 0x55e13868bbd0 2024-04-26T20:29:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd740c00640 1 -- 172.21.15.131:0/1680564253 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd744068050 con 0x55e13868bbd0 2024-04-26T20:29:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 10 client.26911 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e13869d090 con 0x55e13868bbd0 2024-04-26T20:29:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd740c00640 1 -- 172.21.15.131:0/1680564253 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd744036040 con 0x55e13868bbd0 2024-04-26T20:29:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd740c00640 1 -- 172.21.15.131:0/1680564253 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd744043070 con 0x55e13868bbd0 2024-04-26T20:29:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 10 client.26911 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 10 client.26911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 20 client.26911 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e138707d20 0x55e13870a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 4 client.26911 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:29:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.705+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1385f7e00 con 0x55e138707d20 2024-04-26T20:29:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.706+0000 7fd752000640 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e138707d20 0x55e13870a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.706+0000 7fd740c00640 10 client.26911 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.706+0000 7fd752000640 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e138707d20 0x55e13870a110 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1046f7c10580452 server_cookie=9f2947e721bcae39 in_seq=0 out_seq=0 2024-04-26T20:29:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd740c00640 1 -- 172.21.15.131:0/1680564253 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1385f7e00 con 0x55e138707d20 2024-04-26T20:29:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd740c00640 10 client.26911 handle_command_reply: tid=0 2024-04-26T20:29:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 client.26911 shutdown 2024-04-26T20:29:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 msgr2=0x55e13869a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13869a460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd744009430 tx=0x7fd744057340 comp rx=0 tx=0).stop 2024-04-26T20:29:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 shutdown_connections 2024-04-26T20:29:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55e138707d20 0x55e13870a110 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e138691730 0x55e13869b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e13868c5c0 0x55e13869a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 --2- 172.21.15.131:0/1680564253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e13868bbd0 0x55e13869a460 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd740c00640 0 client.26911 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 >> 172.21.15.131:0/1680564253 conn(0x55e138687240 msgr2=0x55e138695c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 shutdown_connections 2024-04-26T20:29:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 1 -- 172.21.15.131:0/1680564253 wait complete. 2024-04-26T20:29:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.710+0000 7fd75c582740 20 client.26911 trim_cache size 0 max 16384 2024-04-26T20:29:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:29:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:29:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:29:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7380a3cf0 msgr2=0x7fd7380a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7380a3cf0 0x7fd7380a61b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55e13868ce20 tx=0x7fd73c013040 comp rx=0 tx=0).stop 2024-04-26T20:29:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754106760 msgr2=0x7fd7541a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754106760 0x7fd7541a22b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00e970 tx=0x7fd74c00ee30 comp rx=0 tx=0).stop 2024-04-26T20:29:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 shutdown_connections 2024-04-26T20:29:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd754106760 0x7fd7541a22b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd754105da0 0x7fd75419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd7380a3cf0 0x7fd7380a61b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 --2- 172.21.15.131:0/1694293840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd754104bb0 0x7fd75419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 >> 172.21.15.131:0/1694293840 conn(0x7fd754100120 msgr2=0x7fd754109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 shutdown_connections 2024-04-26T20:29:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:03.711+0000 7fd759800640 1 -- 172.21.15.131:0/1694293840 wait complete. 2024-04-26T20:29:20.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:03 smithi131 ceph-mon[27919]: pgmap v981: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T20:29:20.161 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:29:20.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:29:20.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:29:20.235 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:05 smithi131 ceph-mon[27919]: pgmap v982: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T20:29:20.244 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:15 smithi148 ceph-mon[32281]: pgmap v987: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T20:29:20.253 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:17 smithi148 ceph-mon[32281]: pgmap v988: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:29:20.262 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:19 smithi148 ceph-mon[32281]: pgmap v989: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T20:29:20.270 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:15 smithi157 ceph-mon[32703]: pgmap v987: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T20:29:20.328 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:17 smithi157 ceph-mon[32703]: pgmap v988: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:29:20.390 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:19 smithi157 ceph-mon[32703]: pgmap v989: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T20:29:20.399 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-04-26T20:29:20.407 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T20:29:20.415 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:07 smithi131 ceph-mon[27919]: pgmap v983: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:20.423 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:09 smithi131 ceph-mon[27919]: pgmap v984: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:29:20.429 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:11 smithi131 ceph-mon[27919]: pgmap v985: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T20:29:20.474 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:13 smithi131 ceph-mon[27919]: pgmap v986: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T20:29:20.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:15 smithi131 ceph-mon[27919]: pgmap v987: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T20:29:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.852+0000 7f27b6400640 1 -- 172.21.15.131:0/3659575014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0103310 msgr2=0x7f27b0105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.852+0000 7f27b6400640 1 --2- 172.21.15.131:0/3659575014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0103310 0x7f27b0105700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009860 tx=0x7f27a402a340 comp rx=0 tx=0).stop 2024-04-26T20:29:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.852+0000 7f27b6400640 1 -- 172.21.15.131:0/3659575014 shutdown_connections 2024-04-26T20:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.853+0000 7f27b6400640 1 --2- 172.21.15.131:0/3659575014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b0108570 0x7f27b010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.853+0000 7f27b6400640 1 --2- 172.21.15.131:0/3659575014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0105c40 0x7f27b0108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.853+0000 7f27b6400640 1 --2- 172.21.15.131:0/3659575014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0103310 0x7f27b0105700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.853+0000 7f27b6400640 1 -- 172.21.15.131:0/3659575014 >> 172.21.15.131:0/3659575014 conn(0x7f27b00fcc60 msgr2=0x7f27b00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.853+0000 7f27b6400640 1 -- 172.21.15.131:0/3659575014 shutdown_connections 2024-04-26T20:29:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.853+0000 7f27b6400640 1 -- 172.21.15.131:0/3659575014 wait complete. 2024-04-26T20:29:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 Processor -- start 2024-04-26T20:29:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 -- start start 2024-04-26T20:29:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b0103310 0x7f27b019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0105c40 0x7f27b019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0108570 0x7f27b01a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27b010e310 con 0x7f27b0105c40 2024-04-26T20:29:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27b010e190 con 0x7f27b0103310 2024-04-26T20:29:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b6400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27b010e490 con 0x7f27b0108570 2024-04-26T20:29:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0105c40 0x7f27b019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0105c40 0x7f27b019b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33920/0 (socket says 172.21.15.131:33920) 2024-04-26T20:29:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27afe00640 1 -- 172.21.15.131:0/512614418 learned_addr learned my addr 172.21.15.131:0/512614418 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b5000640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b0103310 0x7f27b019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.854+0000 7f27b5a00640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0108570 0x7f27b01a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27b5000640 1 -- 172.21.15.131:0/512614418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0108570 msgr2=0x7f27b01a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27b5000640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0108570 0x7f27b01a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27b5000640 1 -- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0105c40 msgr2=0x7f27b019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27b5000640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0105c40 0x7f27b019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27b5000640 1 -- 172.21.15.131:0/512614418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b01a2860 con 0x7f27b0103310 2024-04-26T20:29:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27b5000640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b0103310 0x7f27b019ac50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f27a402a860 tx=0x7f27a4032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27ad600640 1 -- 172.21.15.131:0/512614418 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4037040 con 0x7f27b0103310 2024-04-26T20:29:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27ad600640 1 -- 172.21.15.131:0/512614418 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a4030bd0 con 0x7f27b0103310 2024-04-26T20:29:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b01a2af0 con 0x7f27b0103310 2024-04-26T20:29:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.855+0000 7f27ad600640 1 -- 172.21.15.131:0/512614418 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4003850 con 0x7f27b0103310 2024-04-26T20:29:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.856+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01a2f80 con 0x7f27b0103310 2024-04-26T20:29:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ad600640 1 -- 172.21.15.131:0/512614418 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f27a4043020 con 0x7f27b0103310 2024-04-26T20:29:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 Processor -- start 2024-04-26T20:29:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 -- start start 2024-04-26T20:29:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fdde0 0x559fcc1fe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc1fe7d0 0x559fcc2032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc203940 0x559fcc205dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559fcc162380 con 0x559fcc203940 2024-04-26T20:29:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fcc0fec30 con 0x559fcc1fdde0 2024-04-26T20:29:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.859+0000 7f27ba84f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559fcc15e340 con 0x559fcc1fe7d0 2024-04-26T20:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fdde0 0x559fcc1fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc203940 0x559fcc205dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fdde0 0x559fcc1fe200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:58056/0 (socket says 172.21.15.131:58056) 2024-04-26T20:29:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 -- 172.21.15.131:0/616418078 learned_addr learned my addr 172.21.15.131:0/616418078 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5000640 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc1fe7d0 0x559fcc2032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 -- 172.21.15.131:0/616418078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc1fe7d0 msgr2=0x559fcc2032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc1fe7d0 0x559fcc2032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 -- 172.21.15.131:0/616418078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc203940 msgr2=0x559fcc205dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc203940 0x559fcc205dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27b5a00640 1 -- 172.21.15.131:0/616418078 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcc120940 con 0x559fcc1fdde0 2024-04-26T20:29:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.860+0000 7f27ad600640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27980a3c30 0x7f27980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.861+0000 7f27b5a00640 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fdde0 0x559fcc1fe200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009fd0 tx=0x7f27a80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d97c3f9b727f446 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.861+0000 7f27ad600640 1 -- 172.21.15.131:0/512614418 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f27a40ef9f0 con 0x7f27b0103310 2024-04-26T20:29:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.861+0000 7f27b5000640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27980a3c30 0x7f27980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.861+0000 7f27a2000640 1 -- 172.21.15.131:0/616418078 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8004d20 con 0x559fcc1fdde0 2024-04-26T20:29:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.861+0000 7f27a2000640 1 -- 172.21.15.131:0/616418078 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a8004050 con 0x559fcc1fdde0 2024-04-26T20:29:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.861+0000 7f27a2000640 1 -- 172.21.15.131:0/616418078 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8005280 con 0x559fcc1fdde0 2024-04-26T20:29:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 -- 172.21.15.131:0/616418078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fdde0 msgr2=0x559fcc1fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27b5000640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27980a3c30 0x7f27980a60f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559fcc1ff240 tx=0x7f27a40f5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fdde0 0x559fcc1fe200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009fd0 tx=0x7f27a80118d0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 -- 172.21.15.131:0/616418078 shutdown_connections 2024-04-26T20:29:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc203940 0x559fcc205dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc1fe7d0 0x559fcc2032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 --2- 172.21.15.131:0/616418078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fdde0 0x559fcc1fe200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 -- 172.21.15.131:0/616418078 >> 172.21.15.131:0/616418078 conn(0x559fcc1f9450 msgr2=0x559fcc1f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 -- 172.21.15.131:0/616418078 shutdown_connections 2024-04-26T20:29:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 -- 172.21.15.131:0/616418078 wait complete. 2024-04-26T20:29:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 -- start start 2024-04-26T20:29:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.862+0000 7f27ba84f740 1 Processor -- start 2024-04-26T20:29:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc208050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc208590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27ba84f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559fcc2068f0 con 0x559fcc1fdde0 2024-04-26T20:29:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27ba84f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fcc0e9c60 con 0x559fcc1fe7d0 2024-04-26T20:29:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27ba84f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559fcc206a70 con 0x559fcc203940 2024-04-26T20:29:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27ba84f740 10 client.0 _ll_register_callbacks cb 0x559fcc115e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc208050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc208050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33938/0 (socket says 172.21.15.131:33938) 2024-04-26T20:29:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27b5a00640 1 -- 172.21.15.131:0/3556246914 learned_addr learned my addr 172.21.15.131:0/3556246914 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.863+0000 7f27afe00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc208590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27b5a00640 1 -- 172.21.15.131:0/3556246914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 msgr2=0x559fcc20f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27b5a00640 1 -- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 msgr2=0x559fcc208590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc208590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27b5a00640 1 -- 172.21.15.131:0/3556246914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcc0f8510 con 0x559fcc1fdde0 2024-04-26T20:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27afe00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc208590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc208050 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009b00 tx=0x7f27a8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca3ac85c89b3f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f2796a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f2796a00640 1 -- 172.21.15.131:0/3556246914 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8004c50 con 0x559fcc1fdde0 2024-04-26T20:29:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f2796a00640 1 -- 172.21.15.131:0/3556246914 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f27a8004df0 con 0x559fcc1fdde0 2024-04-26T20:29:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.864+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a8004050 con 0x559fcc1fdde0 2024-04-26T20:29:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f2796a00640 1 -- 172.21.15.131:0/3556246914 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27a8033050 con 0x559fcc1fdde0 2024-04-26T20:29:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 10 client.16818 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fcbe74210 con 0x559fcc1fdde0 2024-04-26T20:29:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f2796a00640 1 -- 172.21.15.131:0/3556246914 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a8004050 con 0x559fcc1fdde0 2024-04-26T20:29:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f2796a00640 1 -- 172.21.15.131:0/3556246914 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27a8005040 con 0x559fcc1fdde0 2024-04-26T20:29:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 10 client.16818 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 20 client.16818 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc213930 0x559fcc215d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 4 client.16818 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.865+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fcc16ff90 con 0x559fcc213930 2024-04-26T20:29:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.866+0000 7f27afe00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc213930 0x559fcc215d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.866+0000 7f2796a00640 10 client.16818 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.866+0000 7f27afe00640 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc213930 0x559fcc215d20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae154a673770c39 server_cookie=679034133a5678f7 in_seq=0 out_seq=0 2024-04-26T20:29:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.867+0000 7f2796a00640 1 -- 172.21.15.131:0/3556246914 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fcc16ff90 con 0x559fcc213930 2024-04-26T20:29:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.867+0000 7f2796a00640 10 client.16818 handle_command_reply: tid=0 2024-04-26T20:29:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.867+0000 7f27ba84f740 1 client.16818 shutdown 2024-04-26T20:29:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 msgr2=0x559fcc208050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc208050 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009b00 tx=0x7f27a8009a90 comp rx=0 tx=0).stop 2024-04-26T20:29:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 shutdown_connections 2024-04-26T20:29:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc213930 0x559fcc215d20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc208590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f2796a00640 0 client.16818 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3556246914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc208050 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 >> 172.21.15.131:0/3556246914 conn(0x559fcc1f9450 msgr2=0x559fcc1f8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 shutdown_connections 2024-04-26T20:29:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 1 -- 172.21.15.131:0/3556246914 wait complete. 2024-04-26T20:29:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.868+0000 7f27ba84f740 20 client.16818 trim_cache size 0 max 16384 2024-04-26T20:29:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 Processor -- start 2024-04-26T20:29:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 -- start start 2024-04-26T20:29:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc1fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc208370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01b0010 con 0x559fcc1fdde0 2024-04-26T20:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ad8e0 con 0x559fcc1fe7d0 2024-04-26T20:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.884+0000 7f27ba84f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ab730 con 0x559fcc203940 2024-04-26T20:29:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc1fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc1fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33940/0 (socket says 172.21.15.131:33940) 2024-04-26T20:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 -- 172.21.15.131:0/3435243184 learned_addr learned my addr 172.21.15.131:0/3435243184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27afe00640 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc208370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 -- 172.21.15.131:0/3435243184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 msgr2=0x559fcc208370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc208370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 -- 172.21.15.131:0/3435243184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 msgr2=0x559fcc20f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 -- 172.21.15.131:0/3435243184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcbf40620 con 0x559fcc1fdde0 2024-04-26T20:29:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.885+0000 7f27b5a00640 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc1fcc90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f27a8005d70 tx=0x7f27a8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5637a379671c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27a1600640 1 -- 172.21.15.131:0/3435243184 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8003fc0 con 0x559fcc1fdde0 2024-04-26T20:29:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27a1600640 1 -- 172.21.15.131:0/3435243184 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a80047a0 con 0x559fcc1fdde0 2024-04-26T20:29:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27a1600640 1 -- 172.21.15.131:0/3435243184 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8005680 con 0x559fcc1fdde0 2024-04-26T20:29:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 -- 172.21.15.131:0/3435243184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 msgr2=0x559fcc1fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc1fcc90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f27a8005d70 tx=0x7f27a8014040 comp rx=0 tx=0).stop 2024-04-26T20:29:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 -- 172.21.15.131:0/3435243184 shutdown_connections 2024-04-26T20:29:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc208370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 --2- 172.21.15.131:0/3435243184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc1fcc90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 -- 172.21.15.131:0/3435243184 >> 172.21.15.131:0/3435243184 conn(0x559fcc1f9450 msgr2=0x559fcc207c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 -- 172.21.15.131:0/3435243184 shutdown_connections 2024-04-26T20:29:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.886+0000 7f27ba84f740 1 -- 172.21.15.131:0/3435243184 wait complete. 2024-04-26T20:29:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 -- start start 2024-04-26T20:29:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 Processor -- start 2024-04-26T20:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc20c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01abda0 con 0x559fcc1fdde0 2024-04-26T20:29:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ad210 con 0x559fcc1fe7d0 2024-04-26T20:29:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ac470 con 0x559fcc203940 2024-04-26T20:29:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.887+0000 7f27ba84f740 10 client.0 _ll_register_callbacks cb 0x7f279c009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc20c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27b5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc20c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33942/0 (socket says 172.21.15.131:33942) 2024-04-26T20:29:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27b5a00640 1 -- 172.21.15.131:0/1283756814 learned_addr learned my addr 172.21.15.131:0/1283756814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27afe00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27b5a00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27afe00640 1 -- 172.21.15.131:0/1283756814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 msgr2=0x559fcc20dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27afe00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27afe00640 1 -- 172.21.15.131:0/1283756814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 msgr2=0x559fcc20c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27afe00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc20c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27afe00640 1 -- 172.21.15.131:0/1283756814 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcc11ccb0 con 0x559fcc1fe7d0 2024-04-26T20:29:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27b5a00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc20c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.888+0000 7f27afe00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20ced0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x559fcc216550 tx=0x7f279c0108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9558d60fa17e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f2796a00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:29:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f2796a00640 1 -- 172.21.15.131:0/1283756814 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c020430 con 0x559fcc1fe7d0 2024-04-26T20:29:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f2796a00640 1 -- 172.21.15.131:0/1283756814 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f279c0205d0 con 0x559fcc1fe7d0 2024-04-26T20:29:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a80047a0 con 0x559fcc1fe7d0 2024-04-26T20:29:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f2796a00640 1 -- 172.21.15.131:0/1283756814 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f279c02f050 con 0x559fcc1fe7d0 2024-04-26T20:29:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f27ba84f740 10 client.26948 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fcc20f680 con 0x559fcc1fe7d0 2024-04-26T20:29:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.889+0000 7f2796a00640 1 -- 172.21.15.131:0/1283756814 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a80047a0 con 0x559fcc1fe7d0 2024-04-26T20:29:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f2796a00640 1 -- 172.21.15.131:0/1283756814 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f279c01d3f0 con 0x559fcc1fe7d0 2024-04-26T20:29:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f27ba84f740 10 client.26948 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f27ba84f740 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f27ba84f740 20 client.26948 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f27ba84f740 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc27a470 0x559fcc27c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f27ba84f740 4 client.26948 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T20:29:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fcc169e00 con 0x559fcc27a470 2024-04-26T20:29:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.890+0000 7f27b5a00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc27a470 0x559fcc27c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.891+0000 7f2796a00640 10 client.26948 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.891+0000 7f27b5a00640 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc27a470 0x559fcc27c890 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d5eb5aa747ac7c server_cookie=85d9db49ac325496 in_seq=0 out_seq=0 2024-04-26T20:29:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f2796a00640 1 -- 172.21.15.131:0/1283756814 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fcc169e00 con 0x559fcc27a470 2024-04-26T20:29:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f2796a00640 10 client.26948 handle_command_reply: tid=0 2024-04-26T20:29:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 client.26948 shutdown 2024-04-26T20:29:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 msgr2=0x559fcc20ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20ced0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x559fcc216550 tx=0x7f279c0108c0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 shutdown_connections 2024-04-26T20:29:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x559fcc27a470 0x559fcc27c890 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559fcc203940 0x559fcc20dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f2796a00640 0 client.26948 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fcc1fe7d0 0x559fcc20ced0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 --2- 172.21.15.131:0/1283756814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559fcc1fdde0 0x559fcc20c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.900+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 >> 172.21.15.131:0/1283756814 conn(0x559fcc1f9450 msgr2=0x559fcc1fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.901+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 shutdown_connections 2024-04-26T20:29:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.901+0000 7f27ba84f740 1 -- 172.21.15.131:0/1283756814 wait complete. 2024-04-26T20:29:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.901+0000 7f27ba84f740 20 client.26948 trim_cache size 0 max 16384 2024-04-26T20:29:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:{ 2024-04-26T20:29:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "message": "", 2024-04-26T20:29:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout: "return_code": 0 2024-04-26T20:29:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stdout:} 2024-04-26T20:29:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.901+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27980a3c30 msgr2=0x7f27980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.901+0000 7f27b6400640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27980a3c30 0x7f27980a60f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559fcc1ff240 tx=0x7f27a40f5530 comp rx=0 tx=0).stop 2024-04-26T20:29:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.901+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b0103310 msgr2=0x7f27b019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.901+0000 7f27b6400640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b0103310 0x7f27b019ac50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f27a402a860 tx=0x7f27a4032dc0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 shutdown_connections 2024-04-26T20:29:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f27980a3c30 0x7f27980a60f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b0108570 0x7f27b01a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0105c40 0x7f27b019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 --2- 172.21.15.131:0/512614418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b0103310 0x7f27b019ac50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 >> 172.21.15.131:0/512614418 conn(0x7f27b00fcc60 msgr2=0x7f27b0102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 shutdown_connections 2024-04-26T20:29:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi131.front.sepia.ceph.com.smithi131.stderr:2024-04-26T20:29:16.902+0000 7f27b6400640 1 -- 172.21.15.131:0/512614418 wait complete. 2024-04-26T20:29:24.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:17 smithi131 ceph-mon[27919]: pgmap v988: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T20:29:24.997 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:19 smithi131 ceph-mon[27919]: pgmap v989: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T20:29:25.007 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:21 smithi148 ceph-mon[32281]: pgmap v990: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T20:29:25.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:23 smithi148 ceph-mon[32281]: pgmap v991: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T20:29:25.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:21 smithi157 ceph-mon[32703]: pgmap v990: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T20:29:25.085 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:23 smithi157 ceph-mon[32703]: pgmap v991: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T20:29:25.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 -- 172.21.15.131:0/3978342485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 msgr2=0x7f4ac0104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:25.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/3978342485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac0104fb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009900 tx=0x7f4ab002a400 comp rx=0 tx=0).stop 2024-04-26T20:29:25.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 -- 172.21.15.131:0/3978342485 shutdown_connections 2024-04-26T20:29:25.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/3978342485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ac0106740 0x7f4ac010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/3978342485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0105d80 0x7f4ac0106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:25.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/3978342485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac0104fb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:25.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 -- 172.21.15.131:0/3978342485 >> 172.21.15.131:0/3978342485 conn(0x7f4ac0100120 msgr2=0x7f4ac0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:25.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 -- 172.21.15.131:0/3978342485 shutdown_connections 2024-04-26T20:29:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.628+0000 7f4ac6a00640 1 -- 172.21.15.131:0/3978342485 wait complete. 2024-04-26T20:29:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 Processor -- start 2024-04-26T20:29:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 -- start start 2024-04-26T20:29:25.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ac0105d80 0x7f4ac019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0106740 0x7f4ac01a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01106e0 con 0x7f4ac0104b90 2024-04-26T20:29:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0110560 con 0x7f4ac0105d80 2024-04-26T20:29:25.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.629+0000 7f4ac6a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0110860 con 0x7f4ac0106740 2024-04-26T20:29:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:25.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33966/0 (socket says 172.21.15.131:33966) 2024-04-26T20:29:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 -- 172.21.15.131:0/4109850293 learned_addr learned my addr 172.21.15.131:0/4109850293 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abf400640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ac0105d80 0x7f4ac019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:25.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0106740 0x7f4ac01a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 -- 172.21.15.131:0/4109850293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0106740 msgr2=0x7f4ac01a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:25.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0106740 0x7f4ac01a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 -- 172.21.15.131:0/4109850293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ac0105d80 msgr2=0x7f4ac019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:25.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ac0105d80 0x7f4ac019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 -- 172.21.15.131:0/4109850293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac01a2a30 con 0x7f4ac0104b90 2024-04-26T20:29:25.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.630+0000 7f4abfe00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac019ae10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009850 tx=0x7f4ab0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:25.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.631+0000 7f4abcc00640 1 -- 172.21.15.131:0/4109850293 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab003f070 con 0x7f4ac0104b90 2024-04-26T20:29:25.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.631+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac01a2cc0 con 0x7f4ac0104b90 2024-04-26T20:29:25.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.631+0000 7f4abcc00640 1 -- 172.21.15.131:0/4109850293 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab0031c50 con 0x7f4ac0104b90 2024-04-26T20:29:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.631+0000 7f4abcc00640 1 -- 172.21.15.131:0/4109850293 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0035030 con 0x7f4ac0104b90 2024-04-26T20:29:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.631+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac01a31b0 con 0x7f4ac0104b90 2024-04-26T20:29:25.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.634+0000 7f4ac982b740 1 Processor -- start 2024-04-26T20:29:25.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.634+0000 7f4abcc00640 1 -- 172.21.15.131:0/4109850293 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4ab0042070 con 0x7f4ac0104b90 2024-04-26T20:29:25.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.634+0000 7f4ac982b740 1 -- start start 2024-04-26T20:29:25.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.634+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e6790 0x5608498e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:25.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:25.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498ee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:25.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac982b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5608497f7030 con 0x5608498ec2f0 2024-04-26T20:29:25.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac982b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5608498127f0 con 0x5608498e6790 2024-04-26T20:29:25.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac982b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5608497cd7e0 con 0x5608498e7180 2024-04-26T20:29:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e6790 0x5608498e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e6790 0x5608498e6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:58102/0 (socket says 172.21.15.131:58102) 2024-04-26T20:29:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 -- 172.21.15.131:0/1054919996 learned_addr learned my addr 172.21.15.131:0/1054919996 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:25.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4abf400640 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:26.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4abfe00640 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:26.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 -- 172.21.15.131:0/1054919996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 msgr2=0x5608498ebc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498ebc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 -- 172.21.15.131:0/1054919996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 msgr2=0x5608498ee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:26.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498ee780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4ac4a00640 1 -- 172.21.15.131:0/1054919996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608497d9050 con 0x5608498e6790 2024-04-26T20:29:26.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.635+0000 7f4abfe00640 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498ebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:26.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.636+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e6790 0x5608498e6bb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009fd0 tx=0x7f4ab80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acccedfb51073ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:26.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.636+0000 7f4abcc00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4aa40a3d00 0x7f4aa40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.636+0000 7f4ab5200640 1 -- 172.21.15.131:0/1054919996 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8004d20 con 0x5608498e6790 2024-04-26T20:29:26.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.636+0000 7f4ab5200640 1 -- 172.21.15.131:0/1054919996 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab8004050 con 0x5608498e6790 2024-04-26T20:29:26.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.636+0000 7f4ab5200640 1 -- 172.21.15.131:0/1054919996 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8005280 con 0x5608498e6790 2024-04-26T20:29:26.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.636+0000 7f4abcc00640 1 -- 172.21.15.131:0/4109850293 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4ab00f11c0 con 0x7f4ac0104b90 2024-04-26T20:29:26.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.636+0000 7f4abfe00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4aa40a3d00 0x7f4aa40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:26.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 -- 172.21.15.131:0/1054919996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e6790 msgr2=0x5608498e6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e6790 0x5608498e6bb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009fd0 tx=0x7f4ab80118d0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4abfe00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4aa40a3d00 0x7f4aa40a61c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5608498e7580 tx=0x7f4ab0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:26.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 -- 172.21.15.131:0/1054919996 shutdown_connections 2024-04-26T20:29:26.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498ebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1054919996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e6790 0x5608498e6bb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 -- 172.21.15.131:0/1054919996 >> 172.21.15.131:0/1054919996 conn(0x5608498e1db0 msgr2=0x5608498e21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:26.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 -- 172.21.15.131:0/1054919996 shutdown_connections 2024-04-26T20:29:26.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 -- 172.21.15.131:0/1054919996 wait complete. 2024-04-26T20:29:26.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 -- start start 2024-04-26T20:29:26.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.637+0000 7f4ac982b740 1 Processor -- start 2024-04-26T20:29:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:26.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:26.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac982b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56084984f200 con 0x5608498ec2f0 2024-04-26T20:29:26.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac982b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560849849ee0 con 0x5608498e7180 2024-04-26T20:29:26.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac982b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560849856e70 con 0x5608498e6790 2024-04-26T20:29:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac982b740 10 client.0 _ll_register_callbacks cb 0x560849840270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:26.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:26.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:26.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4abf400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:26.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51642/0 (socket says 172.21.15.131:51642) 2024-04-26T20:29:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.638+0000 7f4ac4a00640 1 -- 172.21.15.131:0/4201235049 learned_addr learned my addr 172.21.15.131:0/4201235049 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:26.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4abf400640 1 -- 172.21.15.131:0/4201235049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 msgr2=0x5608498f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:26.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4abf400640 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4abf400640 1 -- 172.21.15.131:0/4201235049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 msgr2=0x5608498f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4abf400640 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:26.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4abf400640 1 -- 172.21.15.131:0/4201235049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608496267d0 con 0x5608498e7180 2024-04-26T20:29:26.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4abf400640 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f0b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ac019c320 tx=0x7f4aac012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a1bdf304f49915 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:26.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4aa9600640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:29:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4aa9600640 1 -- 172.21.15.131:0/4201235049 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00f920 con 0x5608498e7180 2024-04-26T20:29:26.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4aa9600640 1 -- 172.21.15.131:0/4201235049 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4aac00fac0 con 0x5608498e7180 2024-04-26T20:29:26.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.639+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab8004050 con 0x5608498e7180 2024-04-26T20:29:26.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4aa9600640 1 -- 172.21.15.131:0/4201235049 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4aac00fcc0 con 0x5608498e7180 2024-04-26T20:29:26.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 10 client.26960 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:27.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56084955c3f0 con 0x5608498e7180 2024-04-26T20:29:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4aa9600640 1 -- 172.21.15.131:0/4201235049 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab8004050 con 0x5608498e7180 2024-04-26T20:29:27.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4aa9600640 1 -- 172.21.15.131:0/4201235049 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4aac01c420 con 0x5608498e7180 2024-04-26T20:29:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 10 client.26960 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:27.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:27.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 20 client.26960 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:27.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5608498fc1c0 0x5608498fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:27.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 4 client.26960 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:27.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.640+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608498587e0 con 0x5608498fc1c0 2024-04-26T20:29:27.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.641+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5608498fc1c0 0x5608498fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.641+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5608498fc1c0 0x5608498fe5e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3caf1de1284e7f server_cookie=5e3116f3d8f0e5bf in_seq=0 out_seq=0 2024-04-26T20:29:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.641+0000 7f4aa9600640 10 client.26960 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4aa9600640 1 -- 172.21.15.131:0/4201235049 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608498587e0 con 0x5608498fc1c0 2024-04-26T20:29:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4aa9600640 10 client.26960 handle_command_reply: tid=0 2024-04-26T20:29:27.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 client.26960 shutdown 2024-04-26T20:29:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 msgr2=0x5608498f0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:27.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f0b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ac019c320 tx=0x7f4aac012b60 comp rx=0 tx=0).stop 2024-04-26T20:29:27.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 shutdown_connections 2024-04-26T20:29:27.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x5608498fc1c0 0x5608498fe5e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f0b50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4aa9600640 0 client.26960 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 --2- 172.21.15.131:0/4201235049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 >> 172.21.15.131:0/4201235049 conn(0x5608498e1db0 msgr2=0x5608498e1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:27.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 shutdown_connections 2024-04-26T20:29:27.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.642+0000 7f4ac982b740 1 -- 172.21.15.131:0/4201235049 wait complete. 2024-04-26T20:29:27.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.643+0000 7f4ac982b740 20 client.26960 trim_cache size 0 max 16384 2024-04-26T20:29:27.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 Processor -- start 2024-04-26T20:29:27.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 -- start start 2024-04-26T20:29:27.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498e6790 0x5608498efc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:27.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:27.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498ec2f0 0x5608498e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:27.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01adaa0 con 0x5608498e6790 2024-04-26T20:29:27.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ab8f0 con 0x5608498ec2f0 2024-04-26T20:29:27.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac982b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ad3d0 con 0x5608498e7180 2024-04-26T20:29:27.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498e6790 0x5608498efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498e6790 0x5608498efc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33992/0 (socket says 172.21.15.131:33992) 2024-04-26T20:29:27.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4ac4a00640 1 -- 172.21.15.131:0/1132741701 learned_addr learned my addr 172.21.15.131:0/1132741701 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:27.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.662+0000 7f4abf400640 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498ec2f0 0x5608498e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:27.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4abf400640 1 -- 172.21.15.131:0/1132741701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498ec2f0 msgr2=0x5608498e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4abf400640 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498ec2f0 0x5608498e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4abf400640 1 -- 172.21.15.131:0/1132741701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498e6790 msgr2=0x5608498efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:27.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4abf400640 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498e6790 0x5608498efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4abf400640 1 -- 172.21.15.131:0/1132741701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084951c0f0 con 0x5608498e7180 2024-04-26T20:29:27.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4abf400640 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498f7b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5608498dd980 tx=0x7f4aac009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8f4b4fd4f98a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:27.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.663+0000 7f4aabe00640 1 -- 172.21.15.131:0/1132741701 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00f920 con 0x5608498e7180 2024-04-26T20:29:27.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4aabe00640 1 -- 172.21.15.131:0/1132741701 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac00fac0 con 0x5608498e7180 2024-04-26T20:29:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4aabe00640 1 -- 172.21.15.131:0/1132741701 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac01c680 con 0x5608498e7180 2024-04-26T20:29:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 -- 172.21.15.131:0/1132741701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 msgr2=0x5608498f7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:27.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498f7b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5608498dd980 tx=0x7f4aac009f80 comp rx=0 tx=0).stop 2024-04-26T20:29:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 -- 172.21.15.131:0/1132741701 shutdown_connections 2024-04-26T20:29:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498ec2f0 0x5608498e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e7180 0x5608498f7b50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 --2- 172.21.15.131:0/1132741701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498e6790 0x5608498efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:27.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 -- 172.21.15.131:0/1132741701 >> 172.21.15.131:0/1132741701 conn(0x5608498e1db0 msgr2=0x5608498e1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:27.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 -- 172.21.15.131:0/1132741701 shutdown_connections 2024-04-26T20:29:28.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.664+0000 7f4ac982b740 1 -- 172.21.15.131:0/1132741701 wait complete. 2024-04-26T20:29:28.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 -- start start 2024-04-26T20:29:28.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 Processor -- start 2024-04-26T20:29:28.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:28.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:28.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ac630 con 0x5608498ec2f0 2024-04-26T20:29:28.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01abf60 con 0x5608498e7180 2024-04-26T20:29:28.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01acd00 con 0x5608498e6790 2024-04-26T20:29:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.665+0000 7f4ac982b740 10 client.0 _ll_register_callbacks cb 0x7f4aac009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:28.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:28.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4ac4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:58108/0 (socket says 172.21.15.131:58108) 2024-04-26T20:29:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 -- 172.21.15.131:0/793238556 learned_addr learned my addr 172.21.15.131:0/793238556 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:28.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 -- 172.21.15.131:0/793238556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 msgr2=0x5608498f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:28.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:28.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 -- 172.21.15.131:0/793238556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 msgr2=0x5608498f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:28.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:28.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 -- 172.21.15.131:0/793238556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560849812a30 con 0x5608498e7180 2024-04-26T20:29:28.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4abf400640 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f5fd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4aac01ca50 tx=0x7f4aac00f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6788aa06de358e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:28.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.666+0000 7f4aa9600640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-26T20:29:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4aa9600640 1 -- 172.21.15.131:0/793238556 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac005240 con 0x5608498e7180 2024-04-26T20:29:28.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4aa9600640 1 -- 172.21.15.131:0/793238556 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f4aac0053e0 con 0x5608498e7180 2024-04-26T20:29:28.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac00fac0 con 0x5608498e7180 2024-04-26T20:29:28.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4aa9600640 1 -- 172.21.15.131:0/793238556 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4aac005600 con 0x5608498e7180 2024-04-26T20:29:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4ac982b740 10 client.26966 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:28.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560849962c80 con 0x5608498e7180 2024-04-26T20:29:28.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4aa9600640 1 -- 172.21.15.131:0/793238556 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac00fac0 con 0x5608498e7180 2024-04-26T20:29:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.667+0000 7f4aa9600640 1 -- 172.21.15.131:0/793238556 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4aac02a440 con 0x5608498e7180 2024-04-26T20:29:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac982b740 10 client.26966 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:28.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac982b740 10 client.26966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:28.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac982b740 20 client.26966 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:28.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac982b740 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560849963400 0x560849965820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac982b740 4 client.26966 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T20:29:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608498f0400 con 0x560849963400 2024-04-26T20:29:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560849963400 0x560849965820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4aa9600640 10 client.26966 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:28.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac4a00640 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560849963400 0x560849965820 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cfe717595d4334 server_cookie=c40fb9193846a5d3 in_seq=0 out_seq=0 2024-04-26T20:29:28.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4aa9600640 1 -- 172.21.15.131:0/793238556 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5608498f0400 con 0x560849963400 2024-04-26T20:29:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4aa9600640 10 client.26966 handle_command_reply: tid=0 2024-04-26T20:29:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.668+0000 7f4ac982b740 1 client.26966 shutdown 2024-04-26T20:29:28.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 msgr2=0x5608498f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f5fd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4aac01ca50 tx=0x7f4aac00f920 comp rx=0 tx=0).stop 2024-04-26T20:29:28.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 shutdown_connections 2024-04-26T20:29:28.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x560849963400 0x560849965820 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:28.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608498ec2f0 0x5608498f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:28.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4aa9600640 0 client.26966 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:28.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608498e7180 0x5608498f5fd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:28.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 --2- 172.21.15.131:0/793238556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608498e6790 0x5608498f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:28.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 >> 172.21.15.131:0/793238556 conn(0x5608498e1db0 msgr2=0x5608498e1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:28.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 shutdown_connections 2024-04-26T20:29:28.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 1 -- 172.21.15.131:0/793238556 wait complete. 2024-04-26T20:29:28.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.669+0000 7f4ac982b740 20 client.26966 trim_cache size 0 max 16384 2024-04-26T20:29:28.950 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:29:28.961 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-04-26T20:29:28.970 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-04-26T20:29:29.009 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:29:29.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4aa40a3d00 msgr2=0x7f4aa40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4aa40a3d00 0x7f4aa40a61c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5608498e7580 tx=0x7f4ab0000bc0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 msgr2=0x7f4ac019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac019ae10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009850 tx=0x7f4ab0003b90 comp rx=0 tx=0).stop 2024-04-26T20:29:29.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 shutdown_connections 2024-04-26T20:29:29.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f4aa40a3d00 0x7f4aa40a61c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0106740 0x7f4ac01a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ac0105d80 0x7f4ac019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 --2- 172.21.15.131:0/4109850293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac0104b90 0x7f4ac019ae10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 >> 172.21.15.131:0/4109850293 conn(0x7f4ac0100120 msgr2=0x7f4ac01091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:29.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 shutdown_connections 2024-04-26T20:29:29.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:20.670+0000 7f4ac6a00640 1 -- 172.21.15.131:0/4109850293 wait complete. 2024-04-26T20:29:29.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T20:29:29.235 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:29:29.244 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:21 smithi131 ceph-mon[27919]: pgmap v990: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T20:29:29.253 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:23 smithi131 ceph-mon[27919]: pgmap v991: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T20:29:29.363 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:25 smithi131 ceph-mon[27919]: pgmap v992: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T20:29:29.373 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:25 smithi157 ceph-mon[32703]: pgmap v992: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T20:29:29.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:27 smithi157 ceph-mon[32703]: pgmap v993: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-26T20:29:29.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:25 smithi148 ceph-mon[32281]: pgmap v992: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T20:29:29.399 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:27 smithi148 ceph-mon[32281]: pgmap v993: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-26T20:29:29.430 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:27 smithi131 ceph-mon[27919]: pgmap v993: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-26T20:29:29.436 DEBUG:teuthology.parallel:result is None 2024-04-26T20:29:29.444 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T20:29:29.576 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T20:29:29.584 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T20:29:29.667 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:29:29.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 -- 172.21.15.131:0/3071587490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc10a180 msgr2=0x7fbcbc10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 --2- 172.21.15.131:0/3071587490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc10a180 0x7fbcbc10a600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009900 tx=0x7fbca802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 -- 172.21.15.131:0/3071587490 shutdown_connections 2024-04-26T20:29:29.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 --2- 172.21.15.131:0/3071587490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcbc10ab40 0x7fbcbc1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 --2- 172.21.15.131:0/3071587490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc10a180 0x7fbcbc10a600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 --2- 172.21.15.131:0/3071587490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcbc108f90 0x7fbcbc1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:29.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 -- 172.21.15.131:0/3071587490 >> 172.21.15.131:0/3071587490 conn(0x7fbcbc077f40 msgr2=0x7fbcbc07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:29.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 -- 172.21.15.131:0/3071587490 shutdown_connections 2024-04-26T20:29:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.643+0000 7fbcc3400640 1 -- 172.21.15.131:0/3071587490 wait complete. 2024-04-26T20:29:29.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.644+0000 7fbcc3400640 1 Processor -- start 2024-04-26T20:29:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc3400640 1 -- start start 2024-04-26T20:29:29.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc3400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc108f90 0x7fbcbc19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:29.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc3400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcbc10a180 0x7fbcbc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:29.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcbc10ab40 0x7fbcbc1a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:29.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc3400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc114b20 con 0x7fbcbc10ab40 2024-04-26T20:29:29.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc3400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc1149a0 con 0x7fbcbc10a180 2024-04-26T20:29:29.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc3400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc114ca0 con 0x7fbcbc108f90 2024-04-26T20:29:30.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcbbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcbc10a180 0x7fbcbc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc108f90 0x7fbcbc19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:30.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcbbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcbc10a180 0x7fbcbc19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:51858/0 (socket says 172.21.15.131:51858) 2024-04-26T20:29:30.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcbbe00640 1 -- 172.21.15.131:0/19890247 learned_addr learned my addr 172.21.15.131:0/19890247 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:30.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.645+0000 7fbcc1400640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcbc10ab40 0x7fbcbc1a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:30.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcc0a00640 1 -- 172.21.15.131:0/19890247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcbc10a180 msgr2=0x7fbcbc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:30.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcc0a00640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcbc10a180 0x7fbcbc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:30.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcc0a00640 1 -- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcbc10ab40 msgr2=0x7fbcbc1a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:30.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcc0a00640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcbc10ab40 0x7fbcbc1a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:30.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcc0a00640 1 -- 172.21.15.131:0/19890247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc1a6d60 con 0x7fbcbc108f90 2024-04-26T20:29:30.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcc0a00640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc108f90 0x7fbcbc19f140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000e9c0 tx=0x7fbcb000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:30.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcb9600640 1 -- 172.21.15.131:0/19890247 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000cd80 con 0x7fbcbc108f90 2024-04-26T20:29:30.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc1a6ff0 con 0x7fbcbc108f90 2024-04-26T20:29:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcb9600640 1 -- 172.21.15.131:0/19890247 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb00108e0 con 0x7fbcbc108f90 2024-04-26T20:29:30.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.646+0000 7fbcb9600640 1 -- 172.21.15.131:0/19890247 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00187f0 con 0x7fbcbc108f90 2024-04-26T20:29:30.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.647+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc1a7500 con 0x7fbcbc108f90 2024-04-26T20:29:30.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.648+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc1102b0 con 0x7fbcbc108f90 2024-04-26T20:29:30.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.650+0000 7fbcb9600640 1 -- 172.21.15.131:0/19890247 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbcb001f020 con 0x7fbcbc108f90 2024-04-26T20:29:30.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.652+0000 7fbcb9600640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbca40a3c40 0x7fbca40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:30.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.652+0000 7fbcb9600640 1 -- 172.21.15.131:0/19890247 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbcb00d0900 con 0x7fbcbc108f90 2024-04-26T20:29:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.652+0000 7fbcbbe00640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbca40a3c40 0x7fbca40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:30.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.653+0000 7fbcbbe00640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbca40a3c40 0x7fbca40a6100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc1a0650 tx=0x7fbca8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:30.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.656+0000 7fbcb9600640 1 -- 172.21.15.131:0/19890247 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbcb009cd30 con 0x7fbcbc108f90 2024-04-26T20:29:30.419 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:29 smithi157 ceph-mon[32703]: pgmap v994: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T20:29:30.425 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:29 smithi148 ceph-mon[32281]: pgmap v994: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T20:29:30.435 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-26T20:29:30.456 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-26T20:29:30.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.902+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcbc02dd10 con 0x7fbcbc108f90 2024-04-26T20:29:30.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.904+0000 7fbcb9600640 1 -- 172.21.15.131:0/19890247 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7fbcb009c630 con 0x7fbcbc108f90 2024-04-26T20:29:30.521 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:29:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:12:47.696526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24571,24557]},"id":1}]} 2024-04-26T20:29:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-04-26T20:29:30.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbca40a3c40 msgr2=0x7fbca40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:30.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbca40a3c40 0x7fbca40a6100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc1a0650 tx=0x7fbca8005a20 comp rx=0 tx=0).stop 2024-04-26T20:29:30.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc108f90 msgr2=0x7fbcbc19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:30.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc108f90 0x7fbcbc19f140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000e9c0 tx=0x7fbcb000ee80 comp rx=0 tx=0).stop 2024-04-26T20:29:30.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 shutdown_connections 2024-04-26T20:29:30.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcbc10ab40 0x7fbcbc1a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:30.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcbc10a180 0x7fbcbc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:30.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbca40a3c40 0x7fbca40a6100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 --2- 172.21.15.131:0/19890247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcbc108f90 0x7fbcbc19f140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 >> 172.21.15.131:0/19890247 conn(0x7fbcbc077f40 msgr2=0x7fbcbc10d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 shutdown_connections 2024-04-26T20:29:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:29.907+0000 7fbcc3400640 1 -- 172.21.15.131:0/19890247 wait complete. 2024-04-26T20:29:30.741 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:29 smithi131 ceph-mon[27919]: pgmap v994: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T20:29:30.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T20:29:31.005 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:30 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/19890247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:29:31.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 -- 172.21.15.131:0/2657855974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 msgr2=0x7fab34108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:31.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 --2- 172.21.15.131:0/2657855974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 0x7fab34108fe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005fd0 tx=0x7fab2c02b330 comp rx=0 tx=0).stop 2024-04-26T20:29:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 -- 172.21.15.131:0/2657855974 shutdown_connections 2024-04-26T20:29:31.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 --2- 172.21.15.131:0/2657855974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 0x7fab34108fe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 --2- 172.21.15.131:0/2657855974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34101d90 0x7fab34102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:31.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 --2- 172.21.15.131:0/2657855974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab34100ba0 0x7fab34100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:31.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 -- 172.21.15.131:0/2657855974 >> 172.21.15.131:0/2657855974 conn(0x7fab340fc130 msgr2=0x7fab340fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:31.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 -- 172.21.15.131:0/2657855974 shutdown_connections 2024-04-26T20:29:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.038+0000 7fab3c600640 1 -- 172.21.15.131:0/2657855974 wait complete. 2024-04-26T20:29:31.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:30 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/19890247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:29:31.240 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:30 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/19890247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:29:31.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 Processor -- start 2024-04-26T20:29:31.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 -- start start 2024-04-26T20:29:31.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34100ba0 0x7fab3419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:31.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab34101d90 0x7fab3419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:31.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 0x7fab341a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410c200 con 0x7fab34100ba0 2024-04-26T20:29:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410c080 con 0x7fab34101d90 2024-04-26T20:29:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.039+0000 7fab3c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410c380 con 0x7fab34102750 2024-04-26T20:29:31.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34100ba0 0x7fab3419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:31.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34100ba0 0x7fab3419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51210/0 (socket says 172.21.15.131:51210) 2024-04-26T20:29:31.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 -- 172.21.15.131:0/3301587068 learned_addr learned my addr 172.21.15.131:0/3301587068 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:31.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab3a600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 0x7fab341a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39200640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab34101d90 0x7fab3419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:31.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 -- 172.21.15.131:0/3301587068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 msgr2=0x7fab341a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:31.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 0x7fab341a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:31.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 -- 172.21.15.131:0/3301587068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab34101d90 msgr2=0x7fab3419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:31.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab34101d90 0x7fab3419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab39c00640 1 -- 172.21.15.131:0/3301587068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab341a2990 con 0x7fab34100ba0 2024-04-26T20:29:31.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.040+0000 7fab3a600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 0x7fab341a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:31.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.041+0000 7fab39c00640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34100ba0 0x7fab3419ad70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9c0 tx=0x7fab2800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:31.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.041+0000 7fab32000640 1 -- 172.21.15.131:0/3301587068 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800ccc0 con 0x7fab34100ba0 2024-04-26T20:29:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.041+0000 7fab32000640 1 -- 172.21.15.131:0/3301587068 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab280035f0 con 0x7fab34100ba0 2024-04-26T20:29:31.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.041+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab341a2c80 con 0x7fab34100ba0 2024-04-26T20:29:31.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.041+0000 7fab32000640 1 -- 172.21.15.131:0/3301587068 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800ce60 con 0x7fab34100ba0 2024-04-26T20:29:31.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.041+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a31d0 con 0x7fab34100ba0 2024-04-26T20:29:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.045+0000 7fab3f42e740 1 Processor -- start 2024-04-26T20:29:31.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.045+0000 7fab3f42e740 1 -- start start 2024-04-26T20:29:31.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0251ea0 0x55cbf02522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab32000640 1 -- 172.21.15.131:0/3301587068 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fab280037a0 con 0x7fab34100ba0 2024-04-26T20:29:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0252890 0x55cbf0257390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:31.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0259e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3f42e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cbf01b5c70 con 0x55cbf0251ea0 2024-04-26T20:29:31.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3f42e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cbf016e980 con 0x55cbf0252890 2024-04-26T20:29:31.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3f42e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cbf01b1c30 con 0x55cbf0257a00 2024-04-26T20:29:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3a600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0251ea0 0x55cbf02522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:31.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3a600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0251ea0 0x55cbf02522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51222/0 (socket says 172.21.15.131:51222) 2024-04-26T20:29:31.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab3a600640 1 -- 172.21.15.131:0/3624513793 learned_addr learned my addr 172.21.15.131:0/3624513793 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:31.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab39c00640 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0252890 0x55cbf0257390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:31.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.046+0000 7fab39200640 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0259e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab3a600640 1 -- 172.21.15.131:0/3624513793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 msgr2=0x55cbf0259e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab3a600640 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0259e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:31.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab3a600640 1 -- 172.21.15.131:0/3624513793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0252890 msgr2=0x55cbf0257390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab3a600640 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0252890 0x55cbf0257390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab3a600640 1 -- 172.21.15.131:0/3624513793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf01467b0 con 0x55cbf0251ea0 2024-04-26T20:29:31.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab3a600640 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0251ea0 0x55cbf02522c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fab2c02b850 tx=0x7fab2c007440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d6cfd0e471682a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:31.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab26a00640 1 -- 172.21.15.131:0/3624513793 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c042070 con 0x55cbf0251ea0 2024-04-26T20:29:31.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab26a00640 1 -- 172.21.15.131:0/3624513793 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab2c007a50 con 0x55cbf0251ea0 2024-04-26T20:29:31.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab26a00640 1 -- 172.21.15.131:0/3624513793 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00a6e0 con 0x55cbf0251ea0 2024-04-26T20:29:31.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.047+0000 7fab32000640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fab1c0a3cc0 0x7fab1c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:31.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab32000640 1 -- 172.21.15.131:0/3301587068 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fab280d0c90 con 0x7fab34100ba0 2024-04-26T20:29:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab39c00640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fab1c0a3cc0 0x7fab1c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:32.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 -- 172.21.15.131:0/3624513793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0251ea0 msgr2=0x55cbf02522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:32.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0251ea0 0x55cbf02522c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fab2c02b850 tx=0x7fab2c007440 comp rx=0 tx=0).stop 2024-04-26T20:29:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 -- 172.21.15.131:0/3624513793 shutdown_connections 2024-04-26T20:29:32.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0259e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0252890 0x55cbf0257390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3624513793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0251ea0 0x55cbf02522c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 -- 172.21.15.131:0/3624513793 >> 172.21.15.131:0/3624513793 conn(0x55cbf024d510 msgr2=0x55cbf024d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 -- 172.21.15.131:0/3624513793 shutdown_connections 2024-04-26T20:29:32.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab3f42e740 1 -- 172.21.15.131:0/3624513793 wait complete. 2024-04-26T20:29:32.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.048+0000 7fab39c00640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fab1c0a3cc0 0x7fab1c0a6180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55cbf0253300 tx=0x7fab280d47e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.049+0000 7fab3f42e740 1 -- start start 2024-04-26T20:29:32.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.049+0000 7fab3f42e740 1 Processor -- start 2024-04-26T20:29:32.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf025bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:32.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf025c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:32.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf0263280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3f42e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cbf01b4f80 con 0x55cbf0252890 2024-04-26T20:29:32.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3f42e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cbf0162260 con 0x55cbf0257a00 2024-04-26T20:29:32.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3f42e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cbf01ba2a0 con 0x55cbf0251ea0 2024-04-26T20:29:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3f42e740 10 client.0 _ll_register_callbacks cb 0x55cbf01ac960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:32.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf025bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:32.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab39200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf025c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:32.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf025bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:49564/0 (socket says 172.21.15.131:49564) 2024-04-26T20:29:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab39200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf025c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51224/0 (socket says 172.21.15.131:51224) 2024-04-26T20:29:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 -- 172.21.15.131:0/2347824561 learned_addr learned my addr 172.21.15.131:0/2347824561 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:32.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf0263280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 -- 172.21.15.131:0/2347824561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 msgr2=0x55cbf0263280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:32.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf0263280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 -- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 msgr2=0x55cbf025c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:32.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf025c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab3a600640 1 -- 172.21.15.131:0/2347824561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf01441c0 con 0x55cbf0251ea0 2024-04-26T20:29:32.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.050+0000 7fab39200640 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf025c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab3a600640 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf025bd70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab2c033430 tx=0x7fab2c00aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d00a1e9550b020d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab1b400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-26T20:29:32.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab1b400640 1 -- 172.21.15.131:0/2347824561 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c042040 con 0x55cbf0251ea0 2024-04-26T20:29:32.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab1b400640 1 -- 172.21.15.131:0/2347824561 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fab2c00a680 con 0x55cbf0251ea0 2024-04-26T20:29:32.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2c007a50 con 0x55cbf0251ea0 2024-04-26T20:29:32.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab1b400640 1 -- 172.21.15.131:0/2347824561 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab2c004b40 con 0x55cbf0251ea0 2024-04-26T20:29:32.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab3f42e740 10 client.26965 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:32.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbefec73f0 con 0x55cbf0251ea0 2024-04-26T20:29:32.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.051+0000 7fab1b400640 1 -- 172.21.15.131:0/2347824561 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab2c007a50 con 0x55cbf0251ea0 2024-04-26T20:29:32.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab1b400640 1 -- 172.21.15.131:0/2347824561 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab2c0042a0 con 0x55cbf0251ea0 2024-04-26T20:29:32.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab3f42e740 10 client.26965 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:32.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab3f42e740 10 client.26965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-26T20:29:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab3f42e740 20 client.26965 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:32.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02673e0 0x55cbf02697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab3f42e740 4 client.26965 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf01c3880 con 0x55cbf02673e0 2024-04-26T20:29:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab39200640 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02673e0 0x55cbf02697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:32.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab39200640 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02673e0 0x55cbf02697d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85ec97f7c829e8f5 server_cookie=bd110b61dfc33f74 in_seq=0 out_seq=0 2024-04-26T20:29:32.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.052+0000 7fab1b400640 10 client.26965 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:29:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.053+0000 7fab1b400640 1 -- 172.21.15.131:0/2347824561 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbf01c3880 con 0x55cbf02673e0 2024-04-26T20:29:32.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.053+0000 7fab1b400640 10 client.26965 handle_command_reply: tid=0 2024-04-26T20:29:32.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.053+0000 7fab3f42e740 1 client.26965 shutdown 2024-04-26T20:29:32.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.053+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 msgr2=0x55cbf025bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:32.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.053+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf025bd70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab2c033430 tx=0x7fab2c00aea0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 shutdown_connections 2024-04-26T20:29:32.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02673e0 0x55cbf02697d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf0263280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf025c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2347824561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf025bd70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab1b400640 0 client.26965 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:29:33.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 >> 172.21.15.131:0/2347824561 conn(0x55cbf024d510 msgr2=0x55cbf024c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:33.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 shutdown_connections 2024-04-26T20:29:33.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 1 -- 172.21.15.131:0/2347824561 wait complete. 2024-04-26T20:29:33.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.054+0000 7fab3f42e740 20 client.26965 trim_cache size 0 max 16384 2024-04-26T20:29:33.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.068+0000 7fab3f42e740 1 Processor -- start 2024-04-26T20:29:33.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.068+0000 7fab3f42e740 1 -- start start 2024-04-26T20:29:33.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0251ea0 0x55cbf025c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0263240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:33.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0250bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3f42e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b0000 con 0x55cbf0252890 2024-04-26T20:29:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3f42e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ad8d0 con 0x55cbf0251ea0 2024-04-26T20:29:33.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3f42e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ab720 con 0x55cbf0257a00 2024-04-26T20:29:33.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0251ea0 0x55cbf025c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:33.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0251ea0 0x55cbf025c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:51922/0 (socket says 172.21.15.131:51922) 2024-04-26T20:29:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 -- 172.21.15.131:0/2777432811 learned_addr learned my addr 172.21.15.131:0/2777432811 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:33.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab39200640 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0263240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:33.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0250bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:33.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 -- 172.21.15.131:0/2777432811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 msgr2=0x55cbf0250bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:33.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0250bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:33.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 -- 172.21.15.131:0/2777432811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 msgr2=0x55cbf0263240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:33.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0263240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:33.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 -- 172.21.15.131:0/2777432811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbeff917d0 con 0x55cbf0251ea0 2024-04-26T20:29:33.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab39200640 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0263240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:33.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.069+0000 7fab3a600640 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0251ea0 0x55cbf025c210 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006010 tx=0x7fab2c00ac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166d6c967a32bfa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab26000640 1 -- 172.21.15.131:0/2777432811 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c042070 con 0x55cbf0251ea0 2024-04-26T20:29:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab26000640 1 -- 172.21.15.131:0/2777432811 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab2c007a50 con 0x55cbf0251ea0 2024-04-26T20:29:33.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab26000640 1 -- 172.21.15.131:0/2777432811 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00a6e0 con 0x55cbf0251ea0 2024-04-26T20:29:33.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab3f42e740 1 -- 172.21.15.131:0/2777432811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0251ea0 msgr2=0x55cbf025c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:33.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0251ea0 0x55cbf025c210 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006010 tx=0x7fab2c00ac10 comp rx=0 tx=0).stop 2024-04-26T20:29:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab3f42e740 1 -- 172.21.15.131:0/2777432811 shutdown_connections 2024-04-26T20:29:33.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0257a00 0x55cbf0250bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0263240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab3f42e740 1 --2- 172.21.15.131:0/2777432811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0251ea0 0x55cbf025c210 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:33.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.070+0000 7fab3f42e740 1 -- 172.21.15.131:0/2777432811 >> 172.21.15.131:0/2777432811 conn(0x55cbf024d510 msgr2=0x55cbf02504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:33.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 -- 172.21.15.131:0/2777432811 shutdown_connections 2024-04-26T20:29:33.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 -- 172.21.15.131:0/2777432811 wait complete. 2024-04-26T20:29:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 -- start start 2024-04-26T20:29:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 Processor -- start 2024-04-26T20:29:33.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf0260220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:33.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0260760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:33.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf02cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:33.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.071+0000 7fab3f42e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fab341abd90 con 0x55cbf0252890 2024-04-26T20:29:33.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab3f42e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ad200 con 0x55cbf0257a00 2024-04-26T20:29:33.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab3f42e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ac460 con 0x55cbf0251ea0 2024-04-26T20:29:33.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab3f42e740 10 client.0 _ll_register_callbacks cb 0x7fab280d46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:33.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0260760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab3a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf02cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:33.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0260760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51240/0 (socket says 172.21.15.131:51240) 2024-04-26T20:29:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 -- 172.21.15.131:0/3118655902 learned_addr learned my addr 172.21.15.131:0/3118655902 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab3a600640 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf0260220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:33.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 -- 172.21.15.131:0/3118655902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 msgr2=0x55cbf0260220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf0260220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 -- 172.21.15.131:0/3118655902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 msgr2=0x55cbf02cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:34.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf02cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 -- 172.21.15.131:0/3118655902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf015c2a0 con 0x55cbf0252890 2024-04-26T20:29:34.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab39200640 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0260760 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fab3419c280 tx=0x7fab2001f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca57b87d60c9bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.072+0000 7fab1b400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab1b400640 1 -- 172.21.15.131:0/3118655902 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20012d20 con 0x55cbf0252890 2024-04-26T20:29:34.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab1b400640 1 -- 172.21.15.131:0/3118655902 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fab20005020 con 0x55cbf0252890 2024-04-26T20:29:34.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2c007a50 con 0x55cbf0252890 2024-04-26T20:29:34.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab1b400640 1 -- 172.21.15.131:0/3118655902 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab2002f050 con 0x55cbf0252890 2024-04-26T20:29:34.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab3f42e740 10 client.16866 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbf02cd130 con 0x55cbf0252890 2024-04-26T20:29:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab1b400640 1 -- 172.21.15.131:0/3118655902 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab2c007a50 con 0x55cbf0252890 2024-04-26T20:29:34.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.073+0000 7fab1b400640 1 -- 172.21.15.131:0/3118655902 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab20010430 con 0x55cbf0252890 2024-04-26T20:29:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3f42e740 10 client.16866 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:34.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3f42e740 10 client.16866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-26T20:29:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3f42e740 20 client.16866 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:34.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02cdb60 0x55cbf02cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:34.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3f42e740 4 client.16866 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:29:34.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 --> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbf01bd6f0 con 0x55cbf02cdb60 2024-04-26T20:29:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3a600640 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02cdb60 0x55cbf02cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab3a600640 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02cdb60 0x55cbf02cff80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1224457dce0be4af server_cookie=abe147abf6ce7937 in_seq=0 out_seq=0 2024-04-26T20:29:34.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.074+0000 7fab1b400640 10 client.16866 ms_handle_connect on v2:172.21.15.131:6838/2686418281 2024-04-26T20:29:34.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab1b400640 1 -- 172.21.15.131:0/3118655902 <== mds.2 v2:172.21.15.131:6838/2686418281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbf01bd6f0 con 0x55cbf02cdb60 2024-04-26T20:29:34.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab1b400640 10 client.16866 handle_command_reply: tid=0 2024-04-26T20:29:34.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 client.16866 shutdown 2024-04-26T20:29:34.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 msgr2=0x55cbf0260760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:34.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0260760 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fab3419c280 tx=0x7fab2001f7d0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 shutdown_connections 2024-04-26T20:29:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:6838/2686418281,v1:172.21.15.131:6839/2686418281] conn(0x55cbf02cdb60 0x55cbf02cff80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf0257a00 0x55cbf02cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cbf0252890 0x55cbf0260760 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 --2- 172.21.15.131:0/3118655902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbf0251ea0 0x55cbf0260220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 >> 172.21.15.131:0/3118655902 conn(0x55cbf024d510 msgr2=0x55cbf025c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:34.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab1b400640 0 client.16866 ms_handle_reset on v2:172.21.15.131:6838/2686418281 2024-04-26T20:29:34.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 shutdown_connections 2024-04-26T20:29:34.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 1 -- 172.21.15.131:0/3118655902 wait complete. 2024-04-26T20:29:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.075+0000 7fab3f42e740 20 client.16866 trim_cache size 0 max 16384 2024-04-26T20:29:34.677 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:29:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fab1c0a3cc0 msgr2=0x7fab1c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:34.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fab1c0a3cc0 0x7fab1c0a6180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55cbf0253300 tx=0x7fab280d47e0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34100ba0 msgr2=0x7fab3419ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34100ba0 0x7fab3419ad70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9c0 tx=0x7fab2800edc0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 shutdown_connections 2024-04-26T20:29:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fab1c0a3cc0 0x7fab1c0a6180 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab34102750 0x7fab341a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab34101d90 0x7fab3419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 --2- 172.21.15.131:0/3301587068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab34100ba0 0x7fab3419ad70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:34.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 >> 172.21.15.131:0/3301587068 conn(0x7fab340fc130 msgr2=0x7fab341052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:34.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 shutdown_connections 2024-04-26T20:29:34.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:31.076+0000 7fab3c600640 1 -- 172.21.15.131:0/3301587068 wait complete. 2024-04-26T20:29:34.961 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T20:29:34.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:31 smithi131 ceph-mon[27919]: pgmap v995: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:29:34.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:33 smithi131 ceph-mon[27919]: pgmap v996: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:29:34.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:31 smithi157 ceph-mon[32703]: pgmap v995: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:29:34.993 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:33 smithi157 ceph-mon[32703]: pgmap v996: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:29:35.002 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:31 smithi148 ceph-mon[32281]: pgmap v995: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:29:35.113 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:33 smithi148 ceph-mon[32281]: pgmap v996: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T20:29:35.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 -- 172.21.15.131:0/1827990765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 msgr2=0x7f221c103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:35.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 --2- 172.21.15.131:0/1827990765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 0x7f221c103eb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2204009900 tx=0x7f220402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:29:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 -- 172.21.15.131:0/1827990765 shutdown_connections 2024-04-26T20:29:35.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 --2- 172.21.15.131:0/1827990765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c1043f0 0x7f221c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:35.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 --2- 172.21.15.131:0/1827990765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 0x7f221c103eb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:35.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 --2- 172.21.15.131:0/1827990765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c102840 0x7f221c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:35.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 -- 172.21.15.131:0/1827990765 >> 172.21.15.131:0/1827990765 conn(0x7f221c0fddd0 msgr2=0x7f221c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:35.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 -- 172.21.15.131:0/1827990765 shutdown_connections 2024-04-26T20:29:35.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.201+0000 7f2221400640 1 -- 172.21.15.131:0/1827990765 wait complete. 2024-04-26T20:29:35.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 Processor -- start 2024-04-26T20:29:35.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 -- start start 2024-04-26T20:29:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c102840 0x7f221c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:35.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 0x7f221c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:35.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c1043f0 0x7f221c1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:35.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f221c10e3c0 con 0x7f221c1043f0 2024-04-26T20:29:35.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f221c10e240 con 0x7f221c103a30 2024-04-26T20:29:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.203+0000 7f2221400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f221c10e540 con 0x7f221c102840 2024-04-26T20:29:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c1043f0 0x7f221c1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:35.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f221be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c102840 0x7f221c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:35.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c1043f0 0x7f221c1a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40146/0 (socket says 172.21.15.131:40146) 2024-04-26T20:29:35.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f2220a00640 1 -- 172.21.15.131:0/3630437732 learned_addr learned my addr 172.21.15.131:0/3630437732 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:35.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f2213e00640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 0x7f221c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:35.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f221be00640 1 -- 172.21.15.131:0/3630437732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 msgr2=0x7f221c19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:35.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f221be00640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 0x7f221c19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:35.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f221be00640 1 -- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c1043f0 msgr2=0x7f221c1a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:35.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f221be00640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c1043f0 0x7f221c1a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:35.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f221be00640 1 -- 172.21.15.131:0/3630437732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c1a2900 con 0x7f221c102840 2024-04-26T20:29:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f2220a00640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c1043f0 0x7f221c1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:35.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.204+0000 7f221be00640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c102840 0x7f221c19acf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f220c00e9c0 tx=0x7f220c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.205+0000 7f2219600640 1 -- 172.21.15.131:0/3630437732 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c00ccc0 con 0x7f221c102840 2024-04-26T20:29:35.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.205+0000 7f2219600640 1 -- 172.21.15.131:0/3630437732 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220c0035f0 con 0x7f221c102840 2024-04-26T20:29:35.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.205+0000 7f2219600640 1 -- 172.21.15.131:0/3630437732 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c00ce60 con 0x7f221c102840 2024-04-26T20:29:35.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.205+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c1a2bf0 con 0x7f221c102840 2024-04-26T20:29:35.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.205+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c1a3190 con 0x7f221c102840 2024-04-26T20:29:35.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 Processor -- start 2024-04-26T20:29:35.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 -- start start 2024-04-26T20:29:35.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f2219600640 1 -- 172.21.15.131:0/3630437732 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f220c004000 con 0x7f221c102840 2024-04-26T20:29:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463bf050 0x5637463bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bfa40 0x5637463c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:35.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463c4bb0 0x5637463c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:35.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563746322c70 con 0x5637463bfa40 2024-04-26T20:29:35.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5637462db980 con 0x5637463c4bb0 2024-04-26T20:29:35.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.208+0000 7f22257a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56374631ec30 con 0x5637463bf050 2024-04-26T20:29:36.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f221be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bfa40 0x5637463c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:36.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f2220a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463bf050 0x5637463bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:36.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f2213e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463c4bb0 0x5637463c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:36.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f221be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bfa40 0x5637463c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40154/0 (socket says 172.21.15.131:40154) 2024-04-26T20:29:36.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f221be00640 1 -- 172.21.15.131:0/513392636 learned_addr learned my addr 172.21.15.131:0/513392636 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:36.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f2220a00640 1 -- 172.21.15.131:0/513392636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463c4bb0 msgr2=0x5637463c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:36.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f2220a00640 1 --2- 172.21.15.131:0/513392636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463c4bb0 0x5637463c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f2220a00640 1 -- 172.21.15.131:0/513392636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bfa40 msgr2=0x5637463c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:36.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f2220a00640 1 --2- 172.21.15.131:0/513392636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bfa40 0x5637463c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.209+0000 7f2220a00640 1 -- 172.21.15.131:0/513392636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637462b37b0 con 0x5637463bf050 2024-04-26T20:29:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.210+0000 7f2220a00640 1 --2- 172.21.15.131:0/513392636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463bf050 0x5637463bf470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f22140117e0 tx=0x7f2214011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215a6e906ab84b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.210+0000 7f2211200640 1 -- 172.21.15.131:0/513392636 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214011310 con 0x5637463bf050 2024-04-26T20:29:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.210+0000 7f2211200640 1 -- 172.21.15.131:0/513392636 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221400f9a0 con 0x5637463bf050 2024-04-26T20:29:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.210+0000 7f2211200640 1 -- 172.21.15.131:0/513392636 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214005700 con 0x5637463bf050 2024-04-26T20:29:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.210+0000 7f2219600640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22000a3cc0 0x7f22000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.210+0000 7f2219600640 1 -- 172.21.15.131:0/3630437732 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f220c0d0db0 con 0x7f221c102840 2024-04-26T20:29:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.210+0000 7f221be00640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22000a3cc0 0x7f22000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 -- 172.21.15.131:0/513392636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463bf050 msgr2=0x5637463bf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 --2- 172.21.15.131:0/513392636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463bf050 0x5637463bf470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f22140117e0 tx=0x7f2214011ca0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 -- 172.21.15.131:0/513392636 shutdown_connections 2024-04-26T20:29:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 --2- 172.21.15.131:0/513392636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463c4bb0 0x5637463c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 --2- 172.21.15.131:0/513392636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bfa40 0x5637463c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 --2- 172.21.15.131:0/513392636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463bf050 0x5637463bf470 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 -- 172.21.15.131:0/513392636 >> 172.21.15.131:0/513392636 conn(0x5637463ba6c0 msgr2=0x5637463baac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 -- 172.21.15.131:0/513392636 shutdown_connections 2024-04-26T20:29:36.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 -- 172.21.15.131:0/513392636 wait complete. 2024-04-26T20:29:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f221be00640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22000a3cc0 0x7f22000a6180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f220c00c6a0 tx=0x7f220c0d4860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:36.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.211+0000 7f22257a1740 1 -- start start 2024-04-26T20:29:36.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 1 Processor -- start 2024-04-26T20:29:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:36.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:36.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:36.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563746321f80 con 0x5637463bf050 2024-04-26T20:29:36.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5637462cf260 con 0x5637463bfa40 2024-04-26T20:29:36.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5637463272a0 con 0x5637463c4bb0 2024-04-26T20:29:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f22257a1740 10 client.0 _ll_register_callbacks cb 0x563746319960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:36.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40160/0 (socket says 172.21.15.131:40160) 2024-04-26T20:29:36.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f2220a00640 1 -- 172.21.15.131:0/3395781953 learned_addr learned my addr 172.21.15.131:0/3395781953 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:36.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.212+0000 7f2213e00640 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:36.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2220a00640 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:36.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2220a00640 1 -- 172.21.15.131:0/3395781953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 msgr2=0x5637463d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:36.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2220a00640 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2220a00640 1 -- 172.21.15.131:0/3395781953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 msgr2=0x5637463c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:36.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2220a00640 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:36.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2220a00640 1 -- 172.21.15.131:0/3395781953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637462b11c0 con 0x5637463bf050 2024-04-26T20:29:36.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2220a00640 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c8dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2214009a70 tx=0x7f221400fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b816cb48e0b1db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:36.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2209600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:36.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.213+0000 7f2209600640 1 -- 172.21.15.131:0/3395781953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221402e060 con 0x5637463bf050 2024-04-26T20:29:36.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221400f9a0 con 0x5637463bf050 2024-04-26T20:29:36.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f2209600640 1 -- 172.21.15.131:0/3395781953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2214005440 con 0x5637463bf050 2024-04-26T20:29:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f2209600640 1 -- 172.21.15.131:0/3395781953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2214035050 con 0x5637463bf050 2024-04-26T20:29:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 10 client.16878 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:36.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637460343f0 con 0x5637463bf050 2024-04-26T20:29:36.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f2209600640 1 -- 172.21.15.131:0/3395781953 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221400f9a0 con 0x5637463bf050 2024-04-26T20:29:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f2209600640 1 -- 172.21.15.131:0/3395781953 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2214018070 con 0x5637463bf050 2024-04-26T20:29:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 10 client.16878 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:37.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 10 client.16878 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-26T20:29:37.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 20 client.16878 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:37.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5637463d4640 0x5637463d6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:37.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 4 client.16878 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:37.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.214+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563746330880 con 0x5637463d4640 2024-04-26T20:29:37.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.215+0000 7f2213e00640 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5637463d4640 0x5637463d6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:37.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.215+0000 7f2213e00640 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5637463d4640 0x5637463d6a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41dda506f5855189 server_cookie=8ba4dbe6ba30dee9 in_seq=0 out_seq=0 2024-04-26T20:29:37.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.215+0000 7f2209600640 10 client.16878 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:29:37.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f2209600640 1 -- 172.21.15.131:0/3395781953 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563746330880 con 0x5637463d4640 2024-04-26T20:29:37.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f2209600640 10 client.16878 handle_command_reply: tid=0 2024-04-26T20:29:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 client.16878 shutdown 2024-04-26T20:29:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 msgr2=0x5637463c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:37.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c8dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2214009a70 tx=0x7f221400fb50 comp rx=0 tx=0).stop 2024-04-26T20:29:37.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 shutdown_connections 2024-04-26T20:29:37.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x5637463d4640 0x5637463d6a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 --2- 172.21.15.131:0/3395781953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c8dc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 >> 172.21.15.131:0/3395781953 conn(0x5637463ba6c0 msgr2=0x5637463b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:37.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.216+0000 7f2209600640 0 client.16878 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:29:37.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.217+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 shutdown_connections 2024-04-26T20:29:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.217+0000 7f22257a1740 1 -- 172.21.15.131:0/3395781953 wait complete. 2024-04-26T20:29:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.217+0000 7f22257a1740 20 client.16878 trim_cache size 0 max 16384 2024-04-26T20:29:37.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.231+0000 7f22257a1740 1 Processor -- start 2024-04-26T20:29:37.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.231+0000 7f22257a1740 1 -- start start 2024-04-26T20:29:37.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f22257a1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:37.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f22257a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:37.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f22257a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:37.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f22257a1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1aff80 con 0x5637463bf050 2024-04-26T20:29:37.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f22257a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1ad850 con 0x5637463bfa40 2024-04-26T20:29:37.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f22257a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1ab6a0 con 0x5637463c4bb0 2024-04-26T20:29:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:37.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40176/0 (socket says 172.21.15.131:40176) 2024-04-26T20:29:37.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 -- 172.21.15.131:0/1408323229 learned_addr learned my addr 172.21.15.131:0/1408323229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:37.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2213e00640 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:37.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 -- 172.21.15.131:0/1408323229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 msgr2=0x5637463b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463b9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 -- 172.21.15.131:0/1408323229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 msgr2=0x5637463d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:37.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2220a00640 1 -- 172.21.15.131:0/1408323229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637460fe7d0 con 0x5637463bf050 2024-04-26T20:29:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.232+0000 7f2213e00640 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:37.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.233+0000 7f2220a00640 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c9260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2214029c20 tx=0x7f22140021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bed8080fba5b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.233+0000 7f220be00640 1 -- 172.21.15.131:0/1408323229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221402e070 con 0x5637463bf050 2024-04-26T20:29:37.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.233+0000 7f220be00640 1 -- 172.21.15.131:0/1408323229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221401f040 con 0x5637463bf050 2024-04-26T20:29:37.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.233+0000 7f220be00640 1 -- 172.21.15.131:0/1408323229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214010ae0 con 0x5637463bf050 2024-04-26T20:29:37.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 -- 172.21.15.131:0/1408323229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 msgr2=0x5637463c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c9260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2214029c20 tx=0x7f22140021d0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 -- 172.21.15.131:0/1408323229 shutdown_connections 2024-04-26T20:29:37.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 --2- 172.21.15.131:0/1408323229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463c9260 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:37.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 -- 172.21.15.131:0/1408323229 >> 172.21.15.131:0/1408323229 conn(0x5637463ba6c0 msgr2=0x5637463b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 -- 172.21.15.131:0/1408323229 shutdown_connections 2024-04-26T20:29:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 -- 172.21.15.131:0/1408323229 wait complete. 2024-04-26T20:29:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 -- start start 2024-04-26T20:29:38.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.234+0000 7f22257a1740 1 Processor -- start 2024-04-26T20:29:38.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f22257a1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:38.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f22257a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:38.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f22257a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:38.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f22257a1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1abd10 con 0x5637463bf050 2024-04-26T20:29:38.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f22257a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1ad180 con 0x5637463bfa40 2024-04-26T20:29:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f22257a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1ac3e0 con 0x5637463c4bb0 2024-04-26T20:29:38.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f22257a1740 10 client.0 _ll_register_callbacks cb 0x7f22140116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:38.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:38.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f2220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40190/0 (socket says 172.21.15.131:40190) 2024-04-26T20:29:38.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f2220a00640 1 -- 172.21.15.131:0/3489274502 learned_addr learned my addr 172.21.15.131:0/3489274502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:38.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f2213e00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:38.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.235+0000 7f2220a00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:38.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2220a00640 1 -- 172.21.15.131:0/3489274502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 msgr2=0x5637463cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2220a00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2220a00640 1 -- 172.21.15.131:0/3489274502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 msgr2=0x5637463ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:38.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2220a00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2220a00640 1 -- 172.21.15.131:0/3489274502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637462c92a0 con 0x5637463bf050 2024-04-26T20:29:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2213e00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:38.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2220a00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463d0310 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2214010540 tx=0x7f221400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e3a8e5713e0716 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:38.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2209600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:38.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2209600640 1 -- 172.21.15.131:0/3489274502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221402e070 con 0x5637463bf050 2024-04-26T20:29:38.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221401f040 con 0x5637463bf050 2024-04-26T20:29:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2209600640 1 -- 172.21.15.131:0/3489274502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f2214029440 con 0x5637463bf050 2024-04-26T20:29:38.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f2209600640 1 -- 172.21.15.131:0/3489274502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2214029640 con 0x5637463bf050 2024-04-26T20:29:38.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f22257a1740 10 client.16884 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:38.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.236+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563746442500 con 0x5637463bf050 2024-04-26T20:29:38.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f2209600640 1 -- 172.21.15.131:0/3489274502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221401f040 con 0x5637463bf050 2024-04-26T20:29:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f2209600640 1 -- 172.21.15.131:0/3489274502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2214018040 con 0x5637463bf050 2024-04-26T20:29:38.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f22257a1740 10 client.16884 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:38.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f22257a1740 10 client.16884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-26T20:29:38.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f22257a1740 20 client.16884 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:38.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f22257a1740 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x563746442ec0 0x5637464452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:38.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f22257a1740 4 client.16884 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:29:38.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 --> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56374632a6f0 con 0x563746442ec0 2024-04-26T20:29:38.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.237+0000 7f2213e00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x563746442ec0 0x5637464452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:38.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f2213e00640 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x563746442ec0 0x5637464452e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0c13b29792418fe server_cookie=4049a88cc68be6a7 in_seq=0 out_seq=0 2024-04-26T20:29:38.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f2209600640 10 client.16884 ms_handle_connect on v2:172.21.15.131:6840/1288851219 2024-04-26T20:29:38.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f2209600640 1 -- 172.21.15.131:0/3489274502 <== mds.1 v2:172.21.15.131:6840/1288851219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56374632a6f0 con 0x563746442ec0 2024-04-26T20:29:38.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f2209600640 10 client.16884 handle_command_reply: tid=0 2024-04-26T20:29:38.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 client.16884 shutdown 2024-04-26T20:29:38.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 msgr2=0x5637463d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463d0310 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2214010540 tx=0x7f221400ff90 comp rx=0 tx=0).stop 2024-04-26T20:29:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 shutdown_connections 2024-04-26T20:29:39.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:6840/1288851219,v1:172.21.15.131:6841/1288851219] conn(0x563746442ec0 0x5637464452e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637463c4bb0 0x5637463cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637463bfa40 0x5637463ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f2209600640 0 client.16884 ms_handle_reset on v2:172.21.15.131:6840/1288851219 2024-04-26T20:29:39.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 --2- 172.21.15.131:0/3489274502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637463bf050 0x5637463d0310 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 >> 172.21.15.131:0/3489274502 conn(0x5637463ba6c0 msgr2=0x5637463b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:39.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 shutdown_connections 2024-04-26T20:29:39.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.238+0000 7f22257a1740 1 -- 172.21.15.131:0/3489274502 wait complete. 2024-04-26T20:29:39.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f22257a1740 20 client.16884 trim_cache size 0 max 16384 2024-04-26T20:29:39.206 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:29:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22000a3cc0 msgr2=0x7f22000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:39.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22000a3cc0 0x7f22000a6180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f220c00c6a0 tx=0x7f220c0d4860 comp rx=0 tx=0).stop 2024-04-26T20:29:39.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c102840 msgr2=0x7f221c19acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:39.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c102840 0x7f221c19acf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f220c00e9c0 tx=0x7f220c00ee80 comp rx=0 tx=0).stop 2024-04-26T20:29:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 shutdown_connections 2024-04-26T20:29:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f221c1043f0 0x7f221c1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c103a30 0x7f221c19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f22000a3cc0 0x7f22000a6180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 --2- 172.21.15.131:0/3630437732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c102840 0x7f221c19acf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 >> 172.21.15.131:0/3630437732 conn(0x7f221c0fddd0 msgr2=0x7f221c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:39.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 shutdown_connections 2024-04-26T20:29:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:35.239+0000 7f2221400640 1 -- 172.21.15.131:0/3630437732 wait complete. 2024-04-26T20:29:39.448 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T20:29:39.459 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:35 smithi131 ceph-mon[27919]: pgmap v997: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:29:39.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:37 smithi131 ceph-mon[27919]: pgmap v998: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-26T20:29:39.501 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:35 smithi157 ceph-mon[32703]: pgmap v997: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:29:39.510 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:37 smithi157 ceph-mon[32703]: pgmap v998: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-26T20:29:39.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:35 smithi148 ceph-mon[32281]: pgmap v997: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T20:29:39.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:37 smithi148 ceph-mon[32281]: pgmap v998: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-26T20:29:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 -- 172.21.15.131:0/3945507402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778105c60 msgr2=0x7f77781060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:39.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 --2- 172.21.15.131:0/3945507402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778105c60 0x7f77781060e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7764009900 tx=0x7f776402a3f0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 -- 172.21.15.131:0/3945507402 shutdown_connections 2024-04-26T20:29:39.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 --2- 172.21.15.131:0/3945507402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778106620 0x7f777810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 --2- 172.21.15.131:0/3945507402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778105c60 0x7f77781060e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 --2- 172.21.15.131:0/3945507402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778104a70 0x7f7778104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:39.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 -- 172.21.15.131:0/3945507402 >> 172.21.15.131:0/3945507402 conn(0x7f77780fffe0 msgr2=0x7f7778102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:39.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 -- 172.21.15.131:0/3945507402 shutdown_connections 2024-04-26T20:29:39.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.750+0000 7f7777e00640 1 -- 172.21.15.131:0/3945507402 wait complete. 2024-04-26T20:29:39.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 Processor -- start 2024-04-26T20:29:39.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 -- start start 2024-04-26T20:29:39.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778104a70 0x7f777819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:39.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778105c60 0x7f777819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778106620 0x7f77781a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:40.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110580 con 0x7f7778106620 2024-04-26T20:29:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110400 con 0x7f7778105c60 2024-04-26T20:29:40.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.751+0000 7f7777e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110700 con 0x7f7778104a70 2024-04-26T20:29:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778104a70 0x7f777819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:40.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778104a70 0x7f777819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46488/0 (socket says 172.21.15.131:46488) 2024-04-26T20:29:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778105c60 0x7f777819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:40.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 -- 172.21.15.131:0/1334474967 learned_addr learned my addr 172.21.15.131:0/1334474967 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7777400640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778106620 0x7f77781a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:40.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 -- 172.21.15.131:0/1334474967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778105c60 msgr2=0x7f777819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778105c60 0x7f777819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:40.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 -- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778106620 msgr2=0x7f77781a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778106620 0x7f77781a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776a00640 1 -- 172.21.15.131:0/1334474967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77781a28c0 con 0x7f7778104a70 2024-04-26T20:29:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.752+0000 7f7776000640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778105c60 0x7f777819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:40.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.753+0000 7f7776a00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778104a70 0x7f777819ac80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9c0 tx=0x7f776800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.753+0000 7f776f400640 1 -- 172.21.15.131:0/1334474967 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776800cde0 con 0x7f7778104a70 2024-04-26T20:29:40.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.753+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77781a2bb0 con 0x7f7778104a70 2024-04-26T20:29:40.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.753+0000 7f776f400640 1 -- 172.21.15.131:0/1334474967 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77680108e0 con 0x7f7778104a70 2024-04-26T20:29:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.753+0000 7f776f400640 1 -- 172.21.15.131:0/1334474967 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680188d0 con 0x7f7778104a70 2024-04-26T20:29:40.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.753+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a30d0 con 0x7f7778104a70 2024-04-26T20:29:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 Processor -- start 2024-04-26T20:29:40.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 -- start start 2024-04-26T20:29:40.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f776f400640 1 -- 172.21.15.131:0/1334474967 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f776801f020 con 0x7f7778104a70 2024-04-26T20:29:40.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:40.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0a40 0x55ab52fe5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe5bb0 0x55ab52fe8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:40.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ab52f43c70 con 0x55ab52fe0050 2024-04-26T20:29:40.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ab52efc980 con 0x55ab52fe5bb0 2024-04-26T20:29:40.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.756+0000 7f7780789740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ab52f3fc30 con 0x55ab52fe0a40 2024-04-26T20:29:40.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:40.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40220/0 (socket says 172.21.15.131:40220) 2024-04-26T20:29:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 -- 172.21.15.131:0/1678084835 learned_addr learned my addr 172.21.15.131:0/1678084835 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:40.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7776000640 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe5bb0 0x55ab52fe8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:40.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7776a00640 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0a40 0x55ab52fe5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:40.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 -- 172.21.15.131:0/1678084835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0a40 msgr2=0x55ab52fe5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:40.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0a40 0x55ab52fe5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 -- 172.21.15.131:0/1678084835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe5bb0 msgr2=0x55ab52fe8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:40.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe5bb0 0x55ab52fe8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.757+0000 7f7777400640 1 -- 172.21.15.131:0/1678084835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab52ed47b0 con 0x55ab52fe0050 2024-04-26T20:29:40.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f7777400640 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe0470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7770009fd0 tx=0x7f77700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c53ec2c821f65d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:41.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f7763e00640 1 -- 172.21.15.131:0/1678084835 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770004d20 con 0x55ab52fe0050 2024-04-26T20:29:41.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f7763e00640 1 -- 172.21.15.131:0/1678084835 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7770004050 con 0x55ab52fe0050 2024-04-26T20:29:41.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f7763e00640 1 -- 172.21.15.131:0/1678084835 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770005280 con 0x55ab52fe0050 2024-04-26T20:29:41.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f776f400640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f775c0a3c40 0x7f775c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:41.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f776f400640 1 -- 172.21.15.131:0/1334474967 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f77680d0b50 con 0x7f7778104a70 2024-04-26T20:29:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f7780789740 1 -- 172.21.15.131:0/1678084835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 msgr2=0x55ab52fe0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f7780789740 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe0470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7770009fd0 tx=0x7f77700118d0 comp rx=0 tx=0).stop 2024-04-26T20:29:41.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.758+0000 7f7776a00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f775c0a3c40 0x7f775c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:41.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 -- 172.21.15.131:0/1678084835 shutdown_connections 2024-04-26T20:29:41.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe5bb0 0x55ab52fe8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:41.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0a40 0x55ab52fe5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:41.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 --2- 172.21.15.131:0/1678084835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe0470 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 -- 172.21.15.131:0/1678084835 >> 172.21.15.131:0/1678084835 conn(0x55ab52fdb6c0 msgr2=0x55ab52fdbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:41.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 -- 172.21.15.131:0/1678084835 shutdown_connections 2024-04-26T20:29:41.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 -- 172.21.15.131:0/1678084835 wait complete. 2024-04-26T20:29:41.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7776a00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f775c0a3c40 0x7f775c0a6100 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55ab52fe0e40 tx=0x7f77680d4600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:41.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 -- start start 2024-04-26T20:29:41.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.759+0000 7f7780789740 1 Processor -- start 2024-04-26T20:29:41.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7780789740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:41.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7780789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:41.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7780789740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab52ff1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:41.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7780789740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ab52f42f80 con 0x55ab52fe0050 2024-04-26T20:29:41.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7780789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ab52ef0260 con 0x55ab52fe0a40 2024-04-26T20:29:41.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7780789740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ab52f482a0 con 0x55ab52fe5bb0 2024-04-26T20:29:41.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7780789740 10 client.0 _ll_register_callbacks cb 0x55ab52f3a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:41.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7777400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7777400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40234/0 (socket says 172.21.15.131:40234) 2024-04-26T20:29:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7777400640 1 -- 172.21.15.131:0/1332881552 learned_addr learned my addr 172.21.15.131:0/1332881552 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:41.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.760+0000 7f7776000640 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:41.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7777400640 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab52ff1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:41.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7777400640 1 -- 172.21.15.131:0/1332881552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 msgr2=0x55ab52ff1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7777400640 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab52ff1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:41.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7777400640 1 -- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 msgr2=0x55ab52fea4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:41.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7777400640 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fea4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:41.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7777400640 1 -- 172.21.15.131:0/1332881552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab52ed21c0 con 0x55ab52fe0050 2024-04-26T20:29:41.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7777400640 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe9f80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7770009fa0 tx=0x7f7770004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b44f3ec8acb18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:41.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7760c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7760c00640 1 -- 172.21.15.131:0/1332881552 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770004e50 con 0x55ab52fe0050 2024-04-26T20:29:41.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7760c00640 1 -- 172.21.15.131:0/1332881552 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f777000f4e0 con 0x55ab52fe0050 2024-04-26T20:29:41.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.761+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7770004050 con 0x55ab52fe0050 2024-04-26T20:29:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7760c00640 1 -- 172.21.15.131:0/1332881552 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7770035050 con 0x55ab52fe0050 2024-04-26T20:29:41.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 10 client.16896 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:41.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab52c553f0 con 0x55ab52fe0050 2024-04-26T20:29:42.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7760c00640 1 -- 172.21.15.131:0/1332881552 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7770004050 con 0x55ab52fe0050 2024-04-26T20:29:42.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7760c00640 1 -- 172.21.15.131:0/1332881552 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f77700052e0 con 0x55ab52fe0050 2024-04-26T20:29:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 10 client.16896 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:42.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 20 client.16896 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:42.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab52ff5810 0x55ab52ff7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 4 client.16896 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:29:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.762+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab52f51880 con 0x55ab52ff5810 2024-04-26T20:29:42.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.763+0000 7f7776000640 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab52ff5810 0x55ab52ff7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:42.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.763+0000 7f7760c00640 10 client.16896 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:42.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.763+0000 7f7776000640 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab52ff5810 0x55ab52ff7c00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbbd1494a642f68 server_cookie=aa4b347cb59b9d5b in_seq=0 out_seq=0 2024-04-26T20:29:42.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.764+0000 7f7760c00640 1 -- 172.21.15.131:0/1332881552 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab52f51880 con 0x55ab52ff5810 2024-04-26T20:29:42.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.764+0000 7f7760c00640 10 client.16896 handle_command_reply: tid=0 2024-04-26T20:29:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.764+0000 7f7780789740 1 client.16896 shutdown 2024-04-26T20:29:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 msgr2=0x55ab52fe9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:42.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe9f80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7770009fa0 tx=0x7f7770004810 comp rx=0 tx=0).stop 2024-04-26T20:29:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 shutdown_connections 2024-04-26T20:29:42.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab52ff5810 0x55ab52ff7c00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab52ff1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:42.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 --2- 172.21.15.131:0/1332881552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fe9f80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:42.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 >> 172.21.15.131:0/1332881552 conn(0x55ab52fdb6c0 msgr2=0x55ab52fda7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:42.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7760c00640 0 client.16896 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:42.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 shutdown_connections 2024-04-26T20:29:42.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 1 -- 172.21.15.131:0/1332881552 wait complete. 2024-04-26T20:29:42.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.765+0000 7f7780789740 20 client.16896 trim_cache size 0 max 16384 2024-04-26T20:29:42.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.781+0000 7f7780789740 1 Processor -- start 2024-04-26T20:29:42.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.781+0000 7f7780789740 1 -- start start 2024-04-26T20:29:42.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7780789740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0050 0x55ab52fd9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7780789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52ff1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7780789740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe5bb0 0x55ab52fda460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7780789740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77781affc0 con 0x55ab52fe5bb0 2024-04-26T20:29:42.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7780789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ad890 con 0x55ab52fe0a40 2024-04-26T20:29:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7780789740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ab6e0 con 0x55ab52fe0050 2024-04-26T20:29:42.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7777400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0050 0x55ab52fd9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:42.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7776000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52ff1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:42.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7776000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52ff1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:42890/0 (socket says 172.21.15.131:42890) 2024-04-26T20:29:42.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7776000640 1 -- 172.21.15.131:0/76375893 learned_addr learned my addr 172.21.15.131:0/76375893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:42.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7777400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0050 0x55ab52fd9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46522/0 (socket says 172.21.15.131:46522) 2024-04-26T20:29:42.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.782+0000 7f7777400640 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe5bb0 0x55ab52fda460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:42.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.783+0000 7f7776000640 1 -- 172.21.15.131:0/76375893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0050 msgr2=0x55ab52fd9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:42.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.783+0000 7f7776000640 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0050 0x55ab52fd9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:42.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.783+0000 7f7776000640 1 -- 172.21.15.131:0/76375893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe5bb0 msgr2=0x55ab52fda460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:42.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.783+0000 7f7776000640 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe5bb0 0x55ab52fda460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.783+0000 7f7776000640 1 -- 172.21.15.131:0/76375893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab52d1f7d0 con 0x55ab52fe0a40 2024-04-26T20:29:42.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.783+0000 7f7777400640 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe5bb0 0x55ab52fda460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:43.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.783+0000 7f7776000640 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52ff1450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7764032fd0 tx=0x7f77640314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc880f559f303f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:43.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7763400640 1 -- 172.21.15.131:0/76375893 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764035040 con 0x55ab52fe0a40 2024-04-26T20:29:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7763400640 1 -- 172.21.15.131:0/76375893 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776404b070 con 0x55ab52fe0a40 2024-04-26T20:29:43.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7763400640 1 -- 172.21.15.131:0/76375893 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77640462c0 con 0x55ab52fe0a40 2024-04-26T20:29:43.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7780789740 1 -- 172.21.15.131:0/76375893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 msgr2=0x55ab52ff1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:43.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7780789740 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52ff1450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7764032fd0 tx=0x7f77640314a0 comp rx=0 tx=0).stop 2024-04-26T20:29:43.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7780789740 1 -- 172.21.15.131:0/76375893 shutdown_connections 2024-04-26T20:29:43.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7780789740 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe5bb0 0x55ab52fda460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:43.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7780789740 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52ff1450 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:43.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7780789740 1 --2- 172.21.15.131:0/76375893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe0050 0x55ab52fd9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.784+0000 7f7780789740 1 -- 172.21.15.131:0/76375893 >> 172.21.15.131:0/76375893 conn(0x55ab52fdb6c0 msgr2=0x55ab52fe1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:43.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.785+0000 7f7780789740 1 -- 172.21.15.131:0/76375893 shutdown_connections 2024-04-26T20:29:43.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.785+0000 7f7780789740 1 -- 172.21.15.131:0/76375893 wait complete. 2024-04-26T20:29:43.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.785+0000 7f7780789740 1 -- start start 2024-04-26T20:29:43.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.785+0000 7f7780789740 1 Processor -- start 2024-04-26T20:29:43.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7780789740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fedf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:43.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7780789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7780789740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab53059d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:43.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7780789740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77781abd50 con 0x55ab52fe0050 2024-04-26T20:29:43.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7780789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ad1c0 con 0x55ab52fe0a40 2024-04-26T20:29:43.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7780789740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ac420 con 0x55ab52fe5bb0 2024-04-26T20:29:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7780789740 10 client.0 _ll_register_callbacks cb 0x7f77700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:29:43.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fedf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:43.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fedf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40250/0 (socket says 172.21.15.131:40250) 2024-04-26T20:29:43.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 -- 172.21.15.131:0/2275808922 learned_addr learned my addr 172.21.15.131:0/2275808922 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7776000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:43.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab53059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:43.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 -- 172.21.15.131:0/2275808922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 msgr2=0x55ab53059d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:43.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab53059d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:43.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 -- 172.21.15.131:0/2275808922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 msgr2=0x55ab52fee460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:43.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fee460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:43.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.786+0000 7f7777400640 1 -- 172.21.15.131:0/2275808922 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab52eea2a0 con 0x55ab52fe0050 2024-04-26T20:29:43.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.787+0000 7f7777400640 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fedf20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f77700058f0 tx=0x7f7770003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cf39d42e81f01e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.787+0000 7f7760c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-26T20:29:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.787+0000 7f7760c00640 1 -- 172.21.15.131:0/2275808922 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770004c50 con 0x55ab52fe0050 2024-04-26T20:29:43.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.787+0000 7f7760c00640 1 -- 172.21.15.131:0/2275808922 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f7770004df0 con 0x55ab52fe0050 2024-04-26T20:29:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.787+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776404b070 con 0x55ab52fe0050 2024-04-26T20:29:43.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.787+0000 7f7760c00640 1 -- 172.21.15.131:0/2275808922 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7770048050 con 0x55ab52fe0050 2024-04-26T20:29:43.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 10 client.16902 fetch_fsmap learned FSMap version 27 2024-04-26T20:29:43.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab5305af20 con 0x55ab52fe0050 2024-04-26T20:29:43.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7760c00640 1 -- 172.21.15.131:0/2275808922 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776404b070 con 0x55ab52fe0050 2024-04-26T20:29:43.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7760c00640 1 -- 172.21.15.131:0/2275808922 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7770019540 con 0x55ab52fe0050 2024-04-26T20:29:43.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 10 client.16902 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T20:29:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T20:29:43.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 20 client.16902 populate_metadata read hostname 'smithi131' 2024-04-26T20:29:43.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab5305b810 0x55ab5305dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 4 client.16902 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T20:29:44.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.788+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 --> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab52f4b6f0 con 0x55ab5305b810 2024-04-26T20:29:44.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.789+0000 7f7776000640 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab5305b810 0x55ab5305dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:44.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.789+0000 7f7776000640 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab5305b810 0x55ab5305dc30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10a18fcd7b772ec server_cookie=998e1bc39b4bbbb6 in_seq=0 out_seq=0 2024-04-26T20:29:44.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.789+0000 7f7760c00640 10 client.16902 ms_handle_connect on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:44.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7760c00640 1 -- 172.21.15.131:0/2275808922 <== mds.0 v2:172.21.15.148:6838/4047119723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab52f4b6f0 con 0x55ab5305b810 2024-04-26T20:29:44.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7760c00640 10 client.16902 handle_command_reply: tid=0 2024-04-26T20:29:44.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 client.16902 shutdown 2024-04-26T20:29:44.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 msgr2=0x55ab52fedf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:44.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fedf20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f77700058f0 tx=0x7f7770003e00 comp rx=0 tx=0).stop 2024-04-26T20:29:44.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 shutdown_connections 2024-04-26T20:29:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.148:6838/4047119723,v1:172.21.15.148:6839/4047119723] conn(0x55ab5305b810 0x55ab5305dc30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab52fe5bb0 0x55ab53059d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab52fe0a40 0x55ab52fee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 --2- 172.21.15.131:0/2275808922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ab52fe0050 0x55ab52fedf20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7760c00640 0 client.16902 ms_handle_reset on v2:172.21.15.148:6838/4047119723 2024-04-26T20:29:44.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 >> 172.21.15.131:0/2275808922 conn(0x55ab52fdb6c0 msgr2=0x55ab52fde3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 shutdown_connections 2024-04-26T20:29:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 1 -- 172.21.15.131:0/2275808922 wait complete. 2024-04-26T20:29:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.790+0000 7f7780789740 20 client.16902 trim_cache size 0 max 16384 2024-04-26T20:29:44.507 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-04-26T20:29:44.516 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:39 smithi157 ceph-mon[32703]: pgmap v999: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-26T20:29:44.593 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:41 smithi157 ceph-mon[32703]: pgmap v1000: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-26T20:29:44.648 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:43 smithi157 ceph-mon[32703]: pgmap v1001: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T20:29:44.655 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:39 smithi148 ceph-mon[32281]: pgmap v999: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-26T20:29:44.662 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:41 smithi148 ceph-mon[32281]: pgmap v1000: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-26T20:29:44.670 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:43 smithi148 ceph-mon[32281]: pgmap v1001: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T20:29:44.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f775c0a3c40 msgr2=0x7f775c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:44.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f775c0a3c40 0x7f775c0a6100 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55ab52fe0e40 tx=0x7f77680d4600 comp rx=0 tx=0).stop 2024-04-26T20:29:44.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778104a70 msgr2=0x7f777819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:44.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778104a70 0x7f777819ac80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9c0 tx=0x7f776800ee80 comp rx=0 tx=0).stop 2024-04-26T20:29:44.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 shutdown_connections 2024-04-26T20:29:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7778106620 0x7f77781a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778105c60 0x7f777819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f775c0a3c40 0x7f775c0a6100 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 --2- 172.21.15.131:0/1334474967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7778104a70 0x7f777819ac80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 >> 172.21.15.131:0/1334474967 conn(0x7f77780fffe0 msgr2=0x7f7778109030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 shutdown_connections 2024-04-26T20:29:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:39.791+0000 7f7777e00640 1 -- 172.21.15.131:0/1334474967 wait complete. 2024-04-26T20:29:44.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:39 smithi131 ceph-mon[27919]: pgmap v999: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-26T20:29:45.043 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:41 smithi131 ceph-mon[27919]: pgmap v1000: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-26T20:29:45.052 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:43 smithi131 ceph-mon[27919]: pgmap v1001: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T20:29:45.986 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-26T20:29:45.993 INFO:tasks.fwd_scrub:done joining 2024-04-26T20:29:46.002 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T20:29:46.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:29:46.081 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:45 smithi148 ceph-mon[32281]: pgmap v1002: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:29:46.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:45 smithi131 ceph-mon[27919]: pgmap v1002: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:29:46.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:45 smithi157 ceph-mon[32703]: pgmap v1002: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T20:29:46.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 -- 172.21.15.131:0/3019245458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c106720 msgr2=0x7fbf3c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 --2- 172.21.15.131:0/3019245458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c106720 0x7fbf3c10cfb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009ca0 tx=0x7fbf3402a8c0 comp rx=0 tx=0).stop 2024-04-26T20:29:46.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 -- 172.21.15.131:0/3019245458 shutdown_connections 2024-04-26T20:29:46.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 --2- 172.21.15.131:0/3019245458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c106720 0x7fbf3c10cfb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 --2- 172.21.15.131:0/3019245458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf3c105d60 0x7fbf3c1061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 --2- 172.21.15.131:0/3019245458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3c104b70 0x7fbf3c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:46.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 -- 172.21.15.131:0/3019245458 >> 172.21.15.131:0/3019245458 conn(0x7fbf3c100120 msgr2=0x7fbf3c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:46.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 -- 172.21.15.131:0/3019245458 shutdown_connections 2024-04-26T20:29:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.295+0000 7fbf44600640 1 -- 172.21.15.131:0/3019245458 wait complete. 2024-04-26T20:29:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 Processor -- start 2024-04-26T20:29:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 -- start start 2024-04-26T20:29:46.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c104b70 0x7fbf3c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:46.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3c105d60 0x7fbf3c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:46.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf3c106720 0x7fbf3c1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:46.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1106d0 con 0x7fbf3c105d60 2024-04-26T20:29:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c110550 con 0x7fbf3c104b70 2024-04-26T20:29:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.296+0000 7fbf44600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c110850 con 0x7fbf3c106720 2024-04-26T20:29:46.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c104b70 0x7fbf3c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf42600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf3c106720 0x7fbf3c1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c104b70 0x7fbf3c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:48340/0 (socket says 172.21.15.131:48340) 2024-04-26T20:29:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3c105d60 0x7fbf3c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:46.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 -- 172.21.15.131:0/3316463643 learned_addr learned my addr 172.21.15.131:0/3316463643 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:46.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 -- 172.21.15.131:0/3316463643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf3c106720 msgr2=0x7fbf3c1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:46.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf3c106720 0x7fbf3c1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:46.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 -- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3c105d60 msgr2=0x7fbf3c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:46.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3c105d60 0x7fbf3c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:46.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41c00640 1 -- 172.21.15.131:0/3316463643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c1a2940 con 0x7fbf3c104b70 2024-04-26T20:29:46.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf42600640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf3c106720 0x7fbf3c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:46.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.297+0000 7fbf41200640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3c105d60 0x7fbf3c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:46.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.298+0000 7fbf41c00640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c104b70 0x7fbf3c19ad30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9c0 tx=0x7fbf3000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:46.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.298+0000 7fbf3a000640 1 -- 172.21.15.131:0/3316463643 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000cd80 con 0x7fbf3c104b70 2024-04-26T20:29:47.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.298+0000 7fbf3a000640 1 -- 172.21.15.131:0/3316463643 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf300035f0 con 0x7fbf3c104b70 2024-04-26T20:29:47.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.298+0000 7fbf3a000640 1 -- 172.21.15.131:0/3316463643 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30010690 con 0x7fbf3c104b70 2024-04-26T20:29:47.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.298+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c1a2c30 con 0x7fbf3c104b70 2024-04-26T20:29:47.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.298+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1a3130 con 0x7fbf3c104b70 2024-04-26T20:29:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.299+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c10be90 con 0x7fbf3c104b70 2024-04-26T20:29:47.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.301+0000 7fbf3a000640 1 -- 172.21.15.131:0/3316463643 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbf300037a0 con 0x7fbf3c104b70 2024-04-26T20:29:47.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.303+0000 7fbf3a000640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf240a3c40 0x7fbf240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:47.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.303+0000 7fbf3a000640 1 -- 172.21.15.131:0/3316463643 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fbf300d0ef0 con 0x7fbf3c104b70 2024-04-26T20:29:47.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.303+0000 7fbf41200640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf240a3c40 0x7fbf240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:47.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.304+0000 7fbf41200640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf240a3c40 0x7fbf240a6100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009fc0 tx=0x7fbf28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.305+0000 7fbf3a000640 1 -- 172.21.15.131:0/3316463643 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbf3009d320 con 0x7fbf3c104b70 2024-04-26T20:29:47.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.510+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf3c02dcc0 con 0x7fbf3c104b70 2024-04-26T20:29:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.512+0000 7fbf3a000640 1 -- 172.21.15.131:0/3316463643 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7fbf3009cc20 con 0x7fbf3c104b70 2024-04-26T20:29:47.320 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:29:47.329 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:29:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf240a3c40 msgr2=0x7fbf240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:47.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf240a3c40 0x7fbf240a6100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009fc0 tx=0x7fbf28005f90 comp rx=0 tx=0).stop 2024-04-26T20:29:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c104b70 msgr2=0x7fbf3c19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:47.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c104b70 0x7fbf3c19ad30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9c0 tx=0x7fbf3000ee80 comp rx=0 tx=0).stop 2024-04-26T20:29:47.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 shutdown_connections 2024-04-26T20:29:47.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fbf240a3c40 0x7fbf240a6100 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf3c106720 0x7fbf3c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:47.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3c105d60 0x7fbf3c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:47.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 --2- 172.21.15.131:0/3316463643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c104b70 0x7fbf3c19ad30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:47.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 >> 172.21.15.131:0/3316463643 conn(0x7fbf3c100120 msgr2=0x7fbf3c109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:47.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 shutdown_connections 2024-04-26T20:29:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:46.515+0000 7fbf44600640 1 -- 172.21.15.131:0/3316463643 wait complete. 2024-04-26T20:29:47.593 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:29:47.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:29:47.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:46 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3316463643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:29:47.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:46 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3316463643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:29:47.695 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:46 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3316463643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:29:47.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 -- 172.21.15.131:0/1771003536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13810ef50 msgr2=0x7ff138111340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:47.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 --2- 172.21.15.131:0/1771003536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13810ef50 0x7ff138111340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff130009be0 tx=0x7ff13002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:29:47.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 -- 172.21.15.131:0/1771003536 shutdown_connections 2024-04-26T20:29:47.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 --2- 172.21.15.131:0/1771003536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13810ef50 0x7ff138111340 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:47.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 --2- 172.21.15.131:0/1771003536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff138075390 0x7ff138075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:47.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 --2- 172.21.15.131:0/1771003536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff138076970 0x7ff138074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:48.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 -- 172.21.15.131:0/1771003536 >> 172.21.15.131:0/1771003536 conn(0x7ff1380fdf10 msgr2=0x7ff138100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 -- 172.21.15.131:0/1771003536 shutdown_connections 2024-04-26T20:29:48.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.901+0000 7ff13da00640 1 -- 172.21.15.131:0/1771003536 wait complete. 2024-04-26T20:29:48.108 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:47 smithi148 ceph-mon[32281]: pgmap v1003: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T20:29:48.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 Processor -- start 2024-04-26T20:29:48.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 -- start start 2024-04-26T20:29:48.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff138075390 0x7ff13819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:48.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff138076970 0x7ff13819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810ef50 0x7ff1381a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114a80 con 0x7ff13810ef50 2024-04-26T20:29:48.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114900 con 0x7ff138075390 2024-04-26T20:29:48.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.902+0000 7ff13da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114c00 con 0x7ff138076970 2024-04-26T20:29:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff138076970 0x7ff13819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff138076970 0x7ff13819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46552/0 (socket says 172.21.15.131:46552) 2024-04-26T20:29:48.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 -- 172.21.15.131:0/4115893976 learned_addr learned my addr 172.21.15.131:0/4115893976 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:48.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff137400640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810ef50 0x7ff1381a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:48.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136a00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff138075390 0x7ff13819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:48.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 -- 172.21.15.131:0/4115893976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff138075390 msgr2=0x7ff13819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:48.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff138075390 0x7ff13819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 -- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810ef50 msgr2=0x7ff1381a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810ef50 0x7ff1381a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:48.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 -- 172.21.15.131:0/4115893976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1381a6d60 con 0x7ff138076970 2024-04-26T20:29:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.903+0000 7ff136000640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff138076970 0x7ff13819f680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e9f0 tx=0x7ff12400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:48.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.904+0000 7ff12f400640 1 -- 172.21.15.131:0/4115893976 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12400cd80 con 0x7ff138076970 2024-04-26T20:29:48.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.904+0000 7ff12f400640 1 -- 172.21.15.131:0/4115893976 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1240108e0 con 0x7ff138076970 2024-04-26T20:29:48.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.904+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1381a6ff0 con 0x7ff138076970 2024-04-26T20:29:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.904+0000 7ff12f400640 1 -- 172.21.15.131:0/4115893976 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124018980 con 0x7ff138076970 2024-04-26T20:29:48.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.904+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1381a7510 con 0x7ff138076970 2024-04-26T20:29:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.905+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff138105120 con 0x7ff138076970 2024-04-26T20:29:48.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.907+0000 7ff12f400640 1 -- 172.21.15.131:0/4115893976 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff124010430 con 0x7ff138076970 2024-04-26T20:29:48.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.908+0000 7ff12f400640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff11c0a3c40 0x7ff11c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.908+0000 7ff12f400640 1 -- 172.21.15.131:0/4115893976 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff1240d0c90 con 0x7ff138076970 2024-04-26T20:29:48.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.908+0000 7ff136a00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff11c0a3c40 0x7ff11c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:48.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.909+0000 7ff136a00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff11c0a3c40 0x7ff11c0a6100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff128009530 tx=0x7ff128009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:48.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:47.912+0000 7ff12f400640 1 -- 172.21.15.131:0/4115893976 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff12409ca70 con 0x7ff138076970 2024-04-26T20:29:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.117+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff13802dd20 con 0x7ff138076970 2024-04-26T20:29:48.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.118+0000 7ff12f400640 1 -- 172.21.15.131:0/4115893976 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7ff1240a0d70 con 0x7ff138076970 2024-04-26T20:29:48.932 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:29:48.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff11c0a3c40 msgr2=0x7ff11c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff11c0a3c40 0x7ff11c0a6100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff128009530 tx=0x7ff128009260 comp rx=0 tx=0).stop 2024-04-26T20:29:48.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff138076970 msgr2=0x7ff13819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:49.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff138076970 0x7ff13819f680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e9f0 tx=0x7ff12400eeb0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 shutdown_connections 2024-04-26T20:29:49.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810ef50 0x7ff1381a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff11c0a3c40 0x7ff11c0a6100 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff138076970 0x7ff13819f680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 --2- 172.21.15.131:0/4115893976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff138075390 0x7ff13819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 >> 172.21.15.131:0/4115893976 conn(0x7ff1380fdf10 msgr2=0x7ff1381022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:49.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 shutdown_connections 2024-04-26T20:29:49.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:48.121+0000 7ff13da00640 1 -- 172.21.15.131:0/4115893976 wait complete. 2024-04-26T20:29:49.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:47 smithi131 ceph-mon[27919]: pgmap v1003: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T20:29:49.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:29:49.321 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:48 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4115893976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:29:49.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:48 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4115893976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:29:49.335 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:47 smithi157 ceph-mon[32703]: pgmap v1003: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T20:29:49.342 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:48 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4115893976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:29:49.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 -- 172.21.15.131:0/3443665034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938104a70 msgr2=0x7fd938104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:49.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 --2- 172.21.15.131:0/3443665034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938104a70 0x7fd938104e90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd928009ca0 tx=0x7fd92802a8c0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 -- 172.21.15.131:0/3443665034 shutdown_connections 2024-04-26T20:29:49.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 --2- 172.21.15.131:0/3443665034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938106620 0x7fd93810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 --2- 172.21.15.131:0/3443665034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938105c60 0x7fd9381060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 --2- 172.21.15.131:0/3443665034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938104a70 0x7fd938104e90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:49.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 -- 172.21.15.131:0/3443665034 >> 172.21.15.131:0/3443665034 conn(0x7fd9380fffe0 msgr2=0x7fd938102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 -- 172.21.15.131:0/3443665034 shutdown_connections 2024-04-26T20:29:49.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.547+0000 7fd93ca00640 1 -- 172.21.15.131:0/3443665034 wait complete. 2024-04-26T20:29:49.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.548+0000 7fd93ca00640 1 Processor -- start 2024-04-26T20:29:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd93ca00640 1 -- start start 2024-04-26T20:29:49.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd93ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938104a70 0x7fd93819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:49.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd93ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938105c60 0x7fd93819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:49.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd93ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938106620 0x7fd9381a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd93ca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381105f0 con 0x7fd938104a70 2024-04-26T20:29:49.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd93ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd938110470 con 0x7fd938105c60 2024-04-26T20:29:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd93ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd938110770 con 0x7fd938106620 2024-04-26T20:29:49.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd936800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938104a70 0x7fd93819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:49.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd936800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938104a70 0x7fd93819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33316/0 (socket says 172.21.15.131:33316) 2024-04-26T20:29:50.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd936800640 1 -- 172.21.15.131:0/457972837 learned_addr learned my addr 172.21.15.131:0/457972837 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:50.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd92de00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938105c60 0x7fd93819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:50.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.549+0000 7fd937200640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938106620 0x7fd9381a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:50.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd936800640 1 -- 172.21.15.131:0/457972837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938106620 msgr2=0x7fd9381a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:50.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd936800640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938106620 0x7fd9381a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:50.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd936800640 1 -- 172.21.15.131:0/457972837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938105c60 msgr2=0x7fd93819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd936800640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938105c60 0x7fd93819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:50.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd936800640 1 -- 172.21.15.131:0/457972837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9381a2900 con 0x7fd938104a70 2024-04-26T20:29:50.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd937200640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938106620 0x7fd9381a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:29:50.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd936800640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938104a70 0x7fd93819acc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd928005e30 tx=0x7fd9280317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:50.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd92fe00640 1 -- 172.21.15.131:0/457972837 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928003bf0 con 0x7fd938104a70 2024-04-26T20:29:50.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd92fe00640 1 -- 172.21.15.131:0/457972837 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd928003d90 con 0x7fd938104a70 2024-04-26T20:29:50.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.550+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9381a2b90 con 0x7fd938104a70 2024-04-26T20:29:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.551+0000 7fd92fe00640 1 -- 172.21.15.131:0/457972837 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92802fc50 con 0x7fd938104a70 2024-04-26T20:29:50.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.551+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9381a30e0 con 0x7fd938104a70 2024-04-26T20:29:50.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.552+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93810bdb0 con 0x7fd938104a70 2024-04-26T20:29:50.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.554+0000 7fd92fe00640 1 -- 172.21.15.131:0/457972837 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd928004040 con 0x7fd938104a70 2024-04-26T20:29:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.555+0000 7fd92fe00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd91c0a3c40 0x7fd91c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.555+0000 7fd92fe00640 1 -- 172.21.15.131:0/457972837 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd9280f1e70 con 0x7fd938104a70 2024-04-26T20:29:50.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.555+0000 7fd92de00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd91c0a3c40 0x7fd91c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:50.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.556+0000 7fd92de00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd91c0a3c40 0x7fd91c0a6100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9200095f0 tx=0x7fd920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:50.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.560+0000 7fd92fe00640 1 -- 172.21.15.131:0/457972837 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd9280be440 con 0x7fd938104a70 2024-04-26T20:29:50.481 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:49 smithi148 ceph-mon[32281]: pgmap v1004: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:29:50.488 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:49 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/457972837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:29:50.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:49 smithi157 ceph-mon[32703]: pgmap v1004: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:29:50.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:49 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/457972837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:29:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.762+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd93802dcd0 con 0x7fd938104a70 2024-04-26T20:29:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.763+0000 7fd92fe00640 1 -- 172.21.15.131:0/457972837 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fd9280bdd40 con 0x7fd938104a70 2024-04-26T20:29:50.587 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:29:50.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.766+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd91c0a3c40 msgr2=0x7fd91c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:50.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.766+0000 7fd93ca00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd91c0a3c40 0x7fd91c0a6100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9200095f0 tx=0x7fd920005f90 comp rx=0 tx=0).stop 2024-04-26T20:29:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.766+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938104a70 msgr2=0x7fd93819acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:50.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.766+0000 7fd93ca00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938104a70 0x7fd93819acc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd928005e30 tx=0x7fd9280317e0 comp rx=0 tx=0).stop 2024-04-26T20:29:50.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 shutdown_connections 2024-04-26T20:29:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd91c0a3c40 0x7fd91c0a6100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938106620 0x7fd9381a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:50.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd938105c60 0x7fd93819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:50.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 --2- 172.21.15.131:0/457972837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd938104a70 0x7fd93819acc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:50.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 >> 172.21.15.131:0/457972837 conn(0x7fd9380fffe0 msgr2=0x7fd938108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:50.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 shutdown_connections 2024-04-26T20:29:50.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:49.767+0000 7fd93ca00640 1 -- 172.21.15.131:0/457972837 wait complete. 2024-04-26T20:29:50.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:29:50.817 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:49 smithi131 ceph-mon[27919]: pgmap v1004: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T20:29:50.825 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:49 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/457972837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:29:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 -- 172.21.15.131:0/924680884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564105b00 msgr2=0x7f2564107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:51.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 --2- 172.21.15.131:0/924680884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564105b00 0x7f2564107ef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f254c009900 tx=0x7f254c02a400 comp rx=0 tx=0).stop 2024-04-26T20:29:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 -- 172.21.15.131:0/924680884 shutdown_connections 2024-04-26T20:29:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 --2- 172.21.15.131:0/924680884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2564108430 0x7f256410a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:51.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 --2- 172.21.15.131:0/924680884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564105b00 0x7f2564107ef0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 --2- 172.21.15.131:0/924680884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25641031d0 0x7f25641055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:51.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 -- 172.21.15.131:0/924680884 >> 172.21.15.131:0/924680884 conn(0x7f25640fcc00 msgr2=0x7f25640ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:51.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 -- 172.21.15.131:0/924680884 shutdown_connections 2024-04-26T20:29:51.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.075+0000 7f2563200640 1 -- 172.21.15.131:0/924680884 wait complete. 2024-04-26T20:29:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.076+0000 7f2563200640 1 Processor -- start 2024-04-26T20:29:51.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.076+0000 7f2563200640 1 -- start start 2024-04-26T20:29:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.076+0000 7f2563200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25641031d0 0x7f256419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:51.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.076+0000 7f2563200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2564105b00 0x7f256419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2563200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564108430 0x7f25641a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:51.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2563200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f256410e2e0 con 0x7f2564105b00 2024-04-26T20:29:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2563200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f256410e160 con 0x7f25641031d0 2024-04-26T20:29:51.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2563200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f256410e460 con 0x7f2564108430 2024-04-26T20:29:51.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2564105b00 0x7f256419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:51.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25641031d0 0x7f256419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:51.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2564105b00 0x7f256419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33328/0 (socket says 172.21.15.131:33328) 2024-04-26T20:29:51.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561400640 1 -- 172.21.15.131:0/1496950199 learned_addr learned my addr 172.21.15.131:0/1496950199 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:51.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2562800640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564108430 0x7f25641a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561e00640 1 -- 172.21.15.131:0/1496950199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564108430 msgr2=0x7f25641a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:51.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561e00640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564108430 0x7f25641a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:51.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561e00640 1 -- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2564105b00 msgr2=0x7f256419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:51.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561e00640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2564105b00 0x7f256419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:51.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.077+0000 7f2561e00640 1 -- 172.21.15.131:0/1496950199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25641a2860 con 0x7f25641031d0 2024-04-26T20:29:51.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.078+0000 7f2561e00640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25641031d0 0x7f256419ac50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f255400e9f0 tx=0x7f255400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:51.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.078+0000 7f255aa00640 1 -- 172.21.15.131:0/1496950199 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255400ccc0 con 0x7f25641031d0 2024-04-26T20:29:51.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.078+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25641a2b50 con 0x7f25641031d0 2024-04-26T20:29:51.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.078+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25641a30a0 con 0x7f25641031d0 2024-04-26T20:29:51.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.078+0000 7f255aa00640 1 -- 172.21.15.131:0/1496950199 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25540044d0 con 0x7f25641031d0 2024-04-26T20:29:51.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.078+0000 7f255aa00640 1 -- 172.21.15.131:0/1496950199 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255401c070 con 0x7f25641031d0 2024-04-26T20:29:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.079+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2540005290 con 0x7f25641031d0 2024-04-26T20:29:51.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.081+0000 7f255aa00640 1 -- 172.21.15.131:0/1496950199 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2554005020 con 0x7f25641031d0 2024-04-26T20:29:51.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.083+0000 7f255aa00640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25480a3cc0 0x7f25480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:51.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.083+0000 7f255aa00640 1 -- 172.21.15.131:0/1496950199 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f25540d0950 con 0x7f25641031d0 2024-04-26T20:29:51.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.083+0000 7f2561400640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25480a3cc0 0x7f25480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:51.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.084+0000 7f2561400640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25480a3cc0 0x7f25480a6180 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f254c000c00 tx=0x7f254c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.087+0000 7f255aa00640 1 -- 172.21.15.131:0/1496950199 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f255409cd80 con 0x7f25641031d0 2024-04-26T20:29:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.290+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2540005850 con 0x7f25641031d0 2024-04-26T20:29:52.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.292+0000 7f255aa00640 1 -- 172.21.15.131:0/1496950199 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f255409c680 con 0x7f25641031d0 2024-04-26T20:29:52.014 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:29:52.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25480a3cc0 msgr2=0x7f25480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:52.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25480a3cc0 0x7f25480a6180 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f254c000c00 tx=0x7f254c005a20 comp rx=0 tx=0).stop 2024-04-26T20:29:52.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25641031d0 msgr2=0x7f256419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:52.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25641031d0 0x7f256419ac50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f255400e9f0 tx=0x7f255400eeb0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 shutdown_connections 2024-04-26T20:29:52.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f25480a3cc0 0x7f25480a6180 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2564108430 0x7f25641a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2564105b00 0x7f256419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 --2- 172.21.15.131:0/1496950199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25641031d0 0x7f256419ac50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.294+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 >> 172.21.15.131:0/1496950199 conn(0x7f25640fcc00 msgr2=0x7f2564102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:52.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.295+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 shutdown_connections 2024-04-26T20:29:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:51.295+0000 7f2563200640 1 -- 172.21.15.131:0/1496950199 wait complete. 2024-04-26T20:29:52.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:29:52.261 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:51 smithi157 ceph-mon[32703]: pgmap v1005: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:52.300 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:51 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1496950199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:29:52.349 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:51 smithi131 ceph-mon[27919]: pgmap v1005: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:52.356 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:51 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1496950199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:29:52.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:51 smithi148 ceph-mon[32281]: pgmap v1005: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:52.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:51 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1496950199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:29:52.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 -- 172.21.15.131:0/25095211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c102980 msgr2=0x7f402c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:52.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 --2- 172.21.15.131:0/25095211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c102980 0x7f402c102da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f401c009860 tx=0x7f401c02a340 comp rx=0 tx=0).stop 2024-04-26T20:29:52.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 -- 172.21.15.131:0/25095211 shutdown_connections 2024-04-26T20:29:52.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 --2- 172.21.15.131:0/25095211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f402c104530 0x7f402c10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 --2- 172.21.15.131:0/25095211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c103b70 0x7f402c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 --2- 172.21.15.131:0/25095211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c102980 0x7f402c102da0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:52.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 -- 172.21.15.131:0/25095211 >> 172.21.15.131:0/25095211 conn(0x7f402c0fdf10 msgr2=0x7f402c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:52.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 -- 172.21.15.131:0/25095211 shutdown_connections 2024-04-26T20:29:52.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.579+0000 7f4032200640 1 -- 172.21.15.131:0/25095211 wait complete. 2024-04-26T20:29:52.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.580+0000 7f4032200640 1 Processor -- start 2024-04-26T20:29:52.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f4032200640 1 -- start start 2024-04-26T20:29:52.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f4032200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c102980 0x7f402c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:52.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f4032200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c103b70 0x7f402c074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:52.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f4032200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f402c104530 0x7f402c075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:52.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f4032200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f402c10e4b0 con 0x7f402c102980 2024-04-26T20:29:52.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f4032200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f402c10e330 con 0x7f402c104530 2024-04-26T20:29:52.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f4032200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f402c10e630 con 0x7f402c103b70 2024-04-26T20:29:52.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f402b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c102980 0x7f402c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:52.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f402b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c102980 0x7f402c0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33356/0 (socket says 172.21.15.131:33356) 2024-04-26T20:29:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f402b400640 1 -- 172.21.15.131:0/1035080036 learned_addr learned my addr 172.21.15.131:0/1035080036 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f402aa00640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c103b70 0x7f402c074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:53.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.581+0000 7f402be00640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f402c104530 0x7f402c075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f402b400640 1 -- 172.21.15.131:0/1035080036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c103b70 msgr2=0x7f402c074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:53.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f402b400640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c103b70 0x7f402c074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:53.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f402b400640 1 -- 172.21.15.131:0/1035080036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f402c104530 msgr2=0x7f402c075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:53.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f402b400640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f402c104530 0x7f402c075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:53.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f402b400640 1 -- 172.21.15.131:0/1035080036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c075af0 con 0x7f402c102980 2024-04-26T20:29:53.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f402be00640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f402c104530 0x7f402c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:53.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f402b400640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c102980 0x7f402c0766e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f401c0057c0 tx=0x7f401c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f4023e00640 1 -- 172.21.15.131:0/1035080036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c037040 con 0x7f402c102980 2024-04-26T20:29:53.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f4023e00640 1 -- 172.21.15.131:0/1035080036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401c030bd0 con 0x7f402c102980 2024-04-26T20:29:53.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c1a2cd0 con 0x7f402c102980 2024-04-26T20:29:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.582+0000 7f4023e00640 1 -- 172.21.15.131:0/1035080036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c005280 con 0x7f402c102980 2024-04-26T20:29:53.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.583+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1a31c0 con 0x7f402c102980 2024-04-26T20:29:53.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.584+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c109e10 con 0x7f402c102980 2024-04-26T20:29:53.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.586+0000 7f4023e00640 1 -- 172.21.15.131:0/1035080036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f401c043020 con 0x7f402c102980 2024-04-26T20:29:53.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.587+0000 7f4023e00640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f40100a3cc0 0x7f40100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:53.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.587+0000 7f4023e00640 1 -- 172.21.15.131:0/1035080036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f401c0efe30 con 0x7f402c102980 2024-04-26T20:29:53.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.588+0000 7f402aa00640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f40100a3cc0 0x7f40100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:53.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.588+0000 7f402aa00640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f40100a3cc0 0x7f40100a6180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f402c076450 tx=0x7f4014009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:53.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.592+0000 7f4023e00640 1 -- 172.21.15.131:0/1035080036 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f401c0bc370 con 0x7f402c102980 2024-04-26T20:29:53.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.796+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f402c063530 con 0x7f402c102980 2024-04-26T20:29:53.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.796+0000 7f4023e00640 1 -- 172.21.15.131:0/1035080036 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f401c0bbc70 con 0x7f402c102980 2024-04-26T20:29:53.609 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:29:53.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f40100a3cc0 msgr2=0x7f40100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:53.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f40100a3cc0 0x7f40100a6180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f402c076450 tx=0x7f4014009260 comp rx=0 tx=0).stop 2024-04-26T20:29:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c102980 msgr2=0x7f402c0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c102980 0x7f402c0766e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f401c0057c0 tx=0x7f401c032da0 comp rx=0 tx=0).stop 2024-04-26T20:29:53.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 shutdown_connections 2024-04-26T20:29:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f402c104530 0x7f402c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:53.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f40100a3cc0 0x7f40100a6180 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f402c103b70 0x7f402c074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:53.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 --2- 172.21.15.131:0/1035080036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f402c102980 0x7f402c0766e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:53.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 >> 172.21.15.131:0/1035080036 conn(0x7f402c0fdf10 msgr2=0x7f402c106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:53.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.799+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 shutdown_connections 2024-04-26T20:29:53.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:52.800+0000 7f4032200640 1 -- 172.21.15.131:0/1035080036 wait complete. 2024-04-26T20:29:53.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T20:29:53.911 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:52 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1035080036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:29:53.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:52 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1035080036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:29:53.979 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:52 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1035080036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:29:54.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 -- 172.21.15.131:0/3862445127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 msgr2=0x7f26a010cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:54.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 --2- 172.21.15.131:0/3862445127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 0x7f26a010cdf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ca0 tx=0x7f269802a8c0 comp rx=0 tx=0).stop 2024-04-26T20:29:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 -- 172.21.15.131:0/3862445127 shutdown_connections 2024-04-26T20:29:54.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 --2- 172.21.15.131:0/3862445127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 0x7f26a010cdf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 --2- 172.21.15.131:0/3862445127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 0x7f26a0106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:54.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 --2- 172.21.15.131:0/3862445127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a01049b0 0x7f26a0104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 -- 172.21.15.131:0/3862445127 >> 172.21.15.131:0/3862445127 conn(0x7f26a00fff40 msgr2=0x7f26a0102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:54.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 -- 172.21.15.131:0/3862445127 shutdown_connections 2024-04-26T20:29:54.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.183+0000 7f26a4c00640 1 -- 172.21.15.131:0/3862445127 wait complete. 2024-04-26T20:29:54.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:53 smithi148 ceph-mon[32281]: pgmap v1006: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:54.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:53 smithi157 ceph-mon[32703]: pgmap v1006: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 Processor -- start 2024-04-26T20:29:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 -- start start 2024-04-26T20:29:54.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a01049b0 0x7f26a019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:54.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 0x7f26a019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:54.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 0x7f26a01a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:54.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01104c0 con 0x7f26a0105ba0 2024-04-26T20:29:54.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0110340 con 0x7f26a01049b0 2024-04-26T20:29:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.184+0000 7f26a4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0110640 con 0x7f26a0106560 2024-04-26T20:29:54.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 0x7f26a019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:54.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 0x7f26a019b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49120/0 (socket says 172.21.15.131:49120) 2024-04-26T20:29:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 -- 172.21.15.131:0/2868635479 learned_addr learned my addr 172.21.15.131:0/2868635479 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:29:54.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f269f400640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a01049b0 0x7f26a019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:54.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f269fe00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 0x7f26a01a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 -- 172.21.15.131:0/2868635479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 msgr2=0x7f26a01a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:54.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 0x7f26a01a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 -- 172.21.15.131:0/2868635479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a01049b0 msgr2=0x7f26a019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:54.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a01049b0 0x7f26a019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 -- 172.21.15.131:0/2868635479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a01a2780 con 0x7f26a0105ba0 2024-04-26T20:29:54.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f269f400640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a01049b0 0x7f26a019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:29:54.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.185+0000 7f2696a00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 0x7f26a019b0a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9f0 tx=0x7f268800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:54.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.186+0000 7f269cc00640 1 -- 172.21.15.131:0/2868635479 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268800cd80 con 0x7f26a0105ba0 2024-04-26T20:29:54.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.186+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a01a2a70 con 0x7f26a0105ba0 2024-04-26T20:29:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.186+0000 7f269cc00640 1 -- 172.21.15.131:0/2868635479 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26880108e0 con 0x7f26a0105ba0 2024-04-26T20:29:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.186+0000 7f269cc00640 1 -- 172.21.15.131:0/2868635479 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2688018980 con 0x7f26a0105ba0 2024-04-26T20:29:54.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.186+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a01a3010 con 0x7f26a0105ba0 2024-04-26T20:29:54.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.187+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a010bcd0 con 0x7f26a0105ba0 2024-04-26T20:29:54.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.189+0000 7f269cc00640 1 -- 172.21.15.131:0/2868635479 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2688010430 con 0x7f26a0105ba0 2024-04-26T20:29:55.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.190+0000 7f269cc00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26840a3cc0 0x7f26840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:55.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.191+0000 7f269cc00640 1 -- 172.21.15.131:0/2868635479 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f26880d0c40 con 0x7f26a0105ba0 2024-04-26T20:29:55.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.191+0000 7f269f400640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26840a3cc0 0x7f26840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:55.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.191+0000 7f269f400640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26840a3cc0 0x7f26840a6180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f26a0105a00 tx=0x7f2690005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:55.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.195+0000 7f269cc00640 1 -- 172.21.15.131:0/2868635479 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f268809ca20 con 0x7f26a0105ba0 2024-04-26T20:29:55.145 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:53 smithi131 ceph-mon[27919]: pgmap v1006: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T20:29:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.446+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f26a00635d0 con 0x7f26a0105ba0 2024-04-26T20:29:55.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.479+0000 7f269cc00640 1 -- 172.21.15.131:0/2868635479 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v17) ==== 81+0+745785 (secure 0 0 0) 0x7f26880a0d20 con 0x7f26a0105ba0 2024-04-26T20:29:55.204 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:29:55.216 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:29:55.269 INFO:teuthology.orchestra.run.smithi131.stdout: "epoch": 17, 2024-04-26T20:29:55.276 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:29:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: "active_gid": 14152, 2024-04-26T20:29:55.340 INFO:teuthology.orchestra.run.smithi131.stdout: "active_name": "x", 2024-04-26T20:29:55.348 INFO:teuthology.orchestra.run.smithi131.stdout: "active_addrs": { 2024-04-26T20:29:55.357 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-04-26T20:29:55.415 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:29:55.464 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-04-26T20:29:55.473 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:6800", 2024-04-26T20:29:55.482 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2537375548 2024-04-26T20:29:55.491 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:55.500 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:29:55.553 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-26T20:29:55.595 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:6801", 2024-04-26T20:29:55.601 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2537375548 2024-04-26T20:29:55.609 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:29:55.618 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:29:55.669 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:55.680 INFO:teuthology.orchestra.run.smithi131.stdout: "active_addr": "172.21.15.131:6801/2537375548", 2024-04-26T20:29:55.736 INFO:teuthology.orchestra.run.smithi131.stdout: "active_change": "2024-04-26T19:59:13.443306+0000", 2024-04-26T20:29:55.744 INFO:teuthology.orchestra.run.smithi131.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T20:29:55.752 INFO:teuthology.orchestra.run.smithi131.stdout: "available": true, 2024-04-26T20:29:55.761 INFO:teuthology.orchestra.run.smithi131.stdout: "standbys": [ 2024-04-26T20:29:55.794 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:29:55.845 INFO:teuthology.orchestra.run.smithi131.stdout: "gid": 24106, 2024-04-26T20:29:55.852 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "z", 2024-04-26T20:29:55.858 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr_features": 4540701547738038271, 2024-04-26T20:29:55.865 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-04-26T20:29:55.872 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:29:55.902 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-04-26T20:29:55.944 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:29:55.952 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:29:55.960 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:29:55.967 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:29:56.019 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:29:56.073 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:29:56.080 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:56.087 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:56.094 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:29:56.102 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:56.131 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:56.180 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:56.189 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T20:29:56.195 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:56.202 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:56.209 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:56.249 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:56.286 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:29:56.292 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:29:56.300 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:29:56.307 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:56.352 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:56.416 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:29:56.423 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:56.430 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:56.436 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:29:56.442 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:29:56.489 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:29:56.544 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:29:56.551 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:29:56.558 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:29:56.566 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:29:56.573 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:29:56.627 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:29:56.636 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:56.644 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:56.686 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:56.694 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:56.753 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:29:56.788 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:29:56.796 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:29:56.804 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:56.811 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:56.818 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:29:56.933 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:56.996 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:57.003 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:57.012 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:29:57.021 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:57.030 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:57.039 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:57.084 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:57.128 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:29:57.142 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:29:57.151 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:29:57.218 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:57.285 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:57.294 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:29:57.303 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:57.312 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:57.320 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:29:57.364 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:29:57.418 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:29:57.427 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:29:57.436 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:29:57.445 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:29:57.453 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:29:57.486 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:29:57.525 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:29:57.564 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:57.570 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:57.578 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:57.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:29:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:29:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:29:57.672 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:57.679 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:57.723 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:29:57.777 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:57.784 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:57.791 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:57.801 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:29:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:57.847 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:57.885 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:57.893 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:57.900 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-04-26T20:29:57.908 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-04-26T20:29:57.951 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:29:58.012 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:58.020 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:58.026 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:29:58.034 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:58.041 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:58.072 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:58.132 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-04-26T20:29:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:58.146 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:58.153 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:58.161 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-04-26T20:29:58.242 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-04-26T20:29:58.247 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:29:58.255 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:58.262 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:58.301 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-04-26T20:29:58.360 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:58.367 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:58.374 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:58.381 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-04-26T20:29:58.389 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:58.434 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:58.490 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:58.496 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:58.503 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-04-26T20:29:58.511 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-04-26T20:29:58.521 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:29:58.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:58.602 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:58.612 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:29:58.621 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:58.732 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-04-26T20:29:58.741 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:58.749 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:58.758 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:58.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:58.797 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-04-26T20:29:58.840 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-04-26T20:29:58.847 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:29:58.854 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:58.862 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:58.871 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:29:58.911 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-04-26T20:29:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:59.026 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:59.035 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:59.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:59.100 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-04-26T20:29:59.108 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-04-26T20:29:59.117 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:29:59.163 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:59.217 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:59.226 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-04-26T20:29:59.235 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:59.244 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:59.253 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-04-26T20:29:59.352 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:59.358 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:59.365 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:59.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:59.436 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-04-26T20:29:59.492 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-04-26T20:29:59.501 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:29:59.508 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:59.514 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:59.521 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:29:59.567 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:59.633 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:59.641 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:59.648 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-04-26T20:29:59.655 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:29:59.662 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:29:59.701 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:29:59.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:29:59.749 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-04-26T20:29:59.757 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-04-26T20:29:59.764 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:29:59.826 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:29:59.881 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:29:59.889 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:29:59.897 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:29:59.904 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:29:59.911 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:29:59.957 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T20:30:00.001 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:00.007 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:00.016 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:00.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:00.035 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-04-26T20:30:00.044 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-04-26T20:30:00.127 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:00.136 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:00.145 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:00.156 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:00.217 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:00.271 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:00.282 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:00.291 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-04-26T20:30:00.302 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:00.311 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:00.338 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:00.392 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:00.401 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:00.410 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:00.418 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:00.426 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-04-26T20:30:00.459 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:00.499 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:00.505 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:00.513 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-04-26T20:30:00.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-04-26T20:30:00.566 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:00.574 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:00.628 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:00.635 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:00.640 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:00.656 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:00.706 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:00.759 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T20:30:00.767 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:00.774 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:00.782 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:00.791 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-04-26T20:30:00.882 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-04-26T20:30:00.890 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:00.899 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:00.907 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:00.959 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-04-26T20:30:01.018 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:01.028 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:01.037 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:01.046 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T20:30:01.055 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T20:30:01.114 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:01.163 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:01.174 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:01.183 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-04-26T20:30:01.193 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-04-26T20:30:01.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:30:01.231 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:01.307 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:01.313 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:30:01.322 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-04-26T20:30:01.331 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-04-26T20:30:01.367 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:01.422 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T20:30:01.431 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T20:30:01.439 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:01.447 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:01.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:01.522 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-04-26T20:30:01.578 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-04-26T20:30:01.587 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:30:01.595 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:01.604 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:01.613 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-04-26T20:30:01.646 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:30:01.653 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-04-26T20:30:01.661 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:01.670 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T20:30:01.678 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:01.717 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:01.777 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:01.786 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:01.794 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-04-26T20:30:01.803 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-04-26T20:30:01.812 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:01.855 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:01.902 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:01.910 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T20:30:01.922 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:01.930 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:01.937 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:01.971 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T20:30:02.007 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T20:30:02.014 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:02.021 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:02.132 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-04-26T20:30:02.191 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-04-26T20:30:02.198 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:30:02.206 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:02.213 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:02.220 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-04-26T20:30:02.280 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-04-26T20:30:02.321 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-04-26T20:30:02.328 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:02.335 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-04-26T20:30:02.344 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T20:30:02.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:02.380 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:02.417 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:02.425 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-04-26T20:30:02.431 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-04-26T20:30:02.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:02.507 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:02.554 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:02.561 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-04-26T20:30:02.568 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:02.574 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:02.580 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:02.638 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T20:30:02.708 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T20:30:02.714 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:02.723 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:02.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-04-26T20:30:02.798 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-04-26T20:30:02.887 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:30:02.896 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:02.905 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:02.914 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:30:03.018 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-04-26T20:30:03.076 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-04-26T20:30:03.086 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:03.094 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T20:30:03.104 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T20:30:03.113 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:03.206 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:03.260 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:03.270 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:03.280 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:03.289 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:03.298 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:03.404 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:03.440 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:03.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:03.458 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:03.467 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:03.523 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:03.574 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:03.582 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:03.593 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:03.601 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:03.609 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:03.671 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:03.712 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:03.719 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:03.726 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:03.736 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:03.744 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:03.788 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:03.832 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:03.839 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:03.847 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:03.854 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:03.900 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:03.955 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:03.963 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:03.971 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:03.978 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:03.986 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:04.046 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:04.083 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:04.091 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:04.099 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:04.106 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:04.113 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:04.154 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:04.232 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:04.238 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:04.246 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:04.253 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:04.312 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:04.369 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:04.376 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:04.382 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:04.390 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:04.396 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:04.468 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:04.540 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:04.548 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:04.555 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:04.564 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:04.570 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:04.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:04.673 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:04.681 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:04.687 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:04.741 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:04.801 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:04.812 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:04.821 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:04.828 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:04.835 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:04.896 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:04.949 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:04.968 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:04.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:04.986 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-04-26T20:30:05.020 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-04-26T20:30:05.090 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:30:05.096 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:05.105 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:05.114 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:30:05.212 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:05.221 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:05.281 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:05.290 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T20:30:05.299 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:05.307 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:05.393 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:05.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:05.433 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-04-26T20:30:05.442 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-04-26T20:30:05.452 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:05.461 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:05.511 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:05.556 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-04-26T20:30:05.565 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:05.574 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:05.583 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:05.699 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-04-26T20:30:05.759 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-04-26T20:30:05.768 INFO:teuthology.orchestra.run.smithi131.stdout: "read", 2024-04-26T20:30:05.775 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap", 2024-04-26T20:30:05.783 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap-read" 2024-04-26T20:30:05.790 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:05.860 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-04-26T20:30:05.922 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:05.929 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:05.938 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:05.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:05.954 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-04-26T20:30:06.014 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-04-26T20:30:06.051 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:06.060 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:06.068 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:06.074 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:06.124 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:06.164 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:06.172 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:06.181 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T20:30:06.188 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:06.237 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:06.265 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:06.273 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:30:06.281 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:30:06.293 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:06.302 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:06.345 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:06.389 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:30:06.395 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:06.402 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:06.409 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:06.516 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T20:30:06.524 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:06.580 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:06.589 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:06.598 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:06.607 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-04-26T20:30:06.671 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-04-26T20:30:06.745 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:06.751 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:06.761 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:06.770 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:30:06.779 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:30:06.831 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:06.888 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:06.897 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T20:30:06.906 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T20:30:06.915 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:06.974 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:07.034 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:07.043 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-04-26T20:30:07.051 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-04-26T20:30:07.058 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:30:07.067 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:07.103 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:07.161 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:30:07.169 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:07.175 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:07.183 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:07.189 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T20:30:07.255 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:07.283 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:07.290 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:07.297 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:07.302 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-04-26T20:30:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-04-26T20:30:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-04-26T20:30:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:30:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-04-26T20:30:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T20:30:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-04-26T20:30:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-04-26T20:30:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-04-26T20:30:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T20:30:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-04-26T20:30:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-04-26T20:30:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-04-26T20:30:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T20:30:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:08.501 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-04-26T20:30:08.510 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-04-26T20:30:08.518 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:08.573 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:08.632 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:08.641 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:08.649 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:08.658 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:08.666 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:08.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T20:30:08.749 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T20:30:08.758 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:08.772 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:08.780 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-04-26T20:30:08.826 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-04-26T20:30:08.873 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:08.883 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:08.890 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:08.897 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:30:08.984 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:09.044 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:09.052 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:09.059 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T20:30:09.067 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:09.074 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:09.139 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:09.187 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-04-26T20:30:09.201 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T20:30:09.209 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:30:09.217 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:09.269 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:09.300 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-04-26T20:30:09.308 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:09.321 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:09.365 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T20:30:09.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:09.432 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:09.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:09.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:09.452 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm_log_destination": { 2024-04-26T20:30:09.507 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm_log_destination", 2024-04-26T20:30:09.566 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:09.573 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:09.582 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:09.591 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:09.599 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:09.688 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:09.696 INFO:teuthology.orchestra.run.smithi131.stdout: "file", 2024-04-26T20:30:09.705 INFO:teuthology.orchestra.run.smithi131.stdout: "file,syslog", 2024-04-26T20:30:09.715 INFO:teuthology.orchestra.run.smithi131.stdout: "syslog" 2024-04-26T20:30:09.748 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:09.800 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T20:30:09.809 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:09.819 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:09.829 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:09.839 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:09.872 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-04-26T20:30:09.921 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-04-26T20:30:09.927 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:09.937 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:09.946 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:09.955 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:10.011 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:10.042 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:10.049 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:10.059 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T20:30:10.069 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:10.101 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:10.155 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:10.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:10.174 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-04-26T20:30:10.183 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-04-26T20:30:10.191 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:10.227 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:10.287 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:10.294 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:10.302 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:10.312 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:10.320 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:10.369 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T20:30:10.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:10.417 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:10.426 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:10.435 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:10.509 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-04-26T20:30:10.558 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-04-26T20:30:10.566 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:10.575 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:10.582 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:10.589 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:10.630 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T20:30:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:10.786 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:10.830 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:10.836 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-04-26T20:30:10.845 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-04-26T20:30:10.852 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:10.903 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:10.930 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:10.938 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T20:30:10.945 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:10.952 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:11.046 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:30:11.093 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:11.101 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:11.109 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:11.118 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:11.127 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-04-26T20:30:11.190 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-04-26T20:30:11.230 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:11.239 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:11.248 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:11.257 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T20:30:11.267 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:11.276 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:11.348 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:11.356 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-04-26T20:30:11.364 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:11.373 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:11.437 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:11.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:11.475 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-04-26T20:30:11.484 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-04-26T20:30:11.494 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:11.502 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:11.621 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T20:30:11.628 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:11.636 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:11.650 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:11.712 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-04-26T20:30:11.769 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:11.775 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:11.784 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:11.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:11.803 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-04-26T20:30:11.844 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-04-26T20:30:11.910 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:11.919 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:11.928 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:11.937 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T20:30:11.946 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:12.016 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:12.054 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:12.063 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:30:12.072 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:12.081 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:12.154 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:12.205 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:12.212 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-04-26T20:30:12.222 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-04-26T20:30:12.229 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:12.237 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:12.284 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:12.371 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T20:30:12.378 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:12.386 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:12.393 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:12.401 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-04-26T20:30:12.506 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:12.556 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:12.562 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:12.570 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:12.577 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-04-26T20:30:12.652 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-04-26T20:30:12.702 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:12.710 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:12.717 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:12.725 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T20:30:12.732 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:12.833 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:12.840 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-04-26T20:30:12.847 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:12.854 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:12.861 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:12.948 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:12.987 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-04-26T20:30:12.994 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-04-26T20:30:13.003 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:13.012 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:13.064 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:13.116 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T20:30:13.124 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:13.131 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:13.142 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:13.150 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-04-26T20:30:13.185 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:13.268 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:13.276 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:13.285 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:13.296 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-04-26T20:30:13.304 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-04-26T20:30:13.360 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:13.395 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:13.403 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:13.411 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T20:30:13.420 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:13.489 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:13.549 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:13.558 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-04-26T20:30:13.567 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:13.576 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:13.586 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:13.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:13.710 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-04-26T20:30:13.719 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-04-26T20:30:13.731 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:13.739 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:13.749 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:13.801 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T20:30:13.847 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:13.854 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:13.863 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:13.870 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-04-26T20:30:13.877 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:13.884 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:13.936 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:13.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:13.963 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-04-26T20:30:13.972 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-04-26T20:30:14.010 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:14.077 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:14.089 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:14.097 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T20:30:14.104 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:14.111 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:14.131 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:14.170 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-04-26T20:30:14.188 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:14.195 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:14.203 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:14.255 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:14.291 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-04-26T20:30:14.298 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-04-26T20:30:14.305 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:14.311 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:14.318 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:14.325 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T20:30:14.399 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:14.406 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:14.414 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:14.422 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:30:14.431 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:14.496 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:14.502 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:14.509 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:14.515 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-04-26T20:30:14.553 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-04-26T20:30:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:14.641 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:14.672 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:14.677 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T20:30:14.683 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:14.689 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:14.722 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:14.785 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-04-26T20:30:14.794 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:14.801 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:14.807 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:14.815 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:14.885 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-04-26T20:30:14.915 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-04-26T20:30:14.922 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:14.931 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:14.940 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:15.003 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T20:30:15.012 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:15.068 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:15.075 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:15.082 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:30:15.089 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:15.157 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:15.213 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:15.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:15.226 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-04-26T20:30:15.233 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-04-26T20:30:15.241 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:15.301 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:15.336 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:15.346 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T20:30:15.353 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:15.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:15.455 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:15.520 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-04-26T20:30:15.528 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:15.536 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:15.543 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:15.550 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:15.640 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_samba": { 2024-04-26T20:30:15.675 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_samba", 2024-04-26T20:30:15.681 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:15.688 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:15.695 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:15.703 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T20:30:15.736 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:15.772 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:15.779 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:15.789 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Samba/SMB container image", 2024-04-26T20:30:15.798 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:15.918 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-04-26T20:30:16.011 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-04-26T20:30:16.020 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:16.064 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:16.117 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:16.126 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T20:30:16.135 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:16.144 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:16.152 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:16.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-04-26T20:30:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:16.346 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-04-26T20:30:16.392 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-04-26T20:30:16.400 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:16.408 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:16.417 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:16.426 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:16.482 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:16.545 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:16.562 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T20:30:16.571 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:16.582 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-04-26T20:30:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-04-26T20:30:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:16.764 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:16.824 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:16.832 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:30:16.840 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:16.847 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:16.855 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:16.934 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T20:30:16.995 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:17.003 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:17.010 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:17.018 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:17.024 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-04-26T20:30:17.084 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T20:30:17.120 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:17.136 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:17.143 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-04-26T20:30:17.195 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:17.238 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:17.246 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T20:30:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:17.265 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:17.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:17.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:17.363 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-04-26T20:30:17.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-04-26T20:30:17.378 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:17.384 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:17.450 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:17.486 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-04-26T20:30:17.494 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:17.532 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:17.538 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:17.609 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T20:30:17.636 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:17.642 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:17.648 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:17.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:17.665 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-04-26T20:30:17.717 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-04-26T20:30:17.774 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:17.784 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:17.792 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-04-26T20:30:17.810 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:17.875 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:17.933 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:17.944 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-04-26T20:30:17.953 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:17.962 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:18.023 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-04-26T20:30:18.099 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-04-26T20:30:18.107 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:18.115 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:18.150 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:18.200 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:18.209 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:18.218 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:18.225 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:18.233 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T20:30:18.308 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:18.352 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:18.361 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:18.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:18.376 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-04-26T20:30:18.444 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-04-26T20:30:18.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:18.504 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:18.511 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:30:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:18.533 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:18.599 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:18.663 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-04-26T20:30:18.671 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:18.679 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:18.687 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:18.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:18.819 INFO:teuthology.orchestra.run.smithi131.stdout: "grafana_dashboards_path": { 2024-04-26T20:30:18.826 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "grafana_dashboards_path", 2024-04-26T20:30:18.832 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:18.839 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:18.897 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:18.967 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T20:30:18.975 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:18.983 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:18.991 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:18.999 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T20:30:19.055 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:19.112 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:19.118 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:19.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:19.133 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-04-26T20:30:19.140 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-04-26T20:30:19.216 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:19.252 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:19.261 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:19.270 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:30:19.277 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:19.325 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:19.332 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:19.388 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-04-26T20:30:19.397 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:19.405 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:19.414 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:19.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:19.545 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-04-26T20:30:19.552 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-04-26T20:30:19.561 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:19.570 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:19.579 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:19.669 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:19.738 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:19.744 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:19.753 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T20:30:19.863 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:19.871 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:19.930 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:19.939 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:19.948 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-04-26T20:30:19.957 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-04-26T20:30:20.041 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:20.102 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:20.110 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:20.118 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:20.126 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:20.136 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:20.200 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:20.235 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T20:30:20.243 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:20.251 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:20.258 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:20.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:20.343 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:20.350 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:20.358 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:20.363 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:20.369 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:20.442 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:20.450 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:20.458 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:20.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:20.507 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:20.514 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:20.566 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:20.602 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:20.609 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:20.616 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:20.623 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:20.669 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:20.722 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:20.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:20.742 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:20.750 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:20.758 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-04-26T20:30:20.829 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-04-26T20:30:20.901 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:20.909 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:20.916 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:20.924 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:20.930 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:21.015 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:21.052 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:21.060 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T20:30:21.069 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:21.083 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:21.141 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:21.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:21.179 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:21.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:21.197 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:21.206 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:21.304 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:21.353 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:21.362 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:21.370 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:21.379 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:21.389 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T20:30:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:21.510 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:21.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:21.525 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:21.569 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:21.637 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:21.647 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:21.654 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:21.709 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:21.717 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:21.726 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:21.734 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:21.778 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:21.791 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:21.849 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:21.885 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:21.892 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:21.898 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:21.905 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:21.947 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:22.005 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:22.013 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:22.021 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:22.028 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:22.035 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:22.101 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:22.153 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:22.159 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:22.166 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:22.173 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:22.181 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:22.188 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:22.277 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:22.286 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:22.294 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:22.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:22.356 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T20:30:22.411 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T20:30:22.419 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:22.426 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:22.434 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:22.441 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:22.498 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:22.567 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:22.574 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:22.582 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T20:30:22.589 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:22.596 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:22.676 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:22.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:22.716 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T20:30:22.723 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T20:30:22.731 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:22.772 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:22.825 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:22.832 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-04-26T20:30:22.839 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:22.846 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:22.853 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:22.932 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T20:30:22.996 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:23.004 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:23.014 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:23.034 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-04-26T20:30:23.118 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-04-26T20:30:23.151 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:23.160 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:23.169 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:23.178 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:30:23.227 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:23.280 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:23.289 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:23.297 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-04-26T20:30:23.306 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:23.315 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:23.368 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:23.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:23.429 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-04-26T20:30:23.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-04-26T20:30:23.446 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:23.455 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:23.557 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:23.593 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:30:23.602 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:23.610 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:23.618 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:23.673 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T20:30:23.728 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:23.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:23.743 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:23.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:23.760 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-04-26T20:30:23.847 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-04-26T20:30:23.894 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:23.900 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:23.907 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:23.914 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:23.922 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:24.003 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:24.042 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:24.051 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-04-26T20:30:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:24.105 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-04-26T20:30:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-04-26T20:30:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:24.247 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:24.307 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-04-26T20:30:24.315 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:24.322 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:24.330 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:24.338 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-04-26T20:30:24.410 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-04-26T20:30:24.451 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:24.458 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T20:30:24.466 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:24.473 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:24.518 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:24.573 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:24.581 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-04-26T20:30:24.588 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-04-26T20:30:24.599 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:24.606 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:24.651 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:24.694 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-04-26T20:30:24.703 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:24.710 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:24.719 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T20:30:24.806 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:24.842 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:24.864 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-04-26T20:30:24.936 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-04-26T20:30:24.995 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:25.005 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:25.014 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:25.023 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T20:30:25.031 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:25.082 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:25.148 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:25.158 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T20:30:25.167 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:25.175 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:25.184 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:25.271 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:25.309 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-04-26T20:30:25.318 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-04-26T20:30:25.326 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:25.335 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T20:30:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-04-26T20:30:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-04-26T20:30:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:25.769 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T20:30:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-04-26T20:30:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-04-26T20:30:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T20:30:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-04-26T20:30:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-04-26T20:30:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T20:30:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-04-26T20:30:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-04-26T20:30:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:26.806 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T20:30:26.813 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:26.822 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:26.830 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:26.839 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:26.893 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-04-26T20:30:26.902 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-04-26T20:30:26.963 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:27.023 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:27.032 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:27.041 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-04-26T20:30:27.050 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:27.058 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:27.120 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:27.176 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-04-26T20:30:27.183 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:27.191 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:27.199 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:27.208 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:27.269 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-04-26T20:30:27.308 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-04-26T20:30:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:27.324 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:27.332 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:27.379 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:27.437 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:27.444 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:27.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:27.459 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T20:30:27.465 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:27.531 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:27.593 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:27.600 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:27.607 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-04-26T20:30:27.614 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-04-26T20:30:27.620 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:27.680 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:27.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:27.739 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:27.745 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:27.753 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:27.802 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:27.809 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T20:30:27.860 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:27.868 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:27.876 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:27.883 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:27.964 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-04-26T20:30:28.026 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-04-26T20:30:28.033 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:28.041 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:28.048 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:28.056 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:28.125 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:28.166 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:28.174 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:28.182 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T20:30:28.191 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:28.246 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:28.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:28.309 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:28.318 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-04-26T20:30:28.328 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-04-26T20:30:28.337 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:28.398 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:28.451 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:28.460 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:28.471 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:28.478 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:28.486 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:28.547 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T20:30:28.583 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:28.591 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:28.600 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:28.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:28.670 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-04-26T20:30:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-04-26T20:30:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:28.814 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:28.824 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:28.833 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T20:30:28.842 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:28.850 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:28.926 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:28.935 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:28.942 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-04-26T20:30:28.974 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-04-26T20:30:28.983 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:29.037 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:29.045 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:29.104 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:29.113 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:29.119 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:29.127 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:29.183 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T20:30:29.242 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:29.249 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:29.256 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:29.263 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:29.269 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:29.413 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-04-26T20:30:29.421 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:29.428 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:29.454 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:29.499 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:29.506 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:29.513 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:29.521 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:29.528 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:29.580 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:29.634 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:29.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:29.651 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:29.658 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:29.665 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:29.790 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:29.824 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:29.830 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:29.837 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:29.844 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:29.903 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:29.955 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:29.965 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:29.976 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:29.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:29.994 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:30.042 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:30.115 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:30.124 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:30.133 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:30.141 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:30.214 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:30.220 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:30.229 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:30.266 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:30.274 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:30.325 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:30.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:30.362 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:30.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:30.382 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:30.391 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:30.434 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:30.484 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:30.491 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:30.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:30.508 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:30.516 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:30.579 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:30.616 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:30.626 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:30.635 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:30.642 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:30.687 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:30.696 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:30.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:30.755 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:30.761 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:30.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:30.825 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:30.892 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:30.899 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:30.906 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:30.913 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:30.921 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:30.972 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:31.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:31.030 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:31.038 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:31.045 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:31.095 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:31.147 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:31.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:31.161 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-04-26T20:30:31.168 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-04-26T20:30:31.177 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:31.233 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:31.292 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:31.300 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-04-26T20:30:31.307 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:31.314 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:31.329 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:31.392 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T20:30:31.448 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:31.455 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:31.464 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:31.474 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:31.537 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-04-26T20:30:31.560 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-04-26T20:30:31.567 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:31.577 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:31.588 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:31.595 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-04-26T20:30:31.651 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:31.659 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:31.666 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:31.713 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T20:30:31.723 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:31.732 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:31.824 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:31.886 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:31.902 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:31.911 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:31.961 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-04-26T20:30:32.008 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:32.017 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:32.026 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:32.034 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T20:30:32.044 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T20:30:32.090 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:32.099 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:32.108 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:32.150 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:30:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:32.251 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:32.336 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:32.348 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:32.355 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:32.362 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:32.445 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:32.508 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T20:30:32.517 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T20:30:32.524 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:32.532 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:32.541 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:32.636 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:32.646 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:32.655 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:32.708 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:32.718 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:32.727 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:32.797 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:32.861 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:32.869 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:32.875 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T20:30:32.881 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T20:30:32.944 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:32.951 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:33.003 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:33.010 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:33.018 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:33.025 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:33.075 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:33.125 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:33.132 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:33.140 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:33.147 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:33.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:33.237 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-04-26T20:30:33.272 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T20:30:33.285 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:33.298 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:33.307 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:33.345 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:33.408 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:33.417 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:33.426 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:33.434 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:33.444 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:33.486 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:33.533 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:33.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:33.552 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T20:30:33.561 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T20:30:33.570 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:33.638 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:33.645 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:33.656 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:33.690 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:33.696 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:33.745 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:33.754 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:33.807 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:33.814 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:33.822 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:33.829 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:33.880 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T20:30:33.887 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T20:30:33.897 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:33.945 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:33.951 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:33.957 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:34.012 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:34.064 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:34.072 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:34.078 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:34.085 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:34.119 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:34.186 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:34.194 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:34.201 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T20:30:34.209 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T20:30:34.216 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:34.297 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:34.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:34.357 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:34.364 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:34.370 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:34.375 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:34.428 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:34.464 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:34.472 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:34.478 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:34.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:34.529 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T20:30:34.592 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T20:30:34.600 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:34.607 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:34.614 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:34.620 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:34.676 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:34.749 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:34.759 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:34.768 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:34.776 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:34.786 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:34.855 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:34.906 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:34.915 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T20:30:34.927 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T20:30:34.936 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:34.969 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:35.023 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:35.033 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:35.042 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:35.051 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:35.060 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:35.174 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:35.181 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:35.190 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:35.196 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:35.201 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T20:30:35.279 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T20:30:35.331 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:35.337 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:35.346 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:35.355 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:35.394 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:35.453 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:35.462 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:35.471 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:35.480 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:35.489 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:35.539 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:35.605 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:35.614 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T20:30:35.622 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T20:30:35.629 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:35.636 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:35.711 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:35.765 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:35.772 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:35.780 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:35.789 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:35.827 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:35.835 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:35.885 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:35.893 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:35.901 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:35.908 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T20:30:35.969 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T20:30:36.007 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:36.013 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:36.034 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:36.111 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:36.158 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:36.165 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:36.173 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:36.183 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:36.237 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:36.290 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:36.297 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T20:30:36.304 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T20:30:36.311 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:36.380 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:36.415 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:36.423 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:36.430 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:36.437 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:36.444 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:36.508 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:36.578 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:36.590 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:36.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:36.611 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T20:30:36.691 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T20:30:36.748 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:36.757 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:36.766 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:36.775 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:36.787 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:36.845 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:36.882 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:36.891 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:36.901 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:36.910 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:36.919 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:36.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:37.037 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T20:30:37.045 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T20:30:37.051 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:37.059 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:37.144 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:30:37.201 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:37.208 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:37.225 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:37.308 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:37.363 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:37.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:37.380 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:37.389 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T20:30:37.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T20:30:37.471 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:37.501 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:37.508 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:37.515 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:37.522 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:37.589 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:37.640 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:37.647 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:37.654 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:37.661 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:37.667 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:37.674 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:37.720 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-04-26T20:30:37.744 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-04-26T20:30:37.752 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:37.758 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:37.765 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:37.837 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:37.940 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:37.948 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:37.957 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:37.965 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:37.995 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:38.029 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:38.036 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:38.044 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:38.051 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T20:30:38.066 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T20:30:38.073 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:38.145 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:38.152 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:38.159 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:30:38.166 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:38.175 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:38.236 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:38.276 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:38.286 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:38.294 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:38.303 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:38.337 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:38.386 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T20:30:38.396 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T20:30:38.401 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:38.410 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:38.422 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:38.432 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:38.440 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:38.448 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:38.457 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:38.465 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:38.473 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:38.482 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:38.492 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:38.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T20:30:38.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T20:30:38.527 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:38.534 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:38.540 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:38.546 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:38.554 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:38.613 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:38.706 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:38.736 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:38.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:38.769 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:38.776 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:38.785 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:38.793 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T20:30:38.801 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T20:30:38.808 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:38.816 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:38.825 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:38.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:38.841 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:38.849 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:38.859 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:38.867 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:38.951 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:38.962 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:38.969 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:38.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:38.985 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T20:30:38.994 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T20:30:39.002 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:39.010 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:39.018 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:39.026 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:39.035 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:39.043 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:39.051 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:39.059 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:39.068 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:39.080 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:39.089 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:39.161 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:39.170 INFO:teuthology.orchestra.run.smithi131.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T20:30:39.178 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T20:30:39.186 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:39.194 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:39.203 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:39.211 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "[]", 2024-04-26T20:30:39.219 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:39.228 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:39.236 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:39.246 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:39.256 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:39.264 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:39.271 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:39.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:39.292 INFO:teuthology.orchestra.run.smithi131.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T20:30:39.300 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T20:30:39.354 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:39.362 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:39.370 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:39.379 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "{}", 2024-04-26T20:30:39.387 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:39.395 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:39.403 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:39.414 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:39.423 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:39.431 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:39.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:39.447 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:39.456 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T20:30:39.465 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T20:30:39.474 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:39.482 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:39.492 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:39.547 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:39.557 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:39.564 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:39.572 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:39.580 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:39.593 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:39.601 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:39.609 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:39.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:39.626 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T20:30:39.633 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T20:30:39.644 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:39.651 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:39.669 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:39.677 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:39.686 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:39.749 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:39.757 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:39.765 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:39.774 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:39.790 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:39.799 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T20:30:39.808 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T20:30:39.815 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:39.823 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:39.832 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:39.839 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:39.847 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:39.858 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:39.867 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:39.878 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:39.887 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:39.945 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:39.954 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:39.962 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:39.971 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T20:30:39.979 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T20:30:39.987 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:39.994 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:40.002 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:40.010 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:40.019 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:40.027 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:40.034 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:40.042 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:40.050 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:40.057 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:40.065 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:40.074 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T20:30:40.142 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T20:30:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:40.159 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:40.168 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:40.178 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:40.186 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:40.194 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:40.202 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:40.209 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:40.216 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:40.224 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:40.231 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:40.238 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:40.246 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T20:30:40.254 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T20:30:40.262 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:40.304 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:40.313 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:40.322 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:40.330 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:40.338 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:40.348 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:40.356 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:40.364 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:40.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:40.381 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:40.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:40.398 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T20:30:40.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T20:30:40.415 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:40.423 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:40.432 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:40.440 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:40.494 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:40.502 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:40.512 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:40.520 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:40.529 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:40.539 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:40.547 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:40.556 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:40.565 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T20:30:40.573 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T20:30:40.583 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:40.592 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:40.601 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:40.623 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:40.633 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:40.641 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:40.718 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:40.727 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:40.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:40.743 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:40.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:40.760 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T20:30:40.768 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T20:30:40.776 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:40.784 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:40.793 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:40.801 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:40.808 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:40.816 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:40.825 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:40.833 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:40.842 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:40.850 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:40.902 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:40.910 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:40.919 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T20:30:40.928 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T20:30:40.936 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:40.944 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:40.953 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:40.961 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:40.969 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:40.977 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:40.986 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:40.995 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:41.003 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:41.012 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:41.020 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:41.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:41.039 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T20:30:41.102 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T20:30:41.111 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:41.120 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:41.128 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:41.137 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T20:30:41.146 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:41.154 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:41.163 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:41.171 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:41.192 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:41.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:41.205 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T20:30:41.215 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T20:30:41.223 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:41.231 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:41.298 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:41.305 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:30:41.313 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:41.326 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:41.334 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:41.342 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:41.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:41.359 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:41.367 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:41.375 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:41.384 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T20:30:41.393 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T20:30:41.399 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:41.406 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:41.412 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:41.420 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-04-26T20:30:41.429 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:41.506 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:41.515 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:41.522 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:41.530 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:41.537 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:41.544 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:41.551 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:41.557 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T20:30:41.564 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T20:30:41.571 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:41.577 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:41.584 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:41.591 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-04-26T20:30:41.598 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:41.605 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:41.612 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:41.619 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:41.689 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:41.702 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:41.711 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:41.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:41.728 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T20:30:41.736 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T20:30:41.744 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:41.752 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:41.761 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:41.769 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:41.777 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:41.785 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:41.796 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:41.804 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:41.811 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:41.819 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:41.828 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:41.868 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T20:30:41.884 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T20:30:41.892 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:41.900 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:41.911 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:41.919 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:30:41.928 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:41.936 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:41.945 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:41.954 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:41.962 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:41.971 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:41.979 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:41.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:41.994 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T20:30:42.002 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T20:30:42.051 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:42.060 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:42.070 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:42.078 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:42.086 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:42.094 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:42.102 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:42.115 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:42.123 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:42.132 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:42.141 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:42.150 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:42.159 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T20:30:42.167 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T20:30:42.176 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:42.185 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:42.194 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:42.237 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:42.245 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:42.254 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:42.262 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:42.271 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:42.279 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:42.287 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:42.294 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:42.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:42.311 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T20:30:42.319 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T20:30:42.327 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:42.335 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:42.344 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:42.352 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:42.361 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:42.367 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:42.409 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:42.418 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:42.426 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:42.435 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:42.443 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:42.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:42.461 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T20:30:42.469 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T20:30:42.478 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:42.485 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:42.493 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:30:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:42.523 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:42.531 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:42.539 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:42.548 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:42.619 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:42.626 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:42.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:42.642 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T20:30:42.650 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T20:30:42.658 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:42.667 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:42.675 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:42.683 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:30:42.690 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:42.699 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:42.706 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:42.712 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:42.719 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:42.725 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:42.731 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:42.738 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:42.798 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T20:30:42.806 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T20:30:42.813 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:42.821 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:42.827 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:42.834 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:30:42.842 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:42.848 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:42.855 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:42.862 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:42.868 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:42.875 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:42.881 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:42.888 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:42.894 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-04-26T20:30:42.901 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-04-26T20:30:42.908 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:42.961 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:42.970 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:42.979 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:42.989 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:42.997 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:43.023 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:43.031 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:43.039 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:43.048 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:43.059 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-04-26T20:30:43.068 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-04-26T20:30:43.076 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:43.084 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:43.092 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:43.100 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:43.162 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:43.171 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:43.181 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:43.189 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:43.198 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:43.206 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:43.214 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:43.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:43.232 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-04-26T20:30:43.241 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-04-26T20:30:43.251 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:43.260 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:43.267 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:43.275 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:43.282 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:43.290 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:43.298 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:43.342 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-04-26T20:30:43.353 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:43.364 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:43.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:43.381 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:43.389 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-04-26T20:30:43.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-04-26T20:30:43.406 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:43.415 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:43.424 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:43.432 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-04-26T20:30:43.441 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:43.450 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:43.458 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:43.467 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:43.476 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:43.484 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:43.546 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:43.555 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:43.564 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-04-26T20:30:43.574 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-04-26T20:30:43.583 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:43.600 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:43.608 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:43.616 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:43.625 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:43.633 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:43.642 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:43.650 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:43.659 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:43.667 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:43.677 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:43.685 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:43.741 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:43.748 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:43.755 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:43.763 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:43.770 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:43.777 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:43.784 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:43.792 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:43.804 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:43.811 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:43.818 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:43.827 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:43.834 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:43.841 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:43.849 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:43.855 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:43.865 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:43.921 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:43.928 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:43.935 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:43.942 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:43.949 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:43.956 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:43.963 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:43.970 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:43.977 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:43.984 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:43.991 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:43.999 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:44.006 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:44.013 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:44.020 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:44.026 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:44.034 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:44.100 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:44.107 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:44.113 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:44.120 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:44.129 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:44.137 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:44.144 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:44.153 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:44.161 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:44.169 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:44.178 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:44.187 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:44.195 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:44.206 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:44.215 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:44.222 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:44.230 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:44.291 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:44.300 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:44.309 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:44.318 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:44.326 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:44.336 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:44.344 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:44.353 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:44.362 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:44.370 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:44.379 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:44.388 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:44.396 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:44.405 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:44.413 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:44.422 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:44.430 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:44.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-04-26T20:30:44.510 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-04-26T20:30:44.519 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:44.528 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:44.536 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:44.544 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:44.553 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:44.561 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:44.570 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:44.579 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-04-26T20:30:44.588 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:44.596 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:44.605 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:44.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:44.622 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-04-26T20:30:44.632 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T20:30:44.694 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:44.703 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:44.712 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:44.720 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:44.728 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:44.736 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:44.744 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:44.753 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:44.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:44.770 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:44.778 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:44.787 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:44.795 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:30:44.803 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:30:44.812 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:44.820 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:44.828 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:44.893 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-04-26T20:30:44.902 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:44.911 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:44.919 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:44.927 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:44.935 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:44.944 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:44.951 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:44.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:44.968 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:30:44.988 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:30:44.996 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:45.005 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:45.013 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:45.023 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-04-26T20:30:45.032 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:45.041 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:45.107 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:45.115 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:45.123 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:45.131 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:45.139 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:45.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:45.153 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-04-26T20:30:45.160 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-04-26T20:30:45.167 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:45.176 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:45.182 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:45.190 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:45.196 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:45.204 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:45.211 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:45.218 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:45.225 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:45.283 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:45.291 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:45.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:45.305 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-04-26T20:30:45.312 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-04-26T20:30:45.321 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:45.331 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:45.339 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:45.346 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-04-26T20:30:45.354 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:45.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:45.371 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:45.379 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:45.387 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:45.395 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:45.404 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:45.414 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:45.468 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-04-26T20:30:45.475 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-04-26T20:30:45.484 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:45.492 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:45.501 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:45.510 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-04-26T20:30:45.518 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:45.526 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:45.535 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:45.544 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:45.553 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-04-26T20:30:45.562 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:45.569 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:45.577 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:45.584 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:45.592 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:45.599 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:45.659 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-04-26T20:30:45.668 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-04-26T20:30:45.676 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:45.685 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:45.693 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:45.701 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-04-26T20:30:45.709 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-04-26T20:30:45.718 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-04-26T20:30:45.726 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:45.734 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:45.743 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:45.751 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:45.756 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:45.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:45.772 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-04-26T20:30:45.780 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-04-26T20:30:45.788 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:45.857 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:45.866 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:45.875 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:45.884 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:45.892 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:45.901 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:45.909 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:45.916 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:45.924 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:45.950 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:45.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:45.967 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:45.977 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-04-26T20:30:45.984 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:45.993 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:46.042 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:46.051 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-04-26T20:30:46.060 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-04-26T20:30:46.071 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:46.079 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:46.088 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:46.097 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:46.106 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:46.115 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:46.126 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:46.135 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-04-26T20:30:46.144 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:46.154 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:46.162 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:46.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:46.179 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:46.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:46.258 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:46.267 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:46.276 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:46.285 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:46.294 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:46.302 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:46.312 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:46.320 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:46.328 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:46.338 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:46.346 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:46.354 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:46.364 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:46.374 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:46.399 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:46.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:46.490 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:46.498 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:46.506 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:46.514 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:46.523 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:46.531 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:46.539 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:46.547 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:46.556 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:46.564 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:46.573 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:46.582 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:46.590 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:46.598 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:46.605 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:46.615 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:46.675 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:46.710 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:46.719 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:46.728 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:46.736 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:46.743 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:46.749 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:46.758 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:46.766 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:46.774 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:46.782 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:46.789 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:46.798 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:46.807 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:46.813 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:46.824 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:46.831 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:46.891 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:46.899 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:46.906 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:46.914 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:46.922 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:46.930 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:46.939 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:46.947 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:46.955 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:46.965 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:46.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:46.981 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:46.989 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:46.997 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:47.005 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:47.014 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:47.022 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:47.091 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-04-26T20:30:47.099 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-04-26T20:30:47.107 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:47.115 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:47.122 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:47.129 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-04-26T20:30:47.136 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:47.143 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:47.151 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:47.158 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T20:30:47.165 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:47.173 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:47.179 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:47.186 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:47.193 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-04-26T20:30:47.201 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-04-26T20:30:47.208 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:47.266 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:47.275 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:47.284 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-04-26T20:30:47.292 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:47.301 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:47.309 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:47.317 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T20:30:47.326 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:47.334 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:47.342 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:47.351 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:47.361 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-04-26T20:30:47.369 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-04-26T20:30:47.377 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:47.394 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:47.403 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-04-26T20:30:47.466 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:47.476 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:47.484 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:47.492 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-04-26T20:30:47.501 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:47.509 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:47.518 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:47.527 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:47.535 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-04-26T20:30:47.544 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-04-26T20:30:47.553 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:47.560 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:47.568 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:47.576 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-04-26T20:30:47.584 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:47.591 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:47.600 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:47.658 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T20:30:47.666 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:47.674 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:47.682 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:47.690 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:47.699 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-04-26T20:30:47.708 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-04-26T20:30:47.716 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:47.725 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:47.734 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:47.742 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:30:47.751 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:47.760 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:47.769 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:47.778 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T20:30:47.787 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:47.795 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:47.854 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:47.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:47.871 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:30:47.880 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:30:47.888 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:47.896 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:47.905 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:47.914 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:30:47.925 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:47.933 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:47.941 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:47.950 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T20:30:47.958 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:47.967 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:47.974 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:47.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:47.991 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-04-26T20:30:48.058 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-04-26T20:30:48.067 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:48.076 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:48.085 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:48.094 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-04-26T20:30:48.103 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:48.110 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:48.117 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:48.125 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T20:30:48.133 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:48.140 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:48.148 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:48.155 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:48.162 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:48.169 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:48.175 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:48.183 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-04-26T20:30:48.239 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:48.246 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:48.253 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:48.260 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:48.268 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:48.275 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:48.285 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:48.293 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:48.300 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:48.307 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:48.314 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:48.320 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:48.326 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:48.333 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:48.341 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:48.348 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:48.357 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:48.408 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:48.415 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:48.422 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:48.429 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:48.435 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:48.442 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:48.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:48.455 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:48.463 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:48.470 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:48.476 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:48.484 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:48.490 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:48.497 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:48.503 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:48.510 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:48.516 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:48.583 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:48.590 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:48.597 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:48.605 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:48.612 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:48.620 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:48.628 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:48.636 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:48.644 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:48.653 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:48.661 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:48.669 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:48.677 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:48.685 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:48.694 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:48.703 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:48.712 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:48.766 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:48.774 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:48.783 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:48.792 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:48.800 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:48.809 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:48.818 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:48.827 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:48.835 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:48.872 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:48.880 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:48.888 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:48.896 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:48.905 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:48.966 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:48.974 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:48.983 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:48.991 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:49.000 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:49.008 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-04-26T20:30:49.017 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-04-26T20:30:49.027 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:49.034 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:49.042 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:49.051 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-04-26T20:30:49.059 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:49.068 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:49.076 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:49.085 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:49.093 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:49.102 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:49.178 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:49.186 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:49.195 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-04-26T20:30:49.204 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-04-26T20:30:49.213 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:49.221 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:49.228 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:49.235 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-04-26T20:30:49.245 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:49.252 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:49.260 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:49.268 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:49.275 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:49.284 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:49.291 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:49.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:49.305 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:30:49.367 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:30:49.374 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:49.381 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:49.389 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:49.395 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:30:49.402 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:49.409 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:49.417 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:49.424 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:49.431 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:49.438 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:49.445 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:49.453 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:49.460 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:49.470 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:49.477 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:49.484 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-04-26T20:30:49.549 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-04-26T20:30:49.556 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-04-26T20:30:49.564 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:49.573 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-04-26T20:30:49.581 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-04-26T20:30:49.590 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:49.600 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:49.609 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:49.618 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-04-26T20:30:49.629 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:49.638 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:49.650 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:49.657 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T20:30:49.664 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:49.671 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:49.678 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:49.686 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:49.749 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-04-26T20:30:49.757 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-04-26T20:30:49.765 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:49.773 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-04-26T20:30:49.796 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:49.803 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:49.811 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:49.820 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T20:30:49.826 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:49.832 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:49.839 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:49.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:49.861 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-04-26T20:30:49.868 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-04-26T20:30:49.876 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:49.932 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:49.939 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:49.945 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:49.952 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:49.959 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:49.968 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:49.976 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-04-26T20:30:49.984 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:49.992 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:50.001 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:50.009 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:50.017 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:30:50.026 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:30:50.034 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:30:50.043 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:50.052 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:50.061 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-04-26T20:30:50.140 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-04-26T20:30:50.149 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:50.159 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:50.167 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T20:30:50.176 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:50.186 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:50.195 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:50.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:50.212 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:50.220 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:50.229 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:50.237 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:50.245 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:50.253 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:50.261 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:50.269 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:50.277 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:50.335 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:50.343 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:50.352 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:50.360 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:50.369 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:50.378 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:50.386 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:50.395 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:50.403 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:50.411 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:50.420 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:50.429 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:50.438 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:50.446 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:50.455 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:50.463 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:50.473 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:50.537 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:50.545 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:50.557 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:50.565 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:50.574 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:50.583 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:50.591 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:50.600 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:50.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:50.616 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:50.626 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:50.634 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:50.642 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:50.649 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:50.658 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:50.665 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:50.673 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:50.731 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:50.740 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:50.748 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:50.774 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:50.782 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:50.791 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:50.800 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:50.808 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:50.816 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:50.824 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:50.832 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:50.841 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:50.850 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:50.858 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:50.867 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:50.915 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:50.922 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:50.932 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:50.941 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:50.948 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:50.956 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:50.963 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:50.970 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:50.978 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:50.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:50.995 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-04-26T20:30:51.003 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-04-26T20:30:51.012 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:51.020 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:51.028 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:51.037 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:51.044 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:51.102 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:51.111 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:51.120 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-04-26T20:30:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:51.135 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:51.144 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:51.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:51.161 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-04-26T20:30:51.170 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-04-26T20:30:51.179 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:51.187 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:51.196 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:51.204 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-04-26T20:30:51.213 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:51.221 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:51.230 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:51.239 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-04-26T20:30:51.312 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:51.321 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:51.329 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:51.338 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:51.347 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-04-26T20:30:51.356 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-04-26T20:30:51.364 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:51.372 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:51.380 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:51.388 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-04-26T20:30:51.396 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:51.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:51.423 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T20:30:51.432 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:51.440 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:51.448 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:51.510 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:51.518 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-04-26T20:30:51.526 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-04-26T20:30:51.537 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:51.546 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:51.554 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:51.563 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:30:51.571 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:30:51.580 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-04-26T20:30:51.589 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:51.598 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T20:30:51.607 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:51.624 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:51.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:51.641 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-04-26T20:30:51.650 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-04-26T20:30:51.698 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:51.707 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:51.717 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:51.727 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:51.735 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:51.758 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-04-26T20:30:51.768 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:51.777 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:51.785 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:51.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:51.801 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-04-26T20:30:51.810 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-04-26T20:30:51.818 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:51.824 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:51.830 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:51.883 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-04-26T20:30:51.892 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:51.900 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:51.909 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:51.916 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T20:30:51.925 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:51.936 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:51.942 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:51.951 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:51.959 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:51.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:51.973 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:51.983 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-04-26T20:30:51.993 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:52.003 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:52.014 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:52.025 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:52.094 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:52.101 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:52.108 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:52.118 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:52.128 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:52.138 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:52.147 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:52.155 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:52.164 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:52.171 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:52.179 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:52.186 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:52.195 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:52.206 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:52.212 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:52.219 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:52.227 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:52.286 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:52.292 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:52.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:52.309 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:52.322 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:52.331 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:52.338 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:52.347 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:52.359 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:52.371 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:52.381 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:52.391 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:52.401 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:52.407 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:52.417 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:52.426 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:52.438 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:52.498 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:52.508 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:52.523 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:52.534 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:52.546 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:52.555 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:52.563 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:52.577 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:52.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:52.604 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:52.615 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:52.629 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:52.638 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:52.650 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:52.662 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:52.673 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:52.690 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:52.756 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:52.766 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:52.774 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:52.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:52.792 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:52.801 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:52.810 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:52.819 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:52.827 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:52.837 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:52.846 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:52.854 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:52.864 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:52.872 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:52.883 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:52.891 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:52.899 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:52.964 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:52.973 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:52.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:52.992 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:53.002 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-04-26T20:30:53.011 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:53.019 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:53.028 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:53.038 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:53.050 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:53.059 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:53.066 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:53.074 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:53.084 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:53.094 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:53.103 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:53.112 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:53.165 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:53.174 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:53.184 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:53.193 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:53.211 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:53.220 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:53.229 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:53.238 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:53.245 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:53.252 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:53.260 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:53.266 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:53.278 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:53.288 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:53.298 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:53.307 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:53.319 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:53.381 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:53.402 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:53.414 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:53.424 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:53.434 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:53.444 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:53.458 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:53.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:53.482 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:53.488 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:53.498 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:53.507 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:53.516 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:53.526 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:53.536 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:53.546 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:53.604 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:53.616 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:53.625 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:53.635 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:53.644 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:53.656 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:53.665 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:53.674 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:53.682 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:53.689 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:53.697 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:53.703 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:53.712 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:53.722 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:53.730 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:53.740 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:53.750 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:53.831 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:53.840 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:53.860 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:53.869 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:53.880 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:53.891 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:53.901 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:53.910 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:53.920 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:53.926 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:53.933 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:53.941 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:53.951 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-04-26T20:30:53.958 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:53.965 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:53.971 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:54.013 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-04-26T20:30:54.023 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-04-26T20:30:54.033 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:54.051 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:54.064 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-04-26T20:30:54.073 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:54.083 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:54.092 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:54.102 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T20:30:54.111 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:54.119 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:54.129 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:54.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:54.150 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-04-26T20:30:54.159 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-04-26T20:30:54.168 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:54.214 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:54.233 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:30:54.242 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:30:54.252 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-04-26T20:30:54.260 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:54.268 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:54.277 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T20:30:54.287 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:54.296 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:54.306 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:54.318 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:54.329 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:54.338 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:54.350 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:54.365 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:54.378 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:54.389 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:54.447 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:54.456 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:54.469 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:54.487 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:54.495 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:54.508 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:54.516 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:54.528 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:54.544 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:54.555 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:54.563 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:54.575 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:54.585 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:54.594 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:54.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:54.613 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:54.624 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:54.693 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:54.704 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:54.715 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:54.729 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:54.743 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:54.751 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:54.762 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:54.776 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:54.787 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:54.794 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:54.801 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:54.807 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:54.819 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:54.829 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:54.838 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:54.848 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:54.857 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:54.921 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:54.932 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:54.940 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:54.949 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:54.958 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:54.966 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:54.977 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:54.988 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:55.002 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:55.010 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:55.021 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:55.030 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:55.041 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:55.051 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:55.060 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:55.069 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:55.079 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:55.134 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:55.144 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:55.154 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:55.173 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:55.181 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:55.191 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:55.202 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:55.208 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:55.218 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:55.229 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:55.238 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:55.246 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:55.254 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:55.263 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:55.273 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:55.282 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:55.291 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-04-26T20:30:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:55.347 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:55.355 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:55.365 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-04-26T20:30:55.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-04-26T20:30:55.381 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:55.389 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:55.398 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:55.406 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-04-26T20:30:55.413 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:55.421 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:55.429 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:55.437 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-04-26T20:30:55.446 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T20:30:55.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:55.460 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:55.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:55.526 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:55.532 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:55.538 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:55.547 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:55.555 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:55.565 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:55.575 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:55.584 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:55.596 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:55.610 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:55.622 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:55.635 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:55.650 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:55.662 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:55.676 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:55.691 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:55.702 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:55.755 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:55.766 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:55.778 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:55.790 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:55.804 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:55.818 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:55.832 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:55.843 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:55.851 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:55.860 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:55.871 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:55.882 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:55.896 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:55.907 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:55.916 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:55.935 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:56.008 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:56.021 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:56.032 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:56.048 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:56.059 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:56.073 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:56.083 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:56.091 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:56.103 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:56.113 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:56.121 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:56.143 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:56.155 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:56.163 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:56.174 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:56.186 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:56.264 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:56.274 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:56.282 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:56.294 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:56.304 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:56.313 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:56.323 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:56.332 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:56.341 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:56.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:56.358 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:56.367 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:56.379 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:56.390 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:56.400 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:56.410 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:56.418 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:56.464 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:56.487 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:56.494 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-04-26T20:30:56.503 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-04-26T20:30:56.512 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:56.522 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:56.531 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:56.541 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:56.555 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:56.564 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:56.572 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:56.578 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-04-26T20:30:56.586 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T20:30:56.596 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:56.604 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:56.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:56.620 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-04-26T20:30:56.660 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-04-26T20:30:56.667 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:56.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:56.682 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:56.689 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-04-26T20:30:56.696 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:56.703 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:56.709 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:56.716 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-04-26T20:30:56.726 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:56.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:56.743 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:56.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:56.758 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-04-26T20:30:56.765 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-04-26T20:30:56.774 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:30:56.782 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:56.833 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:56.841 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-04-26T20:30:56.851 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:56.865 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:56.878 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:56.884 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T20:30:56.893 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:56.903 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:56.913 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:56.932 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:56.944 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-04-26T20:30:56.952 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-04-26T20:30:56.962 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:56.968 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:56.978 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:56.988 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:56.994 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:57.047 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:57.057 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:57.068 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-04-26T20:30:57.083 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:57.096 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:57.107 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:57.119 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:57.131 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-04-26T20:30:57.142 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-04-26T20:30:57.154 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:57.163 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:57.175 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:57.186 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-04-26T20:30:57.199 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:57.211 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:57.223 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:57.235 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T20:30:57.272 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T20:30:57.283 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:57.299 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:57.307 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:57.318 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:57.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:57.346 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:57.357 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-04-26T20:30:57.371 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:57.383 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:57.394 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:57.404 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:57.417 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:57.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:57.434 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:57.446 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:57.458 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:57.509 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:57.517 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:57.529 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:57.542 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:57.554 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:57.567 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:57.578 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:57.592 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:57.603 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:57.611 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:57.622 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:57.631 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:57.654 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:57.665 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:57.684 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:57.738 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:57.747 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:57.755 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:57.764 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:57.772 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:57.785 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:57.794 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:57.804 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:57.814 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:57.822 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:57.832 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:57.843 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:57.854 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:57.861 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:57.869 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:57.875 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:57.883 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:57.955 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:57.962 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:57.978 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:57.986 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:57.994 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:58.002 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:58.009 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:58.016 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:58.023 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:58.031 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:58.038 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:58.045 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:58.053 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:58.059 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:58.070 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:58.076 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:58.130 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:58.145 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:58.154 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:58.162 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:58.169 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:58.175 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:58.183 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:58.190 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:58.208 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:58.214 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:58.220 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:58.226 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:58.234 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:58.242 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:58.250 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-04-26T20:30:58.308 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:58.318 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:58.330 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:58.343 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:58.356 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:58.369 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:58.382 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:58.395 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:58.407 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:58.419 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:58.430 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:58.452 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:58.463 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:58.478 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:58.492 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:58.502 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:58.588 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:58.599 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:58.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:58.622 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:58.687 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:58.698 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:58.711 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:58.724 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:58.733 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:58.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:58.759 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:58.767 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:58.825 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:58.838 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:58.850 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:58.861 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:58.870 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:58.878 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:58.890 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:58.900 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:58.908 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:58.919 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:58.927 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:58.937 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:59.038 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:59.047 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:59.059 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:59.067 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:59.075 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:59.084 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:59.094 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:59.103 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:59.112 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:30:59.118 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:30:59.127 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:59.135 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:59.143 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:59.154 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:59.162 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:59.171 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:59.181 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:59.240 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:59.249 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:59.258 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:59.287 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:30:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:30:59.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-04-26T20:30:59.325 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:30:59.336 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:30:59.345 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:30:59.355 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:30:59.364 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:30:59.373 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:59.381 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:59.389 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:59.430 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:30:59.437 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:59.445 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:59.453 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:59.461 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:59.469 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:59.476 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:59.484 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:59.493 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:59.500 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:59.508 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:59.515 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:59.522 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:59.530 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:59.537 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:30:59.604 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:30:59.611 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:30:59.619 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:59.629 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:59.646 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:30:59.654 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:59.666 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:59.675 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:30:59.683 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:59.693 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:59.705 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:30:59.716 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:30:59.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:30:59.742 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:30:59.756 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:30:59.769 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:30:59.780 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:30:59.845 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:30:59.854 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:30:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:30:59.872 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:30:59.884 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:30:59.893 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:30:59.907 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:30:59.919 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:30:59.930 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:30:59.942 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:30:59.953 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:30:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:30:59.972 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:30:59.981 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:30:59.991 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:00.000 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:00.011 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:00.088 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:00.096 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:00.107 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:00.117 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:00.132 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:00.141 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:00.152 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:00.163 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:00.174 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:00.182 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:00.193 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:00.204 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:00.212 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:00.223 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:00.233 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:00.241 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:00.249 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:00.311 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-04-26T20:31:00.324 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:00.335 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:00.350 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:00.362 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-04-26T20:31:00.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-04-26T20:31:00.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:00.397 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:00.408 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:00.417 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:00.425 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:00.434 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:00.443 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:00.453 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T20:31:00.464 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:00.476 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:00.486 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:00.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:00.542 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:00.552 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:00.561 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:00.571 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:00.581 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:00.597 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:00.606 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:00.614 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:00.624 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:00.637 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:00.645 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:00.653 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:00.664 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:00.675 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:00.685 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:00.763 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:00.771 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:00.780 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:00.789 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:00.799 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:00.809 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:00.817 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:00.828 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:00.846 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:00.869 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:00.878 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:00.888 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:00.895 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:00.904 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:00.974 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:00.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:00.988 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:00.996 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:01.003 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:01.009 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:01.016 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:01.022 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:01.030 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:01.038 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:01.044 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:01.050 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:01.058 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:01.066 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:01.072 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:01.079 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:01.085 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:01.152 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:01.160 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:01.173 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:01.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:01.191 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:01.199 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:01.208 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:01.216 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:01.226 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:01.236 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:01.261 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:01.270 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:01.281 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:01.338 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:01.348 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:01.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:01.366 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-04-26T20:31:01.379 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-04-26T20:31:01.390 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:01.399 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:01.408 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:01.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:01.431 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:01.442 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:01.453 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:01.464 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-04-26T20:31:01.473 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-04-26T20:31:01.482 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-04-26T20:31:01.491 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:01.502 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-04-26T20:31:01.549 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:01.560 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:01.568 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:01.578 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:01.586 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:01.608 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:01.623 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-04-26T20:31:01.633 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:01.644 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:01.656 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:01.667 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:01.674 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:01.686 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:01.694 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:01.709 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:01.717 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:01.774 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:01.785 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:01.793 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:01.807 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:01.816 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:01.824 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:01.835 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:01.844 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:01.855 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:01.869 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:01.882 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:01.893 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:01.906 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:01.918 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:01.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:01.943 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:01.954 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:02.022 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:02.082 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:02.091 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:02.102 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:02.112 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:02.121 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:02.130 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:02.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:02.150 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:02.159 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:02.168 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:02.176 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:02.185 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:02.246 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:02.255 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:02.265 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:02.274 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:02.282 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:02.291 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:02.300 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:02.308 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:02.316 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:02.325 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:02.334 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:02.343 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:02.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:02.358 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:02.366 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:02.373 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:02.381 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:02.453 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:02.460 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:02.469 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:02.514 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:02.519 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:02.526 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:02.534 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:02.541 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:02.547 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:02.555 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:02.563 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:02.571 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:02.577 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:02.585 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:02.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:02.600 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:02.607 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-04-26T20:31:02.671 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:02.680 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:02.691 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:02.701 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:02.710 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:02.720 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:02.729 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:02.754 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:02.763 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:02.773 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:02.783 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:02.792 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:02.801 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:02.811 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:02.820 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:02.876 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:02.887 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:02.895 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:02.905 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:02.917 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:02.928 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:02.939 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:02.950 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:02.961 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:02.969 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:02.982 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:02.991 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:03.003 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:03.015 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:03.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:03.031 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:03.043 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:03.100 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:03.122 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:03.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:03.142 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:03.153 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:03.166 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:03.179 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:03.190 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:03.201 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:03.215 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:03.225 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:03.236 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:03.251 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:03.265 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:03.276 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:03.285 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:03.337 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:03.347 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:03.357 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:03.368 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:03.380 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:03.389 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:03.401 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:03.415 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:03.425 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:03.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:03.450 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:03.461 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:03.470 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:03.478 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:03.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:03.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:03.508 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:03.593 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:03.601 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:03.610 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:03.621 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:03.631 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:03.640 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:03.649 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:03.658 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:03.666 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-04-26T20:31:03.676 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:03.687 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:03.694 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:03.702 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:03.710 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:03.717 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:03.725 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:03.734 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:03.808 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:03.816 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:03.824 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:03.834 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:03.840 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:03.848 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:03.857 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:03.865 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:03.873 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:03.880 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:03.888 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:03.898 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:03.906 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:03.914 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:03.922 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:03.930 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:03.938 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:04.006 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:04.013 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:04.020 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:04.027 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:04.035 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:04.042 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:04.049 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:04.057 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:04.065 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:04.072 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:04.079 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:04.088 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:04.097 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:04.106 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:04.115 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:04.124 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:04.133 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:04.194 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:04.205 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:04.214 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:04.224 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:04.234 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:04.244 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:04.252 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:04.262 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:04.272 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:04.280 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:04.290 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:04.299 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:04.312 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:04.335 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:04.349 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:04.362 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:04.373 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:04.432 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:04.448 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:04.458 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:04.471 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:04.482 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:04.494 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:04.506 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:04.521 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:04.535 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:04.547 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:04.556 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:04.564 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:04.575 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:04.589 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:04.598 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:31:04.607 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:31:04.619 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:04.674 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:04.683 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:04.692 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:31:04.706 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:04.718 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:04.732 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:04.747 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:04.762 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:04.775 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:04.787 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:04.799 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:04.812 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-04-26T20:31:04.821 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-04-26T20:31:04.832 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:31:04.840 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:04.852 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:04.864 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-04-26T20:31:04.913 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-04-26T20:31:04.922 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:04.931 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:04.942 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-04-26T20:31:04.951 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T20:31:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:04.970 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:04.982 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:04.993 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:05.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:05.017 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:05.028 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-04-26T20:31:05.039 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:05.051 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:05.062 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:05.073 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-04-26T20:31:05.084 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-04-26T20:31:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:05.192 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:05.200 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:05.208 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T20:31:05.219 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:05.232 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:05.241 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:05.251 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:05.261 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-04-26T20:31:05.274 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-04-26T20:31:05.284 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:05.293 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:05.301 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:05.381 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:05.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:05.397 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:05.404 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:05.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:05.420 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:05.428 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:05.436 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:05.443 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:05.451 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:05.459 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:05.466 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:05.474 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:05.482 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:05.490 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:05.498 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:05.506 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:05.561 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:05.568 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:05.576 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:05.583 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:05.591 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:05.598 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:05.605 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:05.614 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:05.620 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:05.629 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:05.637 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:05.646 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:05.654 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:05.662 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:05.672 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:05.682 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:05.692 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:05.751 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:05.760 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:05.769 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:05.779 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:05.789 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:05.798 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:05.808 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:05.818 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:05.828 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:05.837 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:05.848 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:05.857 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:05.866 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:05.878 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:05.888 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:05.898 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:05.906 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:05.962 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:05.970 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:05.980 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:05.990 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:06.002 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:06.012 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:06.021 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:06.033 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:06.043 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:06.052 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:06.061 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:06.069 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:06.078 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:06.087 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:06.099 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:06.107 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:06.116 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:06.177 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:06.188 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:06.208 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:06.216 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:06.228 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:06.239 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:06.253 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:06.262 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:06.273 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:06.284 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:06.295 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:06.307 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-04-26T20:31:06.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-04-26T20:31:06.324 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:06.332 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:06.344 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:06.355 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-04-26T20:31:06.394 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:06.424 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:06.440 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:06.450 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-04-26T20:31:06.459 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:06.467 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:06.475 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:06.487 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:06.498 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:31:06.510 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:31:06.522 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:06.536 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:06.548 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:06.559 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:31:06.568 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:06.579 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:06.590 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:06.645 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-04-26T20:31:06.655 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:06.663 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:06.673 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:06.682 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:06.690 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:06.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:06.710 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:06.721 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-04-26T20:31:06.731 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:06.739 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:06.750 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:06.761 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-04-26T20:31:06.770 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-04-26T20:31:06.782 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:06.789 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:06.797 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:06.853 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:06.923 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:06.937 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:06.944 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:06.954 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:06.960 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:06.968 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:06.975 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:06.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:06.990 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-04-26T20:31:06.997 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-04-26T20:31:07.004 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:07.011 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:07.018 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:07.025 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:07.032 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:07.039 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:07.089 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:07.098 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T20:31:07.107 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T20:31:07.116 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:07.124 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:07.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:07.141 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:07.150 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:07.159 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:07.168 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:07.176 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:07.185 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:07.194 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:07.203 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:07.211 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:07.220 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:07.229 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:07.305 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-04-26T20:31:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-04-26T20:31:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T20:31:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T20:31:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-04-26T20:31:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-04-26T20:31:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-04-26T20:31:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:31:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-04-26T20:31:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:31:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:31:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-04-26T20:31:08.504 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:08.512 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:08.520 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:08.529 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T20:31:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:08.545 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:08.554 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:08.561 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:08.570 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:31:08.578 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:31:08.585 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:08.594 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:08.675 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:08.683 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-04-26T20:31:08.692 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:08.701 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:08.710 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:08.718 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T20:31:08.726 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:08.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:08.741 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:08.750 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:08.758 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-04-26T20:31:08.766 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-04-26T20:31:08.774 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:08.782 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:08.789 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:08.798 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-04-26T20:31:08.808 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:08.862 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:08.940 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:08.949 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:08.958 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:08.967 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:08.975 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:08.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:08.992 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-04-26T20:31:09.001 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-04-26T20:31:09.010 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:09.018 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:09.027 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:09.035 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-04-26T20:31:09.043 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:09.051 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:09.059 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:09.067 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-04-26T20:31:09.143 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-04-26T20:31:09.152 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:09.161 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:09.170 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:09.178 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:09.186 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:09.202 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-04-26T20:31:09.211 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-04-26T20:31:09.219 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:09.227 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:09.235 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:09.243 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-04-26T20:31:09.251 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-04-26T20:31:09.259 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-04-26T20:31:09.268 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:09.276 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:09.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:09.358 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:09.366 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:09.374 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:09.382 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:09.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:09.398 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:09.406 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-04-26T20:31:09.414 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:09.423 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:09.432 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:09.443 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:09.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:09.461 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:09.470 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:09.479 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:09.487 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:09.536 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:09.544 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:09.552 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:09.560 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:09.567 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:09.574 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:09.580 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:09.587 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:09.597 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:09.604 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:09.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:09.618 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:09.626 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:09.635 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:09.642 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:09.649 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:09.725 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:09.732 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:09.739 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:09.746 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:09.756 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:09.763 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:09.770 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:09.777 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:09.784 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:09.792 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:09.799 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:09.806 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:09.813 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:09.820 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:09.827 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:09.836 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:09.845 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:09.903 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:09.913 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:09.921 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:09.930 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:09.938 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:09.945 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:09.954 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:09.963 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:09.971 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:09.997 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:10.006 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:10.014 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:10.023 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:10.031 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:10.040 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:10.103 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:10.111 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:10.119 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:10.128 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:10.136 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:10.145 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:10.154 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:10.163 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:10.172 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:10.181 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:10.190 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:10.198 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:10.207 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:10.215 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-04-26T20:31:10.224 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-04-26T20:31:10.232 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:10.240 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:10.323 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:10.333 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:31:10.341 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:10.349 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:10.358 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:10.366 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:10.375 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:10.383 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:10.394 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:10.402 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:10.409 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-04-26T20:31:10.417 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T20:31:10.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:10.434 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:10.442 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:10.452 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:10.460 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:10.526 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:10.534 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:10.543 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:10.550 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:10.559 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:10.568 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:10.576 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:10.584 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-04-26T20:31:10.593 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-04-26T20:31:10.602 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:10.610 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:10.619 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:10.628 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:10.636 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:10.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:10.652 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:10.659 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:10.734 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:10.743 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:10.751 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:10.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:10.775 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:10.784 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-04-26T20:31:10.793 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:10.801 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:10.809 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:10.817 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-04-26T20:31:10.825 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-04-26T20:31:10.833 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:10.842 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:10.851 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:10.859 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:10.921 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:10.928 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:10.936 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:10.944 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:10.951 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-04-26T20:31:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-04-26T20:31:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:11.009 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:11.018 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:11.024 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:11.031 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:11.038 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:11.111 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:11.118 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:11.127 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:11.134 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:11.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:11.149 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:11.156 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:11.163 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:11.172 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:11.181 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:11.190 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:11.198 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:11.207 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:11.216 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:11.225 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:11.234 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:11.243 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:11.299 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:11.307 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:11.317 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:11.325 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:11.333 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:11.342 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:11.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:11.359 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:11.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:11.376 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:11.384 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:11.393 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:11.399 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:11.408 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:11.416 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:11.424 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:11.433 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:11.491 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:11.500 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:11.509 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:11.518 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:11.527 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:11.536 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:11.545 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:11.554 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:11.562 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:11.569 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:11.577 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:11.586 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:11.595 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:11.603 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:11.612 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:11.621 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:11.629 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:11.691 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:11.700 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:11.709 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:11.717 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:11.726 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:11.734 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:11.743 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:11.752 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:11.760 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:11.770 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:11.780 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:11.789 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:11.796 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:11.804 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:11.813 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:11.821 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:11.829 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:11.891 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:11.900 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:11.911 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:11.922 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:11.931 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:11.939 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:11.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:11.970 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:31:11.978 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:31:11.986 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:11.993 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:12.000 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:12.008 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:12.016 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:12.025 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:12.033 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:12.041 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:12.116 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:12.124 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:12.133 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:12.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:12.150 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:31:12.158 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:31:12.165 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:12.173 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:12.181 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:12.192 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:12.200 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:12.209 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:12.217 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:12.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:12.235 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:12.244 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:12.253 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:12.311 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:12.319 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:12.327 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:12.336 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:12.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-04-26T20:31:12.354 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:12.361 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:12.369 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:12.378 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:12.385 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:12.394 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:12.402 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:12.411 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:12.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:12.428 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:12.436 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:12.445 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:12.502 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:12.512 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:12.521 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:12.532 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:12.540 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:12.549 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:12.558 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:12.569 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:12.577 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:12.586 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:12.595 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:12.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:12.612 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:12.621 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:12.629 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:12.639 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:12.647 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:12.729 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:12.737 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:12.745 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:12.753 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:12.762 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:12.772 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:12.780 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:12.789 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:12.797 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:12.805 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:12.814 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:12.822 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:12.830 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:12.840 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:12.848 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:12.857 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:12.866 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:12.921 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:12.969 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:12.977 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:12.984 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:12.990 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:12.997 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:13.004 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:13.011 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:13.020 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:13.028 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:13.037 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:13.046 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:13.054 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:13.063 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:13.071 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:13.079 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:13.086 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:13.155 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:13.162 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:13.172 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:13.181 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:13.190 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:13.198 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:13.207 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:13.216 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:13.224 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:13.233 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:13.244 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:13.252 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:13.261 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:13.269 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-04-26T20:31:13.278 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:13.287 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:13.296 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:13.349 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:13.357 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:13.365 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:13.373 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:13.382 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:13.389 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:13.395 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:13.402 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:13.409 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:13.417 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:13.424 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:13.431 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:13.438 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:13.446 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:13.453 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:13.460 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:13.468 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:13.527 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:13.534 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:13.543 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:13.550 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:13.558 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:13.565 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:13.571 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:13.578 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:13.586 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:13.593 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:13.600 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:13.607 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:13.614 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:13.622 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:13.631 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:13.640 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:13.649 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:13.724 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:13.733 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:13.742 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:13.751 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:13.760 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:13.768 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:13.777 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:13.785 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:13.793 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:13.802 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:13.810 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:13.819 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:13.828 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:13.846 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:13.854 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:13.863 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:13.936 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:13.943 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:13.950 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:13.960 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:13.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:13.977 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:13.986 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:13.995 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:14.004 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:14.013 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:14.021 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:14.029 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:14.038 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:14.047 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:14.054 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:14.063 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:14.071 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:14.152 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:14.162 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:14.171 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-04-26T20:31:14.180 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-04-26T20:31:14.188 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:14.197 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:14.206 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:14.213 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-04-26T20:31:14.222 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:14.231 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:14.239 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:14.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T20:31:14.257 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:14.265 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:14.273 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:14.282 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:14.291 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:14.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:14.380 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:14.388 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-04-26T20:31:14.396 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:14.404 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:14.414 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:14.422 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:14.430 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:14.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:14.447 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:14.456 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:14.464 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:14.473 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:14.481 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:14.489 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:14.498 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:14.507 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:14.573 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:14.581 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:14.589 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:14.605 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:14.613 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:14.622 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:14.638 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:14.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:14.653 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:14.660 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:14.668 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:14.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:14.681 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:14.689 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:14.697 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:14.762 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:14.772 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:14.781 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:14.790 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:14.799 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:14.807 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:14.815 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:14.823 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:14.831 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:14.876 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:14.884 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:14.891 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:14.898 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:14.905 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:14.912 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:14.919 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:14.927 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:15.001 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:15.008 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:15.015 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:15.022 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:15.029 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:15.036 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:15.043 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:15.050 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:15.057 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:15.063 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:15.070 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:15.077 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:15.084 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:15.091 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:15.100 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:15.108 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:15.117 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:15.183 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:15.192 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:15.201 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:15.210 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:15.218 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:15.226 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:15.237 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:15.246 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:15.255 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-04-26T20:31:15.264 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-04-26T20:31:15.273 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:15.281 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:15.289 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:15.297 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:15.306 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:15.314 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:15.324 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:15.389 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:15.398 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:15.407 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:15.416 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:15.426 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:15.434 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-04-26T20:31:15.447 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-04-26T20:31:15.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:15.464 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:15.473 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:15.481 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-04-26T20:31:15.490 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:15.499 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:15.507 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:15.515 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:15.524 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:15.533 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:15.605 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:15.624 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-04-26T20:31:15.633 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-04-26T20:31:15.642 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:15.651 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:15.660 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:15.668 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:15.678 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:15.687 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:15.695 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:15.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:15.711 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:15.719 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:15.728 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:15.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:15.745 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-04-26T20:31:15.843 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-04-26T20:31:15.853 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:15.861 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:15.870 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:15.878 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:15.888 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:15.895 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:15.904 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:15.913 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:15.921 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:15.930 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:15.938 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:15.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:15.956 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-04-26T20:31:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-04-26T20:31:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:31:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:16.060 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:16.068 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:16.076 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:16.085 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:16.093 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:16.101 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:16.110 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:16.117 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:16.125 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:16.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:16.141 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-04-26T20:31:16.149 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-04-26T20:31:16.157 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:16.165 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:16.174 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:16.182 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:16.191 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:16.259 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:16.269 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-04-26T20:31:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-04-26T20:31:16.331 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:16.339 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:16.347 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:16.357 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:16.364 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:16.370 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:16.377 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:16.384 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:16.470 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:16.477 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:16.484 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:16.491 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:16.498 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-04-26T20:31:16.505 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-04-26T20:31:16.513 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:16.519 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:16.526 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:16.533 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:16.541 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:16.548 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:16.556 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:16.563 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:16.570 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:16.578 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:16.587 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:16.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:16.668 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-04-26T20:31:16.676 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-04-26T20:31:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:31:16.729 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-04-26T20:31:16.738 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:16.746 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:16.755 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:16.763 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:16.773 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:16.782 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:16.791 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-04-26T20:31:16.801 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-04-26T20:31:16.881 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:16.890 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:16.899 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:16.908 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:16.917 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:16.926 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:16.935 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:16.944 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:16.953 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:16.962 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-04-26T20:31:16.997 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-04-26T20:31:17.006 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:17.014 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:17.022 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:17.094 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:17.109 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:17.117 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:17.134 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:17.142 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:17.153 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:17.161 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:17.169 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:17.177 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-04-26T20:31:17.186 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-04-26T20:31:17.195 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:17.203 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:17.212 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:17.220 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:17.229 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:17.238 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:17.312 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:17.320 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:17.329 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:17.338 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:17.349 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:17.357 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:17.366 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:17.374 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:17.382 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:17.391 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-04-26T20:31:17.399 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:17.408 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:17.417 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:17.426 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-04-26T20:31:17.434 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-04-26T20:31:17.443 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:17.451 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:17.528 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:17.536 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:17.544 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:17.553 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:17.560 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:17.568 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T20:31:17.576 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:17.585 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:17.593 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:17.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:17.609 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-04-26T20:31:17.618 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-04-26T20:31:17.626 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:17.634 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:17.642 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:17.650 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:17.659 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:17.726 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:17.734 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:17.744 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-04-26T20:31:17.753 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:17.770 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:17.777 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:17.785 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:17.793 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:17.809 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:17.816 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:17.824 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:17.832 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:17.848 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:17.857 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:17.864 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:17.931 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:17.939 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:17.946 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:17.953 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:17.960 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:17.967 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:17.974 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:17.981 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:17.987 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:17.994 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:18.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:18.009 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:18.016 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:18.023 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:18.030 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:18.038 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:18.046 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:18.106 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:18.115 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:18.123 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:18.132 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:18.140 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:18.148 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:18.155 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:18.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:18.173 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:18.181 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:18.190 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:18.199 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:18.209 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:18.218 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:18.226 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:18.235 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:18.244 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:18.315 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:18.324 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:18.333 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:18.342 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:18.351 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:18.359 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:18.367 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:18.374 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:18.382 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:18.436 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:18.443 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:18.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:18.459 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:18.468 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:18.476 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:18.484 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:18.492 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:18.567 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:18.577 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:18.584 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:18.593 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:18.601 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:18.610 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:18.618 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:18.625 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:18.633 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:18.640 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:18.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:18.658 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:18.669 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-04-26T20:31:18.677 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:18.686 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:18.694 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:18.703 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:18.764 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:18.772 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:18.781 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:18.789 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:18.798 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:18.806 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:18.815 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:18.824 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:18.832 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:18.841 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:18.849 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:18.867 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:18.876 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:18.885 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:18.893 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:18.902 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:19.006 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:19.014 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:19.023 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:19.032 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:19.041 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:19.050 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:19.058 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:19.066 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:19.074 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:19.082 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:19.089 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:19.098 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:19.107 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:19.115 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:19.125 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:19.134 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:19.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:19.220 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:19.229 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:19.238 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:19.246 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:19.254 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:19.262 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:19.271 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:19.279 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:19.287 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:19.297 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:19.305 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:19.313 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:19.322 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:19.330 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:19.339 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:19.347 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:19.355 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:19.414 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:19.422 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:19.430 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:19.437 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:19.445 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:19.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:19.459 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:19.467 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:19.473 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:19.482 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:19.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:19.496 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:19.503 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:19.509 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:19.517 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:19.589 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:19.598 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:19.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:19.616 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:19.624 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-04-26T20:31:19.633 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:19.642 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:19.650 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:19.658 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:19.666 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:19.676 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:19.685 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:19.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:19.711 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:19.720 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:19.729 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:19.789 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:19.797 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:19.803 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:19.810 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:19.820 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:19.829 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:19.837 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:19.845 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:19.853 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:19.861 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:19.870 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:19.878 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:19.887 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:19.895 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:19.903 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:19.912 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:19.920 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:19.980 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:19.989 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:19.997 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:20.005 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:20.013 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:20.022 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:20.030 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:20.038 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:20.047 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:20.055 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:20.065 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:20.079 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:20.087 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:20.096 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:20.104 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:20.113 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:20.122 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:20.189 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:20.197 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:20.215 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:20.224 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:20.233 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:20.242 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:20.250 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:20.259 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:20.267 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:20.276 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:20.286 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:20.296 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:20.304 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:20.313 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:20.323 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:20.331 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:20.339 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:20.397 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:20.404 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:20.413 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:20.423 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:20.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:20.440 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:20.449 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:20.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:20.468 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:20.476 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:20.488 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:20.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:20.505 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:20.514 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-04-26T20:31:20.522 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:20.531 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:20.540 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:20.605 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-04-26T20:31:20.615 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-04-26T20:31:20.624 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:20.632 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:20.641 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:20.650 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T20:31:20.657 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:20.666 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:20.674 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:20.682 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:20.691 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:20.699 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:20.708 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:20.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:20.726 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:31:20.735 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:31:20.743 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:20.815 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:20.824 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:20.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-04-26T20:31:20.841 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:20.849 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:20.858 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:20.866 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:20.872 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:20.879 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:20.886 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:20.893 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:20.900 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:20.907 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:20.913 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:20.920 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:20.927 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:20.934 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:20.997 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:21.005 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:21.014 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:21.022 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:21.032 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:21.041 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:21.050 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:21.059 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:21.068 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:21.078 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:21.086 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:21.094 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:21.103 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:21.112 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:21.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:21.128 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:21.136 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:21.213 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:21.222 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:21.232 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:21.241 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:21.250 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:21.258 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:21.267 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:21.275 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:21.284 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:21.293 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:21.301 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:21.312 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:21.320 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:21.329 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:21.337 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:21.344 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:21.355 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:21.416 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:21.508 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:21.517 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:21.525 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:21.534 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:21.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:21.549 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:21.644 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:21.653 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:21.663 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:21.673 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:21.681 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:21.689 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:21.698 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:21.706 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:21.715 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:21.726 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:21.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:21.743 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:21.751 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-04-26T20:31:21.805 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:21.814 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:21.822 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:21.831 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-04-26T20:31:21.839 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-04-26T20:31:21.848 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:21.856 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:21.865 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:21.872 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:21.881 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:21.890 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:21.898 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:21.917 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T20:31:21.926 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:21.934 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:21.942 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:21.950 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:22.037 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-04-26T20:31:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-04-26T20:31:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:22.063 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:22.071 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:22.079 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:22.087 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:22.094 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:22.102 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:22.111 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T20:31:22.119 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:22.127 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:22.135 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:22.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:22.150 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-04-26T20:31:22.156 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-04-26T20:31:22.163 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:22.223 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:22.232 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:22.241 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:22.250 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:22.258 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:22.266 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:22.274 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T20:31:22.282 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:22.290 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:22.298 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:22.305 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:22.312 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-04-26T20:31:22.319 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-04-26T20:31:22.326 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:22.333 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:22.340 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:22.347 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:22.416 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:22.423 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:22.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:22.441 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T20:31:22.450 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:22.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:22.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:22.475 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:22.484 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-04-26T20:31:22.492 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-04-26T20:31:22.501 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:22.509 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:22.518 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:22.526 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:22.534 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:22.542 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:22.551 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:22.605 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T20:31:22.614 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:22.622 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:22.631 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:22.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:22.648 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-04-26T20:31:22.657 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-04-26T20:31:22.666 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:22.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:22.684 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:22.693 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:22.702 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:22.710 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:22.719 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:22.729 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:22.737 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:22.746 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:22.808 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:22.816 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:22.825 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-04-26T20:31:22.834 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-04-26T20:31:22.843 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:22.851 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:22.859 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:22.867 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:22.876 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:22.884 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:22.892 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:22.900 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:22.909 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:22.922 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:22.929 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:22.938 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:22.946 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-04-26T20:31:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-04-26T20:31:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:23.039 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:23.047 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:23.055 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T20:31:23.063 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:23.071 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:23.082 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:23.092 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:23.100 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:23.108 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:23.117 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:23.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:23.134 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-04-26T20:31:23.142 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-04-26T20:31:23.150 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:23.159 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:23.231 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:23.240 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:23.251 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:23.259 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:23.267 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:23.276 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:23.284 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:23.294 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:23.302 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:23.310 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:23.319 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:31:23.327 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:31:23.336 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:23.344 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:23.353 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:23.361 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-04-26T20:31:23.369 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-04-26T20:31:23.429 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:23.438 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:23.446 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:23.454 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:23.462 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:23.471 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:23.479 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:23.488 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-04-26T20:31:23.496 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-04-26T20:31:23.504 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:23.512 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:23.520 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:23.529 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-04-26T20:31:23.537 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:23.545 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:23.554 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:23.562 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:23.629 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:23.638 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:23.646 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:23.654 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:23.663 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-04-26T20:31:23.672 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-04-26T20:31:23.681 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:23.689 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:23.698 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:23.706 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:23.714 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:23.722 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:23.730 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:23.739 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:23.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:23.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:23.760 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:23.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:23.840 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-04-26T20:31:23.848 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-04-26T20:31:23.855 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:23.863 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:23.872 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:23.881 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:23.890 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:23.899 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:23.907 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:23.916 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:23.925 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:23.934 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:23.942 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:23.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:23.960 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:24.037 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:24.047 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:24.101 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:24.109 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:24.118 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:24.126 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:24.133 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:24.140 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:24.156 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:24.230 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:24.238 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:24.246 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:24.255 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:24.263 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:24.272 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:24.280 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:24.289 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:24.299 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:24.308 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:24.316 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:24.325 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:24.334 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:24.342 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:24.351 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:24.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:24.368 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:24.478 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:24.486 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:24.494 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:24.503 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:24.512 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:24.520 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:24.529 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:24.538 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:24.546 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:24.554 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:24.562 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:24.571 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:24.579 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:24.588 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:24.597 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:24.605 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:24.614 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:24.679 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:24.693 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:24.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:24.710 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:24.720 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:24.746 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:24.754 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:24.764 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:24.772 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:24.780 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:24.789 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:24.807 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:24.815 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:24.824 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:24.888 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-04-26T20:31:24.896 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-04-26T20:31:24.904 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:24.915 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:24.923 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:24.932 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:24.940 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:24.949 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:24.958 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:24.967 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:24.976 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:24.984 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:24.993 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:25.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:25.009 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-04-26T20:31:25.017 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-04-26T20:31:25.026 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:25.110 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:25.118 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:25.125 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:25.141 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:25.149 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:25.157 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:25.166 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:25.176 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:25.184 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:25.194 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:25.202 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-04-26T20:31:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-04-26T20:31:25.220 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:25.228 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:25.236 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:25.245 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T20:31:25.304 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:25.311 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:25.318 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:25.327 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:25.337 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:25.355 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-04-26T20:31:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-04-26T20:31:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:26.812 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:26.821 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:26.829 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:26.838 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:26.846 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:26.855 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:26.864 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:26.872 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:26.881 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:26.889 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:26.897 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:26.905 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:26.915 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:26.923 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:26.932 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:26.941 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:26.950 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:27.015 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:27.022 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:27.033 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:27.042 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:27.050 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:27.058 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:27.067 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:27.076 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:27.086 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:27.096 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:27.103 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:27.112 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:27.120 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:27.127 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:27.135 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:27.142 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:27.150 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:27.245 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:27.261 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:27.272 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:27.280 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:27.297 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:27.304 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:27.313 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:27.321 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:27.329 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-04-26T20:31:27.338 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-04-26T20:31:27.347 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:27.355 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:27.364 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:27.372 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-04-26T20:31:27.381 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:27.387 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:27.462 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:27.469 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T20:31:27.477 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:27.484 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:27.491 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:27.499 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:27.506 INFO:teuthology.orchestra.run.smithi131.stdout: "periodic_async_work": { 2024-04-26T20:31:27.513 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "periodic_async_work", 2024-04-26T20:31:27.520 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:27.527 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:27.535 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:27.543 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:27.552 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:27.560 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:27.569 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:27.577 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Periodically check for async work", 2024-04-26T20:31:27.586 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:27.673 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:27.682 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:27.691 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:27.700 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-04-26T20:31:27.709 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-04-26T20:31:27.721 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:27.729 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:27.738 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:27.746 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:31:27.754 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:27.763 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:27.772 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:27.781 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T20:31:27.790 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:27.799 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:27.807 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:27.816 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:27.864 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-04-26T20:31:27.872 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T20:31:27.881 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:27.890 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:27.899 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:27.908 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:27.916 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:27.925 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:27.935 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:27.944 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T20:31:27.953 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:27.963 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:27.972 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:27.981 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:27.989 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:27.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:28.006 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:28.065 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-04-26T20:31:28.075 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:28.084 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-04-26T20:31:28.108 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-04-26T20:31:28.118 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:31:28.126 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:28.134 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:28.142 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-04-26T20:31:28.151 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:28.160 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:28.168 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:28.177 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:28.186 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:28.194 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:28.202 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:28.250 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:28.258 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-04-26T20:31:28.267 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-04-26T20:31:28.280 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:28.288 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:28.297 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:28.305 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:28.314 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:28.322 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:28.331 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:28.339 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:28.348 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:28.356 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:28.366 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:28.376 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:28.385 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:31:28.394 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:31:28.461 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:28.470 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:28.479 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:28.487 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:31:28.495 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:28.502 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:28.510 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:28.518 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:28.526 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:28.542 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:28.550 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:28.557 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:28.564 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:28.572 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:28.579 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:28.587 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:28.647 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:28.654 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:28.661 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:28.668 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:28.678 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:28.685 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:28.692 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:28.699 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:28.706 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:28.713 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:28.762 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:28.811 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:28.818 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:28.832 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:28.839 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:28.846 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:28.854 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:28.861 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:28.877 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:28.884 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:28.893 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:28.900 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:28.907 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:28.915 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:28.923 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:28.932 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:28.941 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:28.986 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:28.995 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:29.003 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:29.011 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:29.020 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:29.028 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:29.037 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:29.046 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:29.054 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:29.063 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:29.072 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:29.081 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:29.089 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:29.097 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:29.106 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:29.115 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:29.124 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:29.180 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:29.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:29.205 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:29.214 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:29.222 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:29.231 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:29.240 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:29.248 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:29.257 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:29.266 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:29.275 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:29.284 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:29.293 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:29.302 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-04-26T20:31:29.311 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-04-26T20:31:29.320 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:29.418 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:29.428 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:29.436 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:29.445 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:29.453 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:29.461 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:29.470 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-04-26T20:31:29.479 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-04-26T20:31:29.487 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:29.496 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:29.506 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:29.515 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-04-26T20:31:29.557 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:29.565 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:29.575 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:29.583 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:29.591 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:29.600 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:29.608 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:29.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:29.624 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-04-26T20:31:29.632 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-04-26T20:31:29.649 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:29.658 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:29.666 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:29.675 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T20:31:29.684 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:29.693 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:29.702 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:29.771 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:29.783 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:29.794 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:29.803 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:29.811 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:29.820 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:29.828 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:29.836 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:31:29.845 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:29.855 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:29.864 INFO:teuthology.orchestra.run.smithi131.stdout: "gid": 24113, 2024-04-26T20:31:29.871 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "y", 2024-04-26T20:31:29.878 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr_features": 4540701547738038271, 2024-04-26T20:31:29.885 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-04-26T20:31:29.892 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:29.899 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-04-26T20:31:29.906 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:29.949 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:29.957 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:29.964 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:31:29.970 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:31:29.977 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:29.984 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:29.990 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:29.997 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:31:30.005 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:30.012 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:30.019 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:30.026 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T20:31:30.033 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:30.040 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:30.047 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:30.054 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:30.061 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:30.119 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:30.128 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:30.135 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:30.142 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:30.149 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:30.156 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:30.163 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:30.170 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:30.178 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:30.185 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:30.192 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:30.199 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:30.205 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:30.212 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:30.219 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:30.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:30.233 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:30.316 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:30.323 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:30.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:30.339 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:30.349 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:30.357 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:30.365 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:30.373 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:30.381 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:30.389 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:30.397 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:30.406 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:30.414 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:30.422 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:30.430 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:30.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:30.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:30.527 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:30.553 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:30.561 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:30.571 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:30.581 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:30.591 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:30.600 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:30.609 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:30.617 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:30.626 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:30.635 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:30.644 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:30.653 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:30.662 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:30.673 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:30.682 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:30.690 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:30.737 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:30.745 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:30.754 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:30.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:30.771 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:30.780 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:30.789 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:30.798 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:30.806 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:30.815 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:30.825 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:30.833 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:30.843 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:30.860 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:30.869 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:30.877 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:30.936 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:30.945 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-04-26T20:31:30.954 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-04-26T20:31:30.963 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:30.972 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:30.980 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:30.989 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:30.997 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:31.007 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:31.016 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:31.024 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-04-26T20:31:31.033 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:31.041 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:31.050 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:31.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:31.067 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-04-26T20:31:31.077 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-04-26T20:31:31.152 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:31.162 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:31.171 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:31.179 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-04-26T20:31:31.188 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:31.197 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:31.207 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:31.216 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-04-26T20:31:31.224 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:31.233 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:31.240 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:31.247 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:31.255 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-04-26T20:31:31.262 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-04-26T20:31:31.269 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:31.277 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:31.284 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:31.339 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:31.348 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:31.355 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:31.362 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:31.371 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-04-26T20:31:31.380 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:31.388 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:31.395 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:31.402 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:31.410 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-04-26T20:31:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-04-26T20:31:31.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:31.432 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:31.440 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:31.447 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:31.454 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:31.461 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:31.518 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:31.525 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-04-26T20:31:31.532 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:31.539 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:31.546 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:31.553 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:31.560 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-04-26T20:31:31.568 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-04-26T20:31:31.575 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:31.582 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:31.589 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:31.595 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-04-26T20:31:31.602 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:31.608 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:31.615 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:31.622 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-04-26T20:31:31.629 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:31.689 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:31.696 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:31.703 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:31.710 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-04-26T20:31:31.718 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-04-26T20:31:31.726 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:31.735 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:31.744 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:31.751 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:31.760 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:31.769 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:31.778 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:31.787 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-04-26T20:31:31.796 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:31.804 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:31.813 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:31.823 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:31.872 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-04-26T20:31:31.881 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-04-26T20:31:31.890 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:31.899 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:31.908 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:31.917 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:31.925 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:31.934 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:31.944 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:31.953 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T20:31:31.961 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:31.969 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:31.978 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:31.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:31.998 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-04-26T20:31:32.006 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-04-26T20:31:32.015 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:32.106 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:32.114 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:32.125 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:32.133 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:32.141 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:32.150 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-04-26T20:31:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:32.204 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:32.214 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:32.224 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:32.233 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:32.242 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:32.251 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-04-26T20:31:32.260 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:32.268 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:32.360 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:32.369 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-04-26T20:31:32.377 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-04-26T20:31:32.386 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:32.410 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:32.418 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:32.426 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:32.434 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:32.442 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T20:31:32.450 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:32.457 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:32.466 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:32.475 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:32.483 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-04-26T20:31:32.491 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-04-26T20:31:32.553 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:32.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:32.572 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:32.580 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-04-26T20:31:32.589 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:32.597 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:32.606 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:32.615 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T20:31:32.622 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T20:31:32.631 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:32.640 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:32.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:32.658 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-04-26T20:31:32.665 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-04-26T20:31:32.672 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:31:32.680 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:32.687 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:32.728 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:31:32.735 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-04-26T20:31:32.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-04-26T20:31:32.751 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:32.758 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T20:31:32.766 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T20:31:32.773 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:32.781 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:32.788 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:32.795 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-04-26T20:31:32.803 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-04-26T20:31:32.810 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:31:32.818 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:32.826 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:32.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-04-26T20:31:32.840 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:31:32.847 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-04-26T20:31:32.902 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:32.909 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T20:31:32.917 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:32.923 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:32.932 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:32.939 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:32.946 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-04-26T20:31:32.954 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-04-26T20:31:32.963 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:32.969 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:32.976 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:32.983 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T20:31:32.990 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:32.997 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:33.005 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:33.012 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T20:31:33.019 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T20:31:33.082 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:33.088 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:33.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:33.103 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-04-26T20:31:33.110 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-04-26T20:31:33.118 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:31:33.126 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:33.134 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:33.142 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-04-26T20:31:33.151 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-04-26T20:31:33.159 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-04-26T20:31:33.168 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:33.181 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-04-26T20:31:33.190 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T20:31:33.199 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:33.207 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:33.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:33.278 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-04-26T20:31:33.287 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-04-26T20:31:33.296 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:33.304 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:33.313 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:33.321 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-04-26T20:31:33.329 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:33.339 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:33.347 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:33.356 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T20:31:33.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T20:31:33.374 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:33.382 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:33.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:33.400 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-04-26T20:31:33.408 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-04-26T20:31:33.417 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:31:33.478 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:33.488 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:33.497 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:31:33.505 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-04-26T20:31:33.513 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-04-26T20:31:33.521 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:33.531 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T20:31:33.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.505+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26840a3cc0 msgr2=0x7f26840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:31:33.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.505+0000 7f26a4c00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26840a3cc0 0x7f26840a6180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f26a0105a00 tx=0x7f2690005f90 comp rx=0 tx=0).stop 2024-04-26T20:31:33.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.505+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 msgr2=0x7f26a019b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:31:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.505+0000 7f26a4c00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 0x7f26a019b0a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9f0 tx=0x7f268800eeb0 comp rx=0 tx=0).stop 2024-04-26T20:31:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 shutdown_connections 2024-04-26T20:31:33.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f26840a3cc0 0x7f26840a6180 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:31:33.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26a0106560 0x7f26a01a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:31:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a0105ba0 0x7f26a019b0a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:31:33.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 --2- 172.21.15.131:0/2868635479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a01049b0 0x7f26a019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:31:33.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 >> 172.21.15.131:0/2868635479 conn(0x7f26a00fff40 msgr2=0x7f26a0108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:31:33.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 shutdown_connections 2024-04-26T20:31:33.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:29:54.506+0000 7f26a4c00640 1 -- 172.21.15.131:0/2868635479 wait complete. 2024-04-26T20:31:33.720 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T20:31:33.728 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:33.737 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:33.746 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:33.754 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:33.762 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:33.771 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:33.779 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:33.787 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:33.795 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:33.804 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:33.812 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:33.821 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:33.828 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:33.835 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:33.880 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:33.887 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:33.894 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:33.901 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:33.908 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:33.915 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:33.922 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:33.929 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:33.936 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:33.942 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:33.949 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:33.956 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:33.963 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:33.970 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:33.976 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:33.984 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:33.991 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:34.047 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:34.054 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:34.061 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:34.068 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:34.076 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:34.085 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:34.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:34.099 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:34.108 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:34.114 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:34.121 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:34.128 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:34.137 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:34.145 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:34.151 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:34.158 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:34.164 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:34.217 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:34.224 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:34.232 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:34.239 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:34.246 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:34.253 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:34.262 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:34.270 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:34.278 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:34.288 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:34.297 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:34.306 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:34.315 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:34.323 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:34.332 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:34.341 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:34.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:34.398 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:34.415 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:34.425 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:34.435 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:34.444 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:34.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:34.462 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:34.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:34.480 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-04-26T20:31:34.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-04-26T20:31:34.498 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:31:34.508 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:34.517 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:34.530 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:31:34.539 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:34.547 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:34.556 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:34.608 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T20:31:34.617 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:34.631 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:34.642 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:34.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:34.659 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-04-26T20:31:34.669 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-04-26T20:31:34.678 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:34.686 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:34.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:34.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-04-26T20:31:34.711 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:34.719 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:34.727 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:34.735 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-04-26T20:31:34.743 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-04-26T20:31:34.751 INFO:teuthology.orchestra.run.smithi131.stdout: "read", 2024-04-26T20:31:34.814 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap", 2024-04-26T20:31:34.822 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap-read" 2024-04-26T20:31:34.831 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:34.840 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-04-26T20:31:34.848 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:34.857 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:34.865 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:34.873 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:34.884 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-04-26T20:31:34.893 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-04-26T20:31:34.901 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:34.909 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:34.918 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:34.927 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:34.935 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:34.943 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:34.950 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:35.050 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T20:31:35.057 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:35.064 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:35.071 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:35.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:35.085 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:31:35.092 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:31:35.100 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:35.114 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:31:35.127 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:35.134 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:35.141 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:35.148 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T20:31:35.155 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:35.164 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:35.227 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:35.234 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:35.241 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-04-26T20:31:35.248 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-04-26T20:31:35.255 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:35.261 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:35.268 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:35.275 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:31:35.282 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:31:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:35.295 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:35.302 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T20:31:35.308 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T20:31:35.315 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:35.322 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:35.329 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:35.335 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-04-26T20:31:35.403 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-04-26T20:31:35.410 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:31:35.417 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:35.423 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:35.430 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:31:35.437 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:35.443 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:35.450 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:35.457 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T20:31:35.464 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:35.471 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:35.478 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:35.485 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:35.492 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:35.500 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:35.508 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:35.517 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-04-26T20:31:35.563 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:35.573 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:35.581 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:35.590 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-04-26T20:31:35.599 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-04-26T20:31:35.606 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:31:35.614 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:35.622 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:35.631 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-04-26T20:31:35.640 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:35.649 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:35.658 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:35.667 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T20:31:35.676 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:35.686 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:35.696 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:35.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:35.752 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-04-26T20:31:35.761 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-04-26T20:31:35.770 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:35.778 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:35.786 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:35.795 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-04-26T20:31:35.804 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:35.812 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:35.821 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:35.829 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T20:31:35.837 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:35.846 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:35.855 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:35.872 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-04-26T20:31:35.880 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-04-26T20:31:35.888 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:35.941 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:35.949 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:35.958 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-04-26T20:31:35.967 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:35.975 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:35.983 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T20:31:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:36.010 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:36.018 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-04-26T20:31:36.043 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-04-26T20:31:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:36.068 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:36.076 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:36.129 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:36.138 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:36.146 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:36.154 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T20:31:36.162 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T20:31:36.170 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:36.178 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:36.187 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:36.195 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-04-26T20:31:36.204 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-04-26T20:31:36.211 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:36.219 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:36.229 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:36.238 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:31:36.246 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:36.253 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:36.260 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:36.303 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T20:31:36.310 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:36.317 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:36.324 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:36.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:36.338 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-04-26T20:31:36.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T20:31:36.352 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:31:36.359 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:36.366 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:36.372 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-04-26T20:31:36.378 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:36.385 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:36.392 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:36.401 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T20:31:36.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:36.415 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:36.449 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:36.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:36.465 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm_log_destination": { 2024-04-26T20:31:36.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm_log_destination", 2024-04-26T20:31:36.482 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:36.491 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:36.499 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:36.507 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:36.516 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:36.525 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:36.534 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:36.543 INFO:teuthology.orchestra.run.smithi131.stdout: "file", 2024-04-26T20:31:36.552 INFO:teuthology.orchestra.run.smithi131.stdout: "file,syslog", 2024-04-26T20:31:36.561 INFO:teuthology.orchestra.run.smithi131.stdout: "syslog" 2024-04-26T20:31:36.570 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:36.579 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T20:31:36.588 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:36.633 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:36.642 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:36.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:36.660 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-04-26T20:31:36.668 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-04-26T20:31:36.677 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:36.685 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:36.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:36.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:36.710 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:36.719 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:36.727 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:36.735 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T20:31:36.744 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:36.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:36.761 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:36.770 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:36.839 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-04-26T20:31:36.848 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-04-26T20:31:36.856 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:36.864 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:36.873 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:36.881 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:36.890 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:36.899 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:36.907 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:36.917 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T20:31:36.926 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:36.935 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:36.943 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:36.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:36.960 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-04-26T20:31:36.968 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-04-26T20:31:36.976 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:37.030 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:37.040 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:37.049 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:37.057 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:37.066 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:37.075 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:37.083 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T20:31:37.093 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:37.102 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:37.109 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:37.117 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:37.124 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-04-26T20:31:37.131 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-04-26T20:31:37.139 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:37.146 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:37.154 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:37.162 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T20:31:37.198 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:37.207 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:37.224 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:31:37.233 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:37.244 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:37.252 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:37.261 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:37.270 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-04-26T20:31:37.278 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-04-26T20:31:37.287 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:37.295 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:37.304 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:37.312 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T20:31:37.320 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:37.329 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:37.337 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:37.393 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-04-26T20:31:37.402 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:37.411 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:37.419 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:37.427 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:37.435 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-04-26T20:31:37.443 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-04-26T20:31:37.450 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:37.458 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:37.466 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:37.474 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T20:31:37.481 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:37.489 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:37.497 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:37.505 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-04-26T20:31:37.513 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:37.521 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:37.576 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:37.583 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:37.590 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-04-26T20:31:37.598 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-04-26T20:31:37.604 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:37.611 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:37.618 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:37.624 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T20:31:37.632 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:37.638 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:37.645 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:37.652 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:31:37.659 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:37.665 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:37.672 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:37.679 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:37.687 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-04-26T20:31:37.729 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-04-26T20:31:37.737 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:37.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:37.752 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:37.761 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T20:31:37.770 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:37.780 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:37.789 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:37.801 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-04-26T20:31:37.811 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:37.820 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:37.830 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:37.839 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:37.848 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-04-26T20:31:37.857 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-04-26T20:31:37.866 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:37.874 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:37.931 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:37.939 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T20:31:37.948 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:37.956 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:37.965 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:37.973 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-04-26T20:31:37.981 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:37.989 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:37.998 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:38.008 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:38.017 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-04-26T20:31:38.025 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-04-26T20:31:38.033 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:38.041 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:38.049 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:38.058 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T20:31:38.067 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:38.130 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:38.141 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:38.151 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-04-26T20:31:38.159 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:38.167 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:38.175 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:38.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:38.191 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-04-26T20:31:38.199 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-04-26T20:31:38.207 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:38.216 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:38.225 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:38.235 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T20:31:38.242 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:38.251 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:38.260 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:38.268 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-04-26T20:31:38.320 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:38.329 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:38.337 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:38.346 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:38.357 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-04-26T20:31:38.366 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-04-26T20:31:38.375 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:38.383 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:38.393 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:38.402 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T20:31:38.412 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:38.422 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:38.430 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:38.439 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-04-26T20:31:38.448 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:38.458 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:38.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:38.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:38.524 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-04-26T20:31:38.533 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-04-26T20:31:38.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:38.553 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:38.563 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:38.572 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T20:31:38.580 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:38.588 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:38.597 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:38.605 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-04-26T20:31:38.613 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:38.622 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:38.631 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:38.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:38.647 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-04-26T20:31:38.657 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-04-26T20:31:38.709 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:38.719 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:38.730 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:38.740 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T20:31:38.749 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:38.759 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:38.769 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:38.778 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:31:38.788 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:38.851 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:38.861 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:38.870 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:38.879 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-04-26T20:31:38.888 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-04-26T20:31:38.898 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:38.907 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:38.916 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:38.969 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T20:31:38.979 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:38.989 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:39.005 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-04-26T20:31:39.012 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:39.019 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:39.026 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:39.033 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:39.041 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-04-26T20:31:39.048 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-04-26T20:31:39.055 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:39.062 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:39.069 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:39.076 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T20:31:39.083 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:39.090 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:39.149 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:39.157 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:31:39.165 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:39.172 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:39.179 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:39.186 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:39.195 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-04-26T20:31:39.203 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-04-26T20:31:39.211 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:39.220 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:39.228 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:39.237 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T20:31:39.245 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:39.255 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:39.266 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:39.276 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-04-26T20:31:39.285 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:39.344 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:39.357 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:39.367 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:39.375 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_samba": { 2024-04-26T20:31:39.385 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_samba", 2024-04-26T20:31:39.394 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:39.404 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:39.413 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:39.423 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T20:31:39.433 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:39.443 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:39.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:39.461 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Samba/SMB container image", 2024-04-26T20:31:39.470 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:39.479 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:39.488 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:39.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:39.568 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-04-26T20:31:39.577 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-04-26T20:31:39.583 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:39.589 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:39.597 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:39.605 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T20:31:39.614 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:39.622 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:39.631 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:39.646 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-04-26T20:31:39.656 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:39.677 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:39.685 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:39.695 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-04-26T20:31:39.704 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-04-26T20:31:39.713 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:39.813 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:39.829 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:39.847 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:39.900 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:39.909 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:39.918 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:39.928 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T20:31:39.938 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:39.947 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:39.957 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:39.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:39.976 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-04-26T20:31:39.985 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-04-26T20:31:39.995 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:40.004 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:40.013 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:40.022 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:31:40.086 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:40.100 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:40.110 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:40.119 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T20:31:40.128 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:40.137 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:40.146 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:40.165 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-04-26T20:31:40.175 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T20:31:40.184 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:40.194 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:40.203 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:40.212 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-04-26T20:31:40.222 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:40.231 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:40.240 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:40.297 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T20:31:40.305 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:40.314 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:40.322 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:40.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:40.340 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-04-26T20:31:40.350 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-04-26T20:31:40.360 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:40.370 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:40.379 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:40.387 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-04-26T20:31:40.397 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:40.406 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:40.415 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:40.424 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T20:31:40.432 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:40.440 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:40.517 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:40.524 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:40.531 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-04-26T20:31:40.537 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-04-26T20:31:40.544 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:40.550 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:40.557 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:40.562 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-04-26T20:31:40.569 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:40.576 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:40.583 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:40.590 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-04-26T20:31:40.597 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:40.605 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:40.611 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:40.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:40.624 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-04-26T20:31:40.672 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-04-26T20:31:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:40.686 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:40.693 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:40.703 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:40.712 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:40.720 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:40.729 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:40.737 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T20:31:40.745 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:40.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:40.761 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:40.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:40.778 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-04-26T20:31:40.786 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-04-26T20:31:40.795 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:40.804 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:40.884 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:40.893 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:31:40.933 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:40.941 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:40.950 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:40.959 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-04-26T20:31:40.968 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:40.977 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:40.985 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:40.994 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:41.005 INFO:teuthology.orchestra.run.smithi131.stdout: "grafana_dashboards_path": { 2024-04-26T20:31:41.014 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "grafana_dashboards_path", 2024-04-26T20:31:41.022 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:41.032 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:41.040 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:41.050 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T20:31:41.059 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:41.130 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:41.138 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T20:31:41.146 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:41.155 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:41.163 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:41.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-04-26T20:31:41.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-04-26T20:31:41.197 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:41.205 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:41.214 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:41.223 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:31:41.231 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:41.256 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-04-26T20:31:41.311 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:41.323 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:41.331 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:41.340 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:41.349 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-04-26T20:31:41.358 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-04-26T20:31:41.368 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:41.377 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:41.385 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:41.394 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:41.402 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:41.411 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:41.419 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:41.429 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T20:31:41.438 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:41.446 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:41.454 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:41.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:41.509 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-04-26T20:31:41.518 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-04-26T20:31:41.527 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:41.547 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:41.564 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:41.573 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:41.581 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:41.590 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:41.598 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T20:31:41.609 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:41.617 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:41.629 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:41.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:41.645 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:41.654 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:41.706 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:41.715 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:41.722 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:41.729 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:41.738 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:41.746 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:41.754 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:41.762 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:41.771 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:41.778 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:41.786 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:41.795 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:41.803 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:41.812 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:41.820 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:41.830 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:41.838 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:41.903 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:41.911 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:41.918 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-04-26T20:31:41.925 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-04-26T20:31:41.936 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:41.943 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:41.950 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:41.957 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:41.965 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:41.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:41.979 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:41.985 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T20:31:41.992 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:42.000 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:42.006 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:42.013 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:42.020 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:42.083 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:42.090 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:42.097 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:42.104 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:42.112 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:42.120 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:42.128 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:42.136 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:42.145 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T20:31:42.154 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:42.163 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:42.172 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:42.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:42.188 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:42.196 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:42.273 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:42.281 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:42.290 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:42.297 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:42.306 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:42.314 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:42.323 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:42.330 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:42.338 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:42.349 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:42.357 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:42.365 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:42.373 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:42.381 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:42.390 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:42.400 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:42.408 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:42.456 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:42.464 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:42.472 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:42.480 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:42.487 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:42.495 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:42.528 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:42.536 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:42.545 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:42.553 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:42.562 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:42.574 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T20:31:42.582 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T20:31:42.590 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:42.673 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:42.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:42.762 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:42.775 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:42.783 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:42.796 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:42.815 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T20:31:42.824 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:42.832 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:42.840 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:42.849 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:42.857 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T20:31:42.864 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T20:31:42.872 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:42.878 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:42.886 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:42.943 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-04-26T20:31:42.952 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:42.962 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:42.970 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:42.979 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T20:31:42.987 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:42.995 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:43.021 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-04-26T20:31:43.029 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-04-26T20:31:43.037 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:43.045 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:43.053 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:43.061 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:31:43.071 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:43.078 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:43.124 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:43.129 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-04-26T20:31:43.136 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:43.142 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:43.149 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:43.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:43.163 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-04-26T20:31:43.170 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-04-26T20:31:43.176 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:43.186 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:43.193 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:43.199 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:31:43.206 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:43.212 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:43.219 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:43.225 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T20:31:43.232 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:43.290 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:43.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:43.308 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:43.316 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-04-26T20:31:43.325 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-04-26T20:31:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:43.342 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:43.351 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:43.358 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:43.367 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:43.375 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:43.384 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:43.392 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-04-26T20:31:43.400 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:43.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:43.418 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:43.426 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:43.486 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-04-26T20:31:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-04-26T20:31:43.503 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:43.511 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:43.520 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:43.529 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-04-26T20:31:43.537 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:43.546 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:43.555 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:43.563 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-04-26T20:31:43.572 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-04-26T20:31:43.580 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:43.589 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T20:31:43.597 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:43.607 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:43.614 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:43.622 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:43.678 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-04-26T20:31:43.687 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-04-26T20:31:43.696 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:43.704 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:43.713 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:43.721 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-04-26T20:31:43.729 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:43.737 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:43.747 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:43.756 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T20:31:43.765 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:43.773 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:43.782 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:43.790 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:43.799 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-04-26T20:31:43.807 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-04-26T20:31:43.815 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:43.892 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:43.902 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:43.911 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T20:31:43.920 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:43.928 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:43.937 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:43.945 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T20:31:43.957 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:43.966 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:43.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:43.991 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-04-26T20:31:44.000 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-04-26T20:31:44.009 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:44.018 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:44.026 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:44.035 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:44.098 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:44.107 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:44.116 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:44.124 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T20:31:44.131 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:44.139 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:44.147 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:44.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:44.164 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-04-26T20:31:44.173 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-04-26T20:31:44.181 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:44.190 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:44.198 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:44.206 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:44.213 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:44.220 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:44.229 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:44.328 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T20:31:44.337 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:44.345 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:44.353 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:44.362 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:44.370 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-04-26T20:31:44.378 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-04-26T20:31:44.387 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:44.395 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:44.404 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:44.412 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:44.419 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:44.426 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:44.433 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:44.440 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T20:31:44.446 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:44.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:44.536 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:44.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:44.550 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-04-26T20:31:44.557 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-04-26T20:31:44.563 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:44.570 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:44.577 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:44.584 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:44.592 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:44.599 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:44.606 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:44.613 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T20:31:44.620 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:44.626 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:44.633 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:44.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:44.651 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-04-26T20:31:44.708 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-04-26T20:31:44.715 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:44.722 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:44.730 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:44.739 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:44.747 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:44.755 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:44.763 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:44.771 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T20:31:44.780 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:44.788 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:44.803 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:44.811 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:44.820 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-04-26T20:31:44.828 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-04-26T20:31:44.838 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:44.847 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:44.902 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:44.910 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-04-26T20:31:44.918 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:44.927 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:44.935 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:44.944 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-04-26T20:31:44.953 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:44.961 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:44.970 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:44.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:44.987 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-04-26T20:31:44.995 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-04-26T20:31:45.003 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:45.011 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:45.020 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:45.028 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:45.038 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:45.094 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:45.102 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:45.110 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T20:31:45.119 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:45.128 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:45.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:45.152 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-04-26T20:31:45.160 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-04-26T20:31:45.169 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:45.177 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:45.185 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:45.192 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:45.201 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:45.209 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:45.218 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:45.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T20:31:45.283 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:45.292 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:45.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:45.309 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:45.319 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-04-26T20:31:45.327 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-04-26T20:31:45.335 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:45.344 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:45.361 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:45.371 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:45.379 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:45.387 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:45.396 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T20:31:45.405 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:45.413 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:45.421 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:45.466 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:45.474 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-04-26T20:31:45.482 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-04-26T20:31:45.491 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:45.500 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:45.509 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:45.517 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:45.525 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:45.533 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:45.542 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:45.550 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T20:31:45.559 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:45.567 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:45.574 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:45.582 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:45.590 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-04-26T20:31:45.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-04-26T20:31:45.646 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:45.654 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:45.662 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:45.671 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:45.680 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:45.688 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:45.697 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:45.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T20:31:45.710 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:45.718 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:45.725 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:45.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:45.738 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-04-26T20:31:45.745 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-04-26T20:31:45.752 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:45.759 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:45.766 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:45.815 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:45.822 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:45.829 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:45.837 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:45.844 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T20:31:45.851 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:45.858 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:45.865 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:45.872 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:45.881 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:45.888 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:45.895 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:45.902 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-04-26T20:31:45.909 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:45.916 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:45.923 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:45.930 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:45.997 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:46.004 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:46.011 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:46.017 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:46.026 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:46.033 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:46.039 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:46.046 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:46.055 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:46.063 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:46.072 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:46.080 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:46.088 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:46.097 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:46.107 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:46.115 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:46.123 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:46.171 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:46.179 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:46.187 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:46.195 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:46.204 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:46.212 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:46.221 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:46.229 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:46.238 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:46.246 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:46.255 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:46.263 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:46.273 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:46.282 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:46.320 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:46.329 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:46.339 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:46.416 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:46.425 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:46.434 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:46.442 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:46.451 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:46.461 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:46.469 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:46.477 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:46.485 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:46.493 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:46.501 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:46.510 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:46.518 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:46.527 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:46.538 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:46.546 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:46.555 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:46.624 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:46.632 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:46.641 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:46.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:46.659 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:46.668 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:46.678 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:46.687 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:46.698 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:46.706 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:46.714 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:46.722 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:46.730 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:46.738 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:46.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:46.754 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:46.764 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:46.826 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:46.834 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-04-26T20:31:46.841 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-04-26T20:31:46.848 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:46.855 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:46.862 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:46.870 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-04-26T20:31:46.877 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:46.884 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:46.891 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:46.898 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T20:31:46.905 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:46.913 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:46.920 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:46.926 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:46.933 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-04-26T20:31:46.946 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-04-26T20:31:46.984 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:46.992 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:46.999 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:47.006 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-04-26T20:31:47.013 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:47.019 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:47.026 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:47.033 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T20:31:47.039 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:47.046 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:47.052 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:47.060 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:47.066 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:47.074 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:47.080 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:47.088 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-04-26T20:31:47.095 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:47.133 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:47.139 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:47.145 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T20:31:47.152 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T20:31:47.159 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:47.166 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:47.172 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:47.179 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:31:47.185 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:47.192 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:47.198 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:47.204 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:47.213 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:47.220 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:47.226 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:47.232 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:47.239 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T20:31:47.283 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T20:31:47.291 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:47.300 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:47.308 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:47.316 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:47.324 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:47.333 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:47.342 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:47.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:47.363 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:47.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:47.380 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:47.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:47.398 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T20:31:47.406 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T20:31:47.415 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:47.423 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:47.473 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:47.482 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:47.491 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:47.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:47.508 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:47.517 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:47.525 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:47.533 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:47.541 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:47.550 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:47.558 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-04-26T20:31:47.570 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T20:31:47.578 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:47.587 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:47.596 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:47.604 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:47.613 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:47.651 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:47.660 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:47.669 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:47.678 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:47.687 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:47.694 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:47.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:47.710 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T20:31:47.720 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T20:31:47.727 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:47.735 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:47.744 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:47.808 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:47.816 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:47.825 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:47.833 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:47.842 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:47.904 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:47.911 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:47.920 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:47.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:47.942 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T20:31:47.951 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T20:31:47.959 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:47.971 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:47.980 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:47.989 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:47.997 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:48.005 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:48.014 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:48.022 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:48.032 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:48.040 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:48.047 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:48.124 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:48.132 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T20:31:48.139 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T20:31:48.146 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:48.153 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:48.163 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:48.171 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:48.178 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:48.184 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:48.191 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:48.198 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:48.205 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:48.212 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:48.221 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:48.228 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:48.236 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T20:31:48.242 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T20:31:48.303 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:48.310 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:48.318 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:48.325 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:48.332 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:48.340 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:48.346 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:48.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:48.360 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:48.366 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:48.374 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:48.382 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:48.389 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T20:31:48.396 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T20:31:48.403 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:48.410 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:48.417 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:48.464 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:48.472 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:48.479 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:48.486 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:48.494 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:48.501 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:48.507 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:48.514 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:48.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:48.527 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T20:31:48.534 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T20:31:48.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:48.549 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:48.556 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:48.562 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:48.569 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:48.578 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:48.623 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:48.630 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:48.637 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:48.644 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:48.651 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:48.658 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:48.664 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T20:31:48.673 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T20:31:48.681 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:48.690 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:48.698 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:48.707 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:48.716 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:48.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:48.736 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:48.745 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:48.754 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:48.797 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:48.806 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:48.816 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:48.825 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T20:31:48.835 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T20:31:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:48.879 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:48.888 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:48.896 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:48.904 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:48.912 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:48.920 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:48.929 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:48.938 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:48.981 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T20:31:48.991 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T20:31:49.000 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:49.008 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:49.016 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:49.024 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:49.032 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:49.040 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:49.049 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:49.057 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:49.065 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:49.073 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:49.081 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:49.089 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:49.097 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T20:31:49.105 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T20:31:49.123 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:49.173 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:49.202 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:49.211 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:49.219 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:49.227 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:49.235 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:49.244 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:49.252 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:49.260 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:49.268 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:49.277 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:49.285 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T20:31:49.293 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T20:31:49.304 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:49.312 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:49.321 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:49.331 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:31:49.381 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:49.388 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:49.395 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:49.403 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:49.410 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:49.418 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:49.425 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:49.432 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:49.441 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T20:31:49.447 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T20:31:49.460 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:49.467 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:49.475 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:49.482 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:49.490 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:49.503 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:49.512 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:49.564 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:49.573 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:49.583 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:49.592 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:49.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:49.615 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-04-26T20:31:49.624 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-04-26T20:31:49.633 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:49.649 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:49.657 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:49.666 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:49.673 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:49.684 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:49.692 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:49.701 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:49.710 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:49.746 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:49.801 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:49.808 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:49.817 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T20:31:49.826 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T20:31:49.835 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:49.844 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:49.851 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:49.858 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:31:49.865 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:49.872 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:49.879 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:49.886 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:49.894 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:49.900 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:49.907 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:49.914 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:49.936 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T20:31:49.974 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T20:31:49.981 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:49.989 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:49.996 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:50.004 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:50.012 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:50.021 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:50.032 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:50.040 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:50.048 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:50.057 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:50.066 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:50.074 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:50.082 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T20:31:50.090 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T20:31:50.097 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:50.124 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:50.173 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:50.183 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:50.192 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:50.200 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:50.209 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:50.217 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:50.226 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:50.235 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:50.243 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:50.252 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:50.261 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T20:31:50.270 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T20:31:50.278 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:50.287 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:50.296 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:50.304 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:50.338 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:50.374 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:50.382 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:50.391 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:50.399 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:50.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:50.417 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:50.425 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:50.434 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T20:31:50.442 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T20:31:50.451 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:50.460 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:50.468 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:50.477 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:50.485 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:50.494 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:50.503 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:50.529 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:50.563 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:50.573 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:50.580 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:50.589 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:50.597 INFO:teuthology.orchestra.run.smithi131.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T20:31:50.605 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T20:31:50.614 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:50.622 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:50.630 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:50.639 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "[]", 2024-04-26T20:31:50.648 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:50.658 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:50.666 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:50.674 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:50.681 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:50.694 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:50.719 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:50.774 INFO:teuthology.orchestra.run.smithi131.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T20:31:50.783 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T20:31:50.793 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:50.802 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:50.809 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:50.817 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "{}", 2024-04-26T20:31:50.825 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:50.832 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:50.841 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:50.848 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:50.855 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:50.863 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:50.870 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:50.878 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:50.886 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T20:31:50.914 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T20:31:50.965 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:50.973 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:50.982 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:50.989 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:50.996 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:51.004 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:51.011 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:51.018 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:51.026 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:51.033 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:51.040 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:51.047 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:51.054 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T20:31:51.061 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T20:31:51.069 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:51.078 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:51.104 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:51.157 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:51.168 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:51.177 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:51.185 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:51.192 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:51.199 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:51.206 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:51.213 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:51.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:51.229 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T20:31:51.236 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T20:31:51.243 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:51.250 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:51.257 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:51.263 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:51.270 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:51.292 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:51.336 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:51.342 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:51.350 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:51.356 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:51.364 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:51.371 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:51.377 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T20:31:51.384 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T20:31:51.394 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:51.402 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:51.410 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:51.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:51.427 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:51.438 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:51.446 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:51.454 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:51.483 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:51.529 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:51.539 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:51.548 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:51.557 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T20:31:51.566 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T20:31:51.575 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:51.583 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:51.591 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:51.600 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:51.609 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:51.618 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:51.626 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:51.634 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:51.642 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:51.651 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:51.666 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:51.723 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T20:31:51.733 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T20:31:51.741 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:51.760 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:51.768 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:51.777 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:51.785 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:51.794 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:51.802 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:51.811 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:51.820 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:51.830 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:51.839 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:51.850 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T20:31:51.858 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T20:31:51.886 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:51.942 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:51.951 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:51.960 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:51.969 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:51.977 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:51.986 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:51.995 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:52.004 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:52.012 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:52.021 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:52.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:52.038 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T20:31:52.046 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T20:31:52.054 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:52.062 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:52.070 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:52.097 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:52.163 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:52.171 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:52.181 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:52.189 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:52.196 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:52.203 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:52.210 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:52.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:52.224 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T20:31:52.231 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T20:31:52.238 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:52.245 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:52.253 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:52.261 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:52.268 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:52.276 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:52.337 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:52.388 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:52.396 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:52.403 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:52.411 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:52.418 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:52.425 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T20:31:52.433 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T20:31:52.440 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:52.447 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:52.454 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:52.461 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:52.468 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:52.476 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:52.482 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:52.490 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:52.497 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:52.519 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:52.588 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:52.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:52.603 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T20:31:52.611 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T20:31:52.618 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:52.625 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:52.632 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:52.637 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T20:31:52.644 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:52.651 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:52.657 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:52.664 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:52.671 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:52.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:52.688 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:52.695 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:52.701 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T20:31:52.748 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T20:31:52.754 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:52.763 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:52.779 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:52.786 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:31:52.796 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:52.805 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:52.814 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:52.823 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:52.832 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:52.840 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:52.849 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:52.858 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:52.866 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T20:31:52.875 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T20:31:52.886 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:52.912 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:52.959 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:52.973 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-04-26T20:31:52.983 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:52.993 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:53.002 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:53.010 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:53.026 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:53.035 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:53.043 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:53.051 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:53.060 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T20:31:53.068 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T20:31:53.077 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:53.086 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:53.095 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:53.103 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-04-26T20:31:53.111 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:53.162 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:53.170 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:53.178 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:53.186 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:53.194 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:53.202 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:53.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:53.218 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T20:31:53.226 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T20:31:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:53.244 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:53.253 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:53.261 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:53.270 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:53.279 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:53.287 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:53.317 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:53.385 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:53.395 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:53.405 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:53.413 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:53.421 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T20:31:53.430 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T20:31:53.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:53.447 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:53.455 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:53.464 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:31:53.473 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:53.482 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:53.490 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:53.499 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:53.507 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:53.515 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:53.539 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:53.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:53.595 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T20:31:53.602 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T20:31:53.609 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:53.616 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:53.622 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:53.630 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:53.636 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:53.642 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:53.649 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:53.658 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:53.665 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:53.673 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:53.685 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:53.692 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:53.699 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T20:31:53.760 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T20:31:53.798 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:53.805 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:53.814 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:53.824 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:53.835 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:53.856 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:53.865 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:53.874 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:53.882 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:53.891 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:53.899 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:53.907 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T20:31:53.917 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T20:31:53.926 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:53.935 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:53.943 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:53.989 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:53.998 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:54.007 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:54.015 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:54.023 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:54.032 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:54.050 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:54.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:54.068 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T20:31:54.077 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T20:31:54.086 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:54.094 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:54.103 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:54.111 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:31:54.119 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:54.127 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:54.172 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:54.180 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:54.188 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:54.196 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:54.204 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:54.212 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:54.221 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T20:31:54.230 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T20:31:54.238 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:54.246 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:54.255 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:54.264 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:31:54.273 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:54.282 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:54.290 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:54.299 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:54.329 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:54.379 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:54.388 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:54.397 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:54.408 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T20:31:54.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T20:31:54.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:54.434 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:54.442 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:54.451 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:31:54.460 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:54.474 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:54.483 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:54.491 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:54.501 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:54.510 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:54.520 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:54.549 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:54.622 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-04-26T20:31:54.631 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-04-26T20:31:54.643 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:54.651 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:54.660 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:54.669 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:54.678 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:54.686 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:54.694 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:54.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:54.710 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:54.718 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:54.726 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:54.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:54.742 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-04-26T20:31:54.750 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-04-26T20:31:54.780 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:54.836 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:54.844 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:54.852 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:54.860 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:54.869 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:54.877 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:54.895 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:54.903 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:54.912 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:54.920 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:54.929 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-04-26T20:31:54.937 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-04-26T20:31:54.945 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:54.953 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:54.961 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:54.986 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:55.043 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:55.051 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:55.059 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:55.068 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-04-26T20:31:55.075 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:55.081 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:55.089 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:55.098 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:55.105 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-04-26T20:31:55.114 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-04-26T20:31:55.122 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:55.130 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:55.139 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:55.147 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-04-26T20:31:55.156 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:55.164 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:55.173 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:55.237 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:55.245 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:55.254 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:55.262 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:55.271 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:55.279 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-04-26T20:31:55.287 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-04-26T20:31:55.293 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:55.301 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:55.309 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:55.318 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:55.337 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:55.346 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:55.355 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:55.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:55.397 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:55.459 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:55.472 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:55.480 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:55.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:55.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:55.516 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:55.524 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:55.538 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:55.547 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:55.555 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:55.564 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:55.573 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:55.582 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:55.590 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:55.598 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:55.629 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:55.675 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:55.683 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:55.693 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:55.701 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:55.710 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:55.719 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:55.727 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:55.736 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:55.744 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:55.753 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:55.762 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:55.771 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:55.779 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:55.788 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:55.797 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:55.806 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:55.833 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:55.880 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:55.888 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:55.896 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:55.905 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:55.913 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:55.922 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:55.931 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:55.938 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:55.947 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:55.956 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:55.964 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:55.973 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:55.981 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:55.989 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:55.997 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:56.006 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:56.032 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:56.080 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:56.088 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:56.096 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:56.103 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:56.111 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:56.120 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:56.128 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:56.136 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:56.144 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:56.152 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:56.161 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:56.167 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:56.173 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:56.180 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:56.188 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:56.194 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:56.221 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:56.262 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:56.285 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:56.294 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:56.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:56.308 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-04-26T20:31:56.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-04-26T20:31:56.323 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:56.333 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:56.340 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:56.347 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:56.355 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:56.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:56.368 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:56.374 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-04-26T20:31:56.381 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:56.390 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:56.420 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:56.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:56.474 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-04-26T20:31:56.484 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T20:31:56.493 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:56.502 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:56.512 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:56.521 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:56.530 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:56.538 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:56.547 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:56.556 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:56.565 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:56.574 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:56.583 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:56.592 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:56.600 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:31:56.609 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:31:56.662 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:56.671 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:56.679 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:56.687 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-04-26T20:31:56.695 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:56.705 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:56.713 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:56.722 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:56.730 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:56.738 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:56.747 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:56.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:56.765 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:31:56.773 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:31:56.782 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:56.791 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:56.820 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:56.873 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-04-26T20:31:56.882 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:56.891 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:56.902 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:56.910 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:56.919 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:56.928 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:56.937 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:56.946 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:56.955 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-04-26T20:31:56.964 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-04-26T20:31:56.973 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:56.982 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:56.990 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:56.999 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:57.008 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:57.067 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:57.124 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:57.132 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:57.141 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:57.151 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:57.159 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:57.169 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:57.178 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-04-26T20:31:57.187 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-04-26T20:31:57.195 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:57.202 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:57.211 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:57.219 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-04-26T20:31:57.228 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:57.238 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:57.246 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:57.255 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:57.283 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:57.361 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:57.369 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:57.378 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:57.385 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-04-26T20:31:57.393 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-04-26T20:31:57.401 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:57.410 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:57.419 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:57.427 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-04-26T20:31:57.435 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:57.444 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:57.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:57.461 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:57.469 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-04-26T20:31:57.478 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:57.486 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:57.513 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:57.557 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:57.566 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:57.574 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:57.582 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-04-26T20:31:57.591 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-04-26T20:31:57.599 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:31:57.608 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:57.627 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-04-26T20:31:57.634 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-04-26T20:31:57.643 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-04-26T20:31:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:57.658 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:57.669 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:57.684 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:57.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:57.777 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-04-26T20:31:57.784 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-04-26T20:31:57.791 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:57.797 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:57.805 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:31:57.812 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:57.821 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:57.830 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:57.838 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:57.845 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:57.853 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:57.861 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:57.868 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:57.874 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:57.883 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:31:57.892 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:57.915 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:31:57.963 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-04-26T20:31:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:31:57.978 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:31:57.986 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:31:57.994 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-04-26T20:31:58.003 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-04-26T20:31:58.010 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:58.018 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:58.028 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:58.049 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:58.058 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:58.067 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:58.075 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:58.083 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-04-26T20:31:58.091 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:58.099 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:58.106 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:58.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:58.185 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:31:58.194 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:31:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:58.211 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:58.219 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:58.228 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:31:58.236 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:58.245 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:58.253 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:58.262 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:58.270 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:58.279 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:58.287 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:58.296 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:58.304 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:58.331 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:58.377 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:58.386 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:58.395 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:58.403 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:58.411 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:58.419 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:31:58.428 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:31:58.437 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:58.445 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:58.454 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:58.462 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:58.471 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:58.479 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:58.487 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:58.495 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:58.504 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:58.526 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:58.605 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:58.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:58.622 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:31:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:31:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:31:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:58.685 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:58.694 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:31:58.702 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:31:58.710 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:31:58.719 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:31:58.727 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:31:58.737 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:31:58.766 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:31:58.812 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:31:58.830 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:58.839 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:58.847 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:58.854 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:58.862 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:58.871 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:31:58.879 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:31:58.887 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:58.895 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:58.904 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:58.913 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:31:58.921 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:58.932 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:58.941 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:31:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:59.012 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:59.018 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:59.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:59.035 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-04-26T20:31:59.045 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-04-26T20:31:59.054 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:59.062 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:59.070 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:59.077 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-04-26T20:31:59.086 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:59.094 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:59.102 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:59.109 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T20:31:59.118 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:59.126 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:59.135 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:59.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:59.185 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-04-26T20:31:59.194 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-04-26T20:31:59.203 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:31:59.211 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:59.220 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:59.229 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-04-26T20:31:59.237 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:59.246 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:59.255 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:59.264 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T20:31:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-04-26T20:31:59.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-04-26T20:31:59.343 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:59.386 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:59.394 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:59.403 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-04-26T20:31:59.411 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:59.420 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:59.428 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:59.436 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-04-26T20:31:59.445 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:59.454 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:59.462 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:59.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:59.480 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-04-26T20:31:59.488 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-04-26T20:31:59.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:59.505 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:59.514 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:59.521 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-04-26T20:31:59.570 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:59.579 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:59.588 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:59.597 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T20:31:59.605 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:59.614 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:59.623 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:59.631 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:59.640 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-04-26T20:31:59.647 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-04-26T20:31:59.655 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:31:59.663 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:59.672 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:59.681 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:31:59.711 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:59.718 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:59.736 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:59.796 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T20:31:59.805 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:59.812 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:31:59.818 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:31:59.825 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:31:59.832 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:31:59.839 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:31:59.848 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:31:59.855 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:31:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:31:59.872 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:31:59.881 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:31:59.889 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:31:59.898 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:31:59.906 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T20:31:59.914 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:31:59.941 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:00.006 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:00.016 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:00.026 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-04-26T20:32:00.035 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-04-26T20:32:00.043 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:32:00.051 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:00.060 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:00.068 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-04-26T20:32:00.077 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:00.086 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:00.095 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:00.104 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T20:32:00.113 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:00.122 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:00.130 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:00.139 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:00.167 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:00.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:00.235 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:00.244 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-04-26T20:32:00.253 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:00.262 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:00.271 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:00.280 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:00.290 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:00.299 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:00.308 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:00.317 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:00.325 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:00.333 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:00.342 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:00.350 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:00.359 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:00.389 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:00.483 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:00.492 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:00.500 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:00.508 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:00.517 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:00.526 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:00.535 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:00.543 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:00.552 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:00.561 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:00.569 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:00.578 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:00.586 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:00.595 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:00.603 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:00.611 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:00.635 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:00.692 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:00.701 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:00.710 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:00.716 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:00.723 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:00.730 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:00.737 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:00.744 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:00.751 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:00.759 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:00.766 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:00.773 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:00.780 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:00.787 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:00.795 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:00.801 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:00.823 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:00.885 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:00.892 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:00.899 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:00.906 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:00.913 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:00.920 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:00.926 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:00.934 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:00.941 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:00.948 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:00.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:00.962 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:00.969 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:00.976 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:00.982 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:00.990 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:01.012 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:01.068 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:01.076 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:01.082 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:01.089 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:01.096 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:01.104 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:01.111 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:01.118 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:01.125 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-04-26T20:32:01.132 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-04-26T20:32:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:01.152 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:01.158 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-04-26T20:32:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:01.172 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:01.202 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:01.294 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:01.301 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:01.308 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:01.314 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:01.323 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:01.331 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-04-26T20:32:01.340 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-04-26T20:32:01.348 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:01.357 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:01.365 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:01.376 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-04-26T20:32:01.384 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:01.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:01.402 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:01.410 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:01.419 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:01.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:01.485 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:01.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:01.504 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:32:01.513 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:32:01.521 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:01.531 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:01.540 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:01.548 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:32:01.557 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:01.566 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:01.575 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:01.583 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:01.592 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:01.601 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:01.610 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:01.618 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:01.649 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:01.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:01.718 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:01.726 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-04-26T20:32:01.735 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-04-26T20:32:01.744 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-04-26T20:32:01.752 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:01.761 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-04-26T20:32:01.768 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-04-26T20:32:01.777 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:01.785 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:01.793 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:01.801 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-04-26T20:32:01.809 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:01.817 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:01.826 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:01.834 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T20:32:01.868 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:01.917 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:01.925 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:01.933 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:01.941 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-04-26T20:32:01.949 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-04-26T20:32:01.957 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:01.966 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:01.974 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:01.983 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-04-26T20:32:01.991 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:01.999 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:02.008 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:02.017 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T20:32:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:02.129 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-04-26T20:32:02.138 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-04-26T20:32:02.147 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:02.155 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:02.164 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:02.173 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:02.181 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:02.187 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:02.194 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:02.201 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-04-26T20:32:02.208 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:02.214 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:02.222 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:02.229 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:02.236 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:32:02.243 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:32:02.272 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:32:02.338 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:02.345 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:02.352 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-04-26T20:32:02.359 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-04-26T20:32:02.368 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:02.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:02.382 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T20:32:02.390 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:02.396 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:02.403 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:02.410 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:02.416 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:02.423 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:02.431 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:02.437 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:02.444 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:02.469 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:02.526 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:02.534 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:02.541 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:02.551 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:02.558 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:02.565 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:02.571 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:02.579 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:02.586 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:02.595 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:02.604 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:02.614 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:02.622 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:02.632 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:02.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:02.650 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:02.683 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:02.718 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:02.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:02.751 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:02.761 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:02.770 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:02.780 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:02.791 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:02.800 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:02.809 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:02.818 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:02.826 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:02.836 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:02.845 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:02.854 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:02.883 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:02.968 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:02.976 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:02.985 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:02.993 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:03.003 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:03.012 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:03.021 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:03.029 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:03.038 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:03.047 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:03.055 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:03.063 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:03.072 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:03.080 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:03.089 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:03.098 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:03.173 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:03.575 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:03.597 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:03.611 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:03.622 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:03.645 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:03.659 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:03.675 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:03.692 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:03.707 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:03.724 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:03.731 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:03.744 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:03.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:03.758 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-04-26T20:32:03.765 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-04-26T20:32:03.792 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:03.799 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:03.888 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:03.895 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:03.902 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:03.908 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:03.915 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:03.922 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-04-26T20:32:03.929 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:03.936 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:03.943 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:03.950 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:03.957 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-04-26T20:32:03.966 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-04-26T20:32:03.973 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:03.982 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:03.990 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:04.022 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-04-26T20:32:04.029 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:04.084 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:04.092 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:04.101 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-04-26T20:32:04.109 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:04.118 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:04.126 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:04.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:04.144 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-04-26T20:32:04.152 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-04-26T20:32:04.160 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:04.168 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:04.177 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:04.185 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-04-26T20:32:04.193 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:04.202 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:04.209 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:04.239 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T20:32:04.293 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:04.302 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:04.310 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:04.318 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:04.325 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-04-26T20:32:04.333 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-04-26T20:32:04.342 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:04.349 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:04.357 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:04.367 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:32:04.375 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:32:04.383 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-04-26T20:32:04.391 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:04.399 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T20:32:04.406 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:04.436 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:04.445 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:04.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:04.507 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-04-26T20:32:04.515 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-04-26T20:32:04.523 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:04.547 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:04.556 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:04.563 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:04.572 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:04.581 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-04-26T20:32:04.589 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:04.596 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:04.604 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:04.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:04.640 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-04-26T20:32:04.647 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-04-26T20:32:04.700 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:04.709 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:04.717 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:04.726 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-04-26T20:32:04.734 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:04.744 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:04.753 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:04.762 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T20:32:04.770 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:04.780 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:04.788 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:04.797 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:04.804 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:04.810 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:04.817 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:04.837 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-04-26T20:32:04.844 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:04.907 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:04.915 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:04.922 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:04.934 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:04.941 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:04.948 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:04.955 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:04.962 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:04.969 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:04.976 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:04.983 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:04.990 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:04.997 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:05.004 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:05.012 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:05.035 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:05.042 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:05.109 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:05.115 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:05.122 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:05.128 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:05.135 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:05.183 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:05.189 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:05.198 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:05.206 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:05.214 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:05.235 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:05.277 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:05.359 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:05.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:05.378 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:05.391 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:05.399 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:05.408 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:05.416 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:05.425 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:05.434 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:05.443 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:05.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:05.462 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:05.470 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:05.480 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:05.513 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:05.521 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:05.606 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:05.626 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:05.635 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:05.643 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:05.652 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:05.662 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:05.670 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:05.679 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:05.688 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:05.697 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:05.705 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:05.714 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:05.723 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:05.733 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:05.742 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:05.778 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:05.786 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:05.858 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:05.871 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:05.879 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:05.887 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:05.894 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:05.901 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:05.909 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:05.917 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-04-26T20:32:05.924 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:05.934 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:05.942 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:05.950 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:05.958 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:05.966 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:05.973 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:06.000 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:06.007 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:06.061 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:06.069 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:06.081 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:06.087 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:06.094 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:06.101 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:06.108 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:06.118 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:06.125 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:06.133 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:06.140 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:06.148 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:06.154 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:06.161 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:06.168 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:06.274 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:06.280 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:06.290 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:06.297 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:06.304 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:06.310 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:06.316 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:06.322 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:06.328 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:06.334 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:06.341 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:06.348 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:06.354 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:06.361 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:06.368 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:06.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:06.402 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:06.450 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:06.457 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:06.464 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:06.471 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:06.480 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:06.487 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:06.494 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:06.501 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:06.512 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:06.518 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:06.525 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:06.533 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:06.539 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:06.546 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:06.552 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:06.575 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:06.584 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:06.657 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:06.666 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:06.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:06.683 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:06.691 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:06.699 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:06.708 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:06.716 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:06.725 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:06.733 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:06.741 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:06.750 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:06.759 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:06.766 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:06.773 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:06.799 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:06.805 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-04-26T20:32:06.862 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:06.872 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:06.880 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:06.888 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-04-26T20:32:06.898 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-04-26T20:32:06.906 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:06.915 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:06.923 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:06.931 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-04-26T20:32:06.939 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:06.946 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:06.954 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:06.961 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T20:32:06.970 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:06.977 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:07.014 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:07.022 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:07.078 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-04-26T20:32:07.107 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-04-26T20:32:07.116 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:07.124 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:07.131 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:07.138 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:32:07.145 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-04-26T20:32:07.152 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:07.161 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:07.168 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T20:32:07.175 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:07.182 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:07.190 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:07.197 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:07.204 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:07.226 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:07.233 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:07.290 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-04-26T20:32:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-04-26T20:32:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-04-26T20:32:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-04-26T20:32:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-04-26T20:32:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T20:32:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:08.520 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:08.582 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:08.591 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:08.631 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:08.684 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:08.691 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:08.698 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:08.707 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:08.714 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:08.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:08.727 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:08.734 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:08.741 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:08.748 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:08.759 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:08.766 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:08.773 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:08.780 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:08.807 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:08.816 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:08.887 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:08.894 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:08.901 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:08.910 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:08.919 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:08.928 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:08.937 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:08.946 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:08.955 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:08.964 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:08.972 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:08.981 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:08.988 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:08.996 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:09.005 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:09.026 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:09.034 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:09.085 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:09.094 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:09.105 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:09.114 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:09.122 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:09.130 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:09.139 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:09.147 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:09.155 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:09.165 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:09.173 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:09.182 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:09.190 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:09.197 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:09.206 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:09.214 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:09.276 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:09.355 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:09.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:09.382 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:09.391 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:09.401 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:09.413 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-04-26T20:32:09.422 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-04-26T20:32:09.430 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:09.438 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:09.447 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:09.455 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:09.464 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:09.473 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:09.482 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:09.491 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-04-26T20:32:09.513 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T20:32:09.521 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:09.574 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:09.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:09.588 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-04-26T20:32:09.596 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-04-26T20:32:09.603 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:09.611 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:09.617 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:09.624 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-04-26T20:32:09.631 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:09.639 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:09.645 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-04-26T20:32:09.657 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:09.664 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:09.672 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:09.695 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:09.702 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-04-26T20:32:09.774 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-04-26T20:32:09.782 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:09.790 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:09.797 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:09.805 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-04-26T20:32:09.811 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:09.821 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:09.828 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:09.836 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T20:32:09.842 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:09.848 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:09.855 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:09.862 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:09.870 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-04-26T20:32:09.878 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-04-26T20:32:09.933 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:09.940 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:10.008 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:10.018 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:10.025 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:10.033 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:10.041 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:10.048 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-04-26T20:32:10.056 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:10.062 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:10.069 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:10.076 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:10.082 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-04-26T20:32:10.090 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-04-26T20:32:10.098 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:10.109 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:10.117 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:10.150 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-04-26T20:32:10.158 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:10.245 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:10.267 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:10.275 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T20:32:10.286 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T20:32:10.294 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:10.303 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:10.311 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:10.319 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:10.327 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:10.335 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:10.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-04-26T20:32:10.352 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:10.361 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:10.369 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:10.378 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:10.403 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:10.412 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:10.475 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:10.485 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:10.493 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:10.502 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:10.511 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:10.521 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:10.530 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:10.538 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:10.547 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:10.556 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:10.564 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:10.572 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:10.580 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:10.589 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:10.597 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:10.605 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:10.662 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:10.730 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:10.739 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:10.747 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:10.756 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:10.764 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:10.772 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:10.782 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:10.791 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:10.799 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:10.807 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:10.815 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:10.823 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:10.831 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:10.840 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:10.866 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:10.874 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:10.930 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:10.937 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:10.945 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:11.007 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:11.019 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:11.028 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:11.037 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:11.046 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:11.054 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:11.080 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:11.087 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:11.174 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:11.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:11.189 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:11.199 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:11.205 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:11.212 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:11.219 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:11.227 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:11.259 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:11.265 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:11.273 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:11.279 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:11.286 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:11.293 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:11.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:11.327 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:11.333 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:11.422 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:11.429 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:11.436 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-04-26T20:32:11.443 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:11.453 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:11.461 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:11.468 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:11.475 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:11.482 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:11.489 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:11.496 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:11.502 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:11.509 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:11.517 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:11.524 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:11.548 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:11.555 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:11.610 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:11.619 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:11.629 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:11.637 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:11.646 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:11.655 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:11.663 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:11.672 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:11.681 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:11.692 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:11.702 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:11.712 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:11.722 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:11.731 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:11.740 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:11.749 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:11.807 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:11.867 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:11.876 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:11.886 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:11.894 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:11.903 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:11.911 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:11.920 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:11.928 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:11.936 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:11.944 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:11.953 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:11.961 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:11.976 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:11.984 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:11.994 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:12.020 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:12.029 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:12.079 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:12.089 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:12.098 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:12.107 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:12.115 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:12.124 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:12.134 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:12.144 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:12.153 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:12.161 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:12.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:12.179 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:12.187 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:12.196 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:12.204 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:12.233 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:12.243 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:12.301 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:12.310 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:12.319 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:12.327 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:12.336 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:12.347 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:12.355 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:12.363 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:12.372 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:12.381 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:12.389 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:12.397 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-04-26T20:32:12.406 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:12.415 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:12.423 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:12.479 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:12.488 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:12.555 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:12.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:12.573 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:12.581 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:12.590 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:12.599 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:12.607 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:12.615 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:12.623 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:12.631 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:12.640 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:12.648 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:12.657 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:12.665 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:12.674 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:12.701 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:12.709 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:12.761 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:12.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:12.777 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:12.786 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:12.793 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:12.800 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:12.815 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:12.822 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:12.829 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:12.836 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:12.842 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:12.849 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:12.856 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:12.862 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:12.890 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:12.896 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:12.967 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:12.975 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:12.983 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:12.990 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:12.997 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:13.004 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:13.011 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:13.018 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:13.025 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:13.032 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:13.040 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:13.047 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:13.054 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:13.061 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:13.068 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:13.128 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:13.135 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:13.207 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:13.221 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:13.229 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:13.237 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:13.245 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:13.254 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:13.262 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:13.272 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:13.281 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:13.289 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:13.298 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:13.306 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:13.314 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:13.323 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:13.330 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:13.339 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:13.363 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:13.449 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:13.455 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:13.464 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:13.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-04-26T20:32:13.484 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:13.492 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:13.501 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:13.510 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-04-26T20:32:13.518 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-04-26T20:32:13.526 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:13.538 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:13.546 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:13.555 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:13.564 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:13.572 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:13.599 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:13.652 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T20:32:13.760 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:13.804 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:13.821 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:13.843 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:13.855 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:13.867 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:13.879 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:13.890 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:13.902 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:13.915 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:13.958 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:13.972 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:14.009 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:14.095 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:14.105 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:14.166 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:14.205 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:14.214 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:14.223 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:14.234 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:14.245 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:14.255 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:14.266 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:14.277 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:14.287 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:14.296 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:14.304 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:14.336 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:14.346 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:14.403 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:14.460 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:14.471 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:14.539 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:14.600 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:14.622 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:14.642 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:14.652 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:14.661 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:14.669 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:14.677 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:14.686 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:14.694 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:14.743 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:14.752 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:14.809 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:14.864 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:14.873 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:14.928 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:14.979 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:14.988 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:14.997 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:15.005 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:15.013 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:15.022 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:15.031 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:15.041 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:15.049 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:15.058 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:15.064 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:15.110 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:15.120 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:15.177 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:15.236 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:15.247 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:15.310 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:15.345 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:15.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:15.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:15.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:15.382 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:15.392 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:15.402 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-04-26T20:32:15.411 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-04-26T20:32:15.421 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:15.430 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:15.440 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:15.476 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:15.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:15.551 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:15.583 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:15.595 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-04-26T20:32:15.651 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-04-26T20:32:15.707 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-04-26T20:32:15.719 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:15.731 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-04-26T20:32:15.743 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:15.755 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:15.766 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:15.778 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:15.789 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:15.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:15.811 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:15.822 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-04-26T20:32:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:15.865 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:15.916 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:15.947 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:15.957 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:16.017 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:16.079 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:16.091 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:16.101 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:16.111 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:16.122 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:16.132 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:16.142 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:16.152 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:16.164 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:16.174 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:16.192 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:16.222 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:16.233 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:16.376 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:16.437 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:16.466 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:16.478 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:16.489 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:16.498 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:16.508 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:16.518 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:16.526 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:16.536 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:16.565 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:16.575 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:16.767 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:16.833 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:16.845 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:16.857 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:16.865 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:16.875 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:16.884 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:16.894 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:16.903 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:16.915 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:16.924 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:16.940 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:16.977 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:17.033 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:17.088 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:17.099 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:17.159 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:17.223 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:17.234 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:17.244 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:17.265 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:17.278 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:17.288 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:17.298 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:17.316 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:17.329 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:17.343 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:17.389 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:17.399 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:17.434 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:17.493 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:17.502 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:17.559 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:17.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:17.622 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:17.631 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-04-26T20:32:17.644 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:17.654 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:17.664 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:17.674 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:17.685 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:17.695 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:17.708 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:17.720 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:17.776 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:17.846 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:17.899 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:17.911 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:17.969 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:18.028 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:18.043 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:18.054 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:18.066 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:18.110 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:18.121 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:18.132 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:18.152 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:18.196 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:18.210 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:18.269 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:18.329 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:18.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:18.408 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:18.453 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:18.461 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:18.471 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:18.481 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:18.490 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:18.499 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:18.510 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:18.543 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:18.553 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:18.588 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:18.598 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:18.633 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:18.684 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:18.694 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:18.767 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:18.828 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:18.838 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:18.848 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:18.860 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:18.870 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:18.878 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:18.888 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:18.900 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:18.910 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:18.919 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:18.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:18.969 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:18.980 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:19.026 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:19.092 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:19.160 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:19.225 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:19.278 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:19.299 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:19.323 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:19.336 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:19.349 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:19.362 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:19.387 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:19.418 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:19.427 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:19.447 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:19.456 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:19.508 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-04-26T20:32:19.520 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:19.612 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:19.623 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:19.683 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:19.736 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:19.748 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:19.773 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:19.794 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:19.804 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:19.813 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:19.828 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:19.839 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:19.854 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:19.866 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:19.920 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:19.931 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:19.989 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:20.042 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:20.058 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:20.119 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:20.191 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:20.201 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:20.212 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:20.224 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:20.240 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:20.266 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:20.280 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:20.291 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:20.304 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:20.319 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:20.358 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:20.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:20.423 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:20.430 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:20.490 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:20.567 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:20.662 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:20.670 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:20.678 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:20.686 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:20.693 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:20.701 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:20.709 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:20.722 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:20.730 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:20.742 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:20.749 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:20.845 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:20.852 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:20.912 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:20.953 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:20.963 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:21.025 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:21.081 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:21.102 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:21.112 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:21.121 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:21.130 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:21.140 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:21.149 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:21.157 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:21.166 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:21.177 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:21.187 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:21.240 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:21.250 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:21.315 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:21.375 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:21.387 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:21.559 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:32:21.570 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:32:21.581 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:32:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:32:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:21.645 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:21.657 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:21.668 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:21.680 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:21.722 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:21.739 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:21.811 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:21.872 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-04-26T20:32:21.881 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-04-26T20:32:21.941 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:32:21.990 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:21.999 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:22.008 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-04-26T20:32:22.017 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-04-26T20:32:22.025 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:22.033 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:22.042 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-04-26T20:32:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T20:32:22.059 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:22.066 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:22.074 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:22.112 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:22.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:22.192 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:22.244 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-04-26T20:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:22.316 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:22.362 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:22.368 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-04-26T20:32:22.377 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-04-26T20:32:22.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:22.393 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:22.401 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:22.409 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:22.417 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:22.424 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:22.431 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:22.438 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T20:32:22.453 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:22.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:22.532 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:22.583 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:22.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-04-26T20:32:22.649 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-04-26T20:32:22.694 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:22.700 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:22.707 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:22.714 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:22.733 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:22.742 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:22.751 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:22.777 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:22.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:22.808 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:22.848 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:22.855 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:22.974 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:23.036 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:23.045 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:23.053 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:23.063 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:23.072 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:23.081 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:23.089 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:23.097 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:23.107 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:23.140 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:23.149 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:23.199 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:23.235 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:23.245 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:23.298 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:23.354 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:23.362 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:23.370 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:23.378 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:23.387 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:23.396 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:23.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:23.413 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:23.421 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:23.430 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:23.438 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:23.490 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:23.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:23.532 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:23.584 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:23.593 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:23.649 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:23.702 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:23.711 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:23.719 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:23.729 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:23.739 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:23.759 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:23.768 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:23.776 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:23.785 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:23.793 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:23.801 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:23.828 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:23.834 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:23.891 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:23.940 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:23.950 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:24.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:24.100 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:24.107 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:24.116 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:24.123 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:24.131 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:24.139 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:24.155 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:24.228 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:24.234 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:24.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:24.313 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-04-26T20:32:24.369 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-04-26T20:32:24.423 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:24.477 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:24.484 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:24.492 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-04-26T20:32:24.499 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:24.506 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:24.513 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:24.519 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-04-26T20:32:24.526 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:24.534 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:24.541 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:24.548 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:24.590 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:32:24.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:32:24.654 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:32:24.661 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:24.716 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:24.782 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:32:24.826 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:24.834 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:24.843 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-04-26T20:32:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:24.863 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:24.869 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:24.877 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:24.884 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:24.891 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:24.898 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:24.936 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-04-26T20:32:24.946 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:24.981 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:25.029 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:25.038 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-04-26T20:32:25.098 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-04-26T20:32:25.107 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:25.191 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:25.197 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:25.205 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:25.213 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:25.221 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:25.229 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:25.237 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:25.246 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:25.255 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:25.264 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:25.319 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:25.328 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-04-26T20:32:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-04-26T20:32:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:25.561 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T20:32:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T20:32:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:26.817 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:26.825 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:26.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:26.882 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:26.909 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:26.937 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:26.984 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:26.992 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:27.001 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:27.010 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:27.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:27.033 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:27.042 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:27.050 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:27.059 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:27.068 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:27.076 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:27.102 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-04-26T20:32:27.111 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-04-26T20:32:27.159 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:27.211 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:27.252 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:27.310 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:27.367 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:27.376 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:27.385 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:27.394 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:27.403 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:27.412 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:27.420 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:27.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:27.439 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T20:32:27.447 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T20:32:27.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:27.477 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:27.486 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:27.550 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-04-26T20:32:27.604 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:27.629 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:27.638 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:27.697 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:27.706 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:27.714 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:27.723 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:27.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:27.745 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-04-26T20:32:27.754 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-04-26T20:32:27.762 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:32:27.770 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:27.779 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:27.788 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-04-26T20:32:27.810 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:27.818 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:27.861 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:27.869 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:27.926 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:27.952 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:28.017 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:28.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:28.033 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:32:28.042 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:32:28.050 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:28.059 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:28.067 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:28.076 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-04-26T20:32:28.083 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:28.154 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T20:32:28.160 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:28.206 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:28.263 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:28.290 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:28.340 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:32:28.392 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:32:28.401 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:28.409 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:28.417 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:28.424 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-04-26T20:32:28.432 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:28.438 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:28.445 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:28.451 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T20:32:28.458 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:28.465 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:28.510 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:28.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:28.584 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-04-26T20:32:28.634 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-04-26T20:32:28.665 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:28.712 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:28.767 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:28.775 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-04-26T20:32:28.783 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:28.791 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:28.798 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:28.806 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:28.813 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:28.822 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:28.831 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:28.839 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:28.845 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-04-26T20:32:28.876 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-04-26T20:32:28.883 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:28.914 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:28.983 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:28.991 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-04-26T20:32:29.000 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:29.051 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:29.061 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:29.070 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-04-26T20:32:29.079 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-04-26T20:32:29.087 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:29.096 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:29.113 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:29.122 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:29.131 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:29.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:29.148 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-04-26T20:32:29.156 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-04-26T20:32:29.164 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:29.203 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:29.232 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:29.266 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-04-26T20:32:29.808 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-04-26T20:32:29.938 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-04-26T20:32:29.955 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:29.973 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:29.982 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:29.987 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:30.001 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:30.009 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:30.017 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:30.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:30.034 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:30.040 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-04-26T20:32:30.084 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:30.092 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:30.179 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:30.234 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:30.306 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:30.353 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:30.421 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:30.430 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:30.439 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:30.448 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:30.457 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:30.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:30.474 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:30.482 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:30.494 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:30.503 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:30.512 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:30.520 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:30.528 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:30.613 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:30.673 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:30.721 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:30.774 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:30.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:30.864 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:30.872 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:30.881 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:30.890 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:30.899 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:30.911 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:30.919 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:30.927 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:30.934 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:30.942 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:31.005 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:31.014 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:31.094 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:31.103 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:31.159 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:31.168 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:31.258 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:31.267 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:31.274 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:31.281 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:31.288 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:31.295 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:31.303 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:31.314 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:31.322 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:31.329 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:31.336 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:31.384 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:31.391 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:31.449 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:31.456 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:31.509 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:31.543 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:31.630 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:31.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:31.645 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:31.651 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:31.659 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:31.666 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:31.673 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:31.681 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:31.689 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:31.696 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:31.703 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:31.735 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:31.742 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:31.748 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:31.805 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:31.857 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:31.866 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-04-26T20:32:31.942 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-04-26T20:32:31.952 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:31.960 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:31.969 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:31.978 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:32:31.987 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:31.996 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:32.006 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:32.014 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:32.023 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:32.032 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:32.070 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:32.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:32.119 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-04-26T20:32:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T20:32:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:32.203 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:32.258 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:32.266 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:32.275 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:32.284 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:32.292 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:32.300 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:32.309 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:32.316 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:32.324 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:32.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:32.341 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-04-26T20:32:32.376 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-04-26T20:32:32.384 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:32.435 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:32.443 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:32.494 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:32.520 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:32.528 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:32.542 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:32.583 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:32.591 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:32.600 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:32.608 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:32.617 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:32.626 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:32.634 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:32.643 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:32.653 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-04-26T20:32:32.675 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:32.684 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:32.719 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:32.750 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-04-26T20:32:32.789 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-04-26T20:32:32.814 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:32.858 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:32.867 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:32.875 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:32.884 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:32.893 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:32.901 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:32.910 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:32.917 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:32.925 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:32.933 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:32.942 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:32.958 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-04-26T20:32:32.966 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-04-26T20:32:33.010 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:33.035 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:33.061 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:33.103 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:33.171 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:33.179 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:33.186 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:33.193 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:33.200 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:33.206 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:33.214 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:33.227 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:33.233 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:33.241 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:33.288 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:33.294 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:33.356 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:33.366 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:33.418 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:33.475 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:33.524 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:33.530 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:33.537 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:33.544 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:33.551 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:33.559 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:33.566 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:33.574 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:33.582 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:33.589 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:33.596 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:33.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:33.625 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:33.661 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:33.721 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:33.790 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:33.847 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:33.854 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:33.861 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:33.916 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:33.923 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:33.931 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:33.938 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:33.945 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:33.951 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:33.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:33.964 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:33.971 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:33.991 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:34.000 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:34.051 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:34.060 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:34.111 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:34.140 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:34.214 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:34.224 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:34.232 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:34.240 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:34.249 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:34.259 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:34.268 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:34.277 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:34.285 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:34.294 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:34.303 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:34.326 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:34.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:34.415 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:34.449 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:34.486 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:34.494 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:34.559 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:34.567 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:34.577 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:34.585 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:34.595 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:34.604 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:34.615 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:34.624 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:34.632 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:34.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:34.651 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:32:34.674 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:32:34.682 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:34.691 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:34.699 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:34.754 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:34.794 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:34.857 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:34.866 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:34.875 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:34.884 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:34.892 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:34.901 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:34.910 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:34.919 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:32:34.927 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:32:34.937 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:34.945 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:34.967 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:34.976 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:35.010 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:35.017 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:35.070 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:35.196 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:35.208 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:35.216 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:35.226 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:35.235 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:35.244 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:35.252 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:35.260 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-04-26T20:32:35.268 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:35.277 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:35.285 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:35.337 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:35.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:35.376 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:35.432 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:35.480 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:35.545 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:35.621 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:35.628 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:35.637 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:35.646 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:35.655 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:35.664 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:35.673 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:35.691 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:35.700 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:35.709 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:35.741 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:35.749 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:35.798 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:35.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:35.894 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:35.926 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:36.005 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:36.013 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:36.038 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:36.067 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:36.074 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:36.090 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:36.097 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:36.142 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:36.192 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:36.245 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:36.255 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:36.323 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:36.332 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:36.341 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:36.350 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:36.359 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:36.367 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:36.375 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:36.383 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:36.392 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:36.401 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:36.410 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:36.452 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:36.464 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:36.527 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:36.581 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:36.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:36.628 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:36.692 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:36.700 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:36.709 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:36.718 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:36.726 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:36.735 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:36.744 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:36.752 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:36.765 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:36.774 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:36.788 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:36.808 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:36.817 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:36.871 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:36.879 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:36.932 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:36.997 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-04-26T20:32:37.056 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:37.064 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:37.072 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:37.079 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:37.087 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:37.095 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:37.103 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:37.112 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:37.121 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:37.130 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:37.138 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:37.146 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:37.155 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:37.267 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:37.311 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:37.368 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:37.377 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:37.441 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:37.449 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:37.459 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:37.468 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:37.478 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:37.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:37.495 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:37.503 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:37.511 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:37.519 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:37.541 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:37.549 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:37.593 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:37.647 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:37.686 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:37.695 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:37.749 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:37.757 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:37.765 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:37.773 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:37.781 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:37.791 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:37.800 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:37.809 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:37.818 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:37.827 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:37.835 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:37.849 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:37.857 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:37.903 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:37.963 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:38.002 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:38.053 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:38.108 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:38.115 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:38.124 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:38.133 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:38.141 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:38.150 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:38.159 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:38.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:38.179 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:38.189 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:38.197 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:38.215 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:38.225 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:38.275 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:38.333 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:38.369 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:38.426 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:38.473 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:38.481 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:38.489 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:38.498 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:38.516 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-04-26T20:32:38.525 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-04-26T20:32:38.537 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:38.546 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:38.554 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:38.563 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-04-26T20:32:38.582 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:38.591 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:38.633 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:38.688 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T20:32:38.736 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:38.780 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:38.852 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:38.861 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:38.870 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:38.879 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:38.887 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:38.896 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-04-26T20:32:38.905 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:38.913 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:38.931 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:38.940 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:38.957 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:38.965 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:39.009 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:39.069 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:39.153 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:39.211 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:39.285 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:39.294 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:39.303 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:39.312 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:39.322 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:39.330 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:39.338 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:39.346 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:39.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:39.363 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:39.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:39.403 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:39.413 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:39.450 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:39.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:39.533 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:39.582 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:39.669 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:39.678 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:39.687 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:39.697 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:39.705 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:39.735 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:39.748 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:39.758 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:39.767 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:39.803 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:39.813 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:39.866 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:39.918 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:39.951 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:40.005 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:40.072 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:40.079 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:40.088 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:40.097 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:40.105 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:40.115 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:40.124 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:40.131 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:40.148 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:40.182 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:40.191 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:40.225 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:40.281 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:40.317 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:40.326 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:40.391 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:40.399 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:40.408 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:40.416 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:40.425 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:40.434 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:40.443 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:40.451 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:40.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:40.469 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:40.477 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:40.507 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-04-26T20:32:40.515 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-04-26T20:32:40.550 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:40.558 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:40.614 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:40.641 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:40.718 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:40.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:40.732 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:40.739 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:40.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:40.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:40.760 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:40.767 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:40.775 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-04-26T20:32:40.782 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-04-26T20:32:40.789 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:40.815 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:40.822 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:40.849 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-04-26T20:32:40.876 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:40.917 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:40.948 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:41.023 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:41.034 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:41.041 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:41.051 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:41.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:41.068 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-04-26T20:32:41.076 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-04-26T20:32:41.085 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:41.095 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:41.104 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:41.112 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:41.121 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:41.129 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:41.203 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:41.256 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:41.367 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:41.495 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:41.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:41.511 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-04-26T20:32:41.520 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-04-26T20:32:41.529 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:41.537 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:41.546 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:41.564 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:41.572 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:41.580 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:41.635 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:41.641 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:41.692 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:41.745 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:41.796 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:41.849 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-04-26T20:32:41.932 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-04-26T20:32:41.939 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:32:41.947 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:41.957 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:41.966 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:41.975 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:41.983 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:42.013 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:42.022 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:42.030 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:42.039 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:42.078 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:42.087 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:42.142 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-04-26T20:32:42.200 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-04-26T20:32:42.255 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:42.305 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:42.312 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:42.320 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:42.328 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:42.338 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:42.408 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:42.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:42.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:42.433 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:42.442 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:42.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:42.459 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-04-26T20:32:42.468 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-04-26T20:32:42.476 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:42.564 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:42.621 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:42.629 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:42.698 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:42.708 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:42.716 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:42.726 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:42.734 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:42.743 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:42.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:42.760 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-04-26T20:32:42.769 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-04-26T20:32:42.782 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:42.791 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:42.831 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:42.838 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:42.865 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:42.919 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:42.963 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:43.015 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:43.095 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:43.102 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:43.110 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:43.118 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:43.125 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-04-26T20:32:43.132 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-04-26T20:32:43.141 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:43.148 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:43.155 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:43.162 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:43.169 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:32:43.209 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-04-26T20:32:43.228 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:43.315 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:43.358 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:43.434 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:43.441 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:43.531 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:43.538 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-04-26T20:32:43.546 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-04-26T20:32:43.555 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:43.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:43.575 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:43.583 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:43.601 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:43.611 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:43.619 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:43.664 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:43.673 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:43.710 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:43.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:43.825 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-04-26T20:32:43.833 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-04-26T20:32:43.898 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:43.907 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:43.915 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:43.925 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:43.934 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:43.944 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:43.952 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:43.960 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:43.969 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:43.978 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:43.987 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:44.036 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:44.045 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-04-26T20:32:44.088 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-04-26T20:32:44.144 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:44.204 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:44.268 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:44.356 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:44.365 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:44.374 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:44.382 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:44.391 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:44.399 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:44.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:44.416 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:44.425 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:44.434 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:44.442 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:44.464 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:44.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-04-26T20:32:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:44.532 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:44.574 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:44.632 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-04-26T20:32:44.714 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-04-26T20:32:44.723 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:44.732 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:44.741 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:44.750 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:44.769 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:44.776 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:44.785 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T20:32:44.794 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:44.803 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:44.827 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:45.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:45.445 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-04-26T20:32:45.509 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-04-26T20:32:45.573 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:45.632 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:45.731 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:45.738 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:45.747 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:45.754 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:45.761 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:45.769 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-04-26T20:32:45.776 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:45.784 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:45.791 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:45.798 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:45.805 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:45.858 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:45.866 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:45.913 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:45.964 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:46.006 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:46.062 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:46.139 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:46.148 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:46.157 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:46.165 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:46.175 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:46.184 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:46.192 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:46.201 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:46.210 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:46.219 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:46.226 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:46.261 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:46.271 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:46.317 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:46.370 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:46.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:46.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:46.531 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:46.540 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:46.549 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:46.558 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:46.566 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:46.577 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:46.586 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:46.595 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:46.603 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:46.612 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:46.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:46.651 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:46.661 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:46.718 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:46.775 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:46.822 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:46.830 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:46.905 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:46.914 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:46.923 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:46.931 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:46.940 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:46.949 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:46.958 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:46.967 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:46.977 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:46.986 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:46.994 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:47.022 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:47.032 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:47.075 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:47.130 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:47.198 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:47.250 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:47.322 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:47.332 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:47.341 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:47.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:47.359 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:47.367 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:47.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:47.393 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:47.402 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:47.411 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:47.445 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:47.454 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:47.492 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:47.544 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:47.595 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-04-26T20:32:47.647 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:47.753 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:47.760 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:47.769 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:47.777 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:47.784 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:47.791 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:47.798 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:47.806 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:47.813 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:47.820 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:47.828 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:47.855 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:47.862 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:47.900 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:47.959 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:47.966 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:47.974 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:48.091 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:48.098 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:48.105 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:48.112 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:48.120 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:48.127 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:48.134 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:48.141 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:48.147 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:48.154 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:48.161 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:48.198 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:48.206 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:48.248 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:48.281 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:48.345 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:48.392 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:48.472 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:48.479 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:48.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:48.493 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:48.500 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:48.508 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:48.517 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:48.527 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:48.536 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:48.544 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:48.553 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:48.576 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:48.584 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:48.637 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:48.645 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:48.704 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:48.760 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:48.856 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:48.874 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:48.883 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:48.892 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:48.901 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:48.910 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:48.920 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:48.929 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:48.939 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:48.960 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:48.968 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:49.008 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:49.016 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:49.073 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:49.082 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:49.163 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:49.180 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:49.189 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:49.198 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:49.207 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:49.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:49.227 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:49.234 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-04-26T20:32:49.242 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:49.251 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:49.264 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:49.284 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:49.294 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:49.383 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:49.440 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:49.531 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:49.570 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:49.646 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:49.655 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:49.663 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:49.672 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:49.680 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:49.690 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:49.698 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:49.708 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:49.717 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:49.725 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:49.735 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:49.776 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:49.849 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:49.912 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:49.961 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:50.069 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:50.171 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:50.183 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:50.190 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:50.198 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:50.206 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:50.213 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:50.220 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:50.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:50.234 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:50.241 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:50.247 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:50.281 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:50.288 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:50.324 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:50.377 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:50.438 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:50.446 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:50.530 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:50.536 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:50.544 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:50.552 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:50.559 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:50.567 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:50.574 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:50.585 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:50.592 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:50.600 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:50.607 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:50.642 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:50.649 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:50.701 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:50.707 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:50.819 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:50.884 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:50.893 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:50.899 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:50.906 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:50.913 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:50.920 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:50.927 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:50.934 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:50.941 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:50.948 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:50.955 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:51.003 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:51.010 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:51.058 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:51.064 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:51.113 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-04-26T20:32:51.204 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:51.213 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:51.222 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:51.230 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-04-26T20:32:51.240 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-04-26T20:32:51.249 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:51.258 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:51.266 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:51.275 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T20:32:51.284 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:51.292 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:51.335 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:51.344 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:51.391 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:51.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:51.492 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:51.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:51.565 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:32:51.574 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:32:51.583 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:32:51.592 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:51.601 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:51.610 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-04-26T20:32:51.619 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:51.627 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:51.644 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:51.665 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:51.673 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:51.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:51.730 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:51.782 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:51.810 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:51.879 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:51.886 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:51.896 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:51.904 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:51.913 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:51.922 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:51.930 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:51.938 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:51.947 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:51.957 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:51.966 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:52.000 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:52.009 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:52.057 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:52.110 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:52.152 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:52.161 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:52.229 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:52.237 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:52.246 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:52.254 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:52.263 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:52.271 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:52.280 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:52.288 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:52.297 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:52.305 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:52.313 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:52.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:52.358 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:52.404 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:52.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:52.502 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:52.551 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:52.629 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:52.637 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:52.653 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:52.660 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:52.668 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:52.675 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:52.682 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:52.690 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:52.697 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:52.704 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:52.712 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:52.751 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:52.758 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:52.796 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:52.846 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:52.902 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:52.957 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:53.037 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:53.045 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:53.052 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:32:53.060 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:32:53.066 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:53.074 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:53.081 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:53.088 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:53.094 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:53.103 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:53.111 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:53.153 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:53.161 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:53.199 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:53.252 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:53.302 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:53.362 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:32:53.479 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:53.485 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:32:53.492 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-04-26T20:32:53.499 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:32:53.507 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:32:53.515 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:32:53.522 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-04-26T20:32:53.528 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-04-26T20:32:53.537 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:53.546 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:53.554 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:53.601 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:53.610 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:53.640 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:53.699 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:53.743 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T20:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:53.890 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:53.900 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:53.909 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:53.918 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-04-26T20:32:53.928 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-04-26T20:32:53.938 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:53.946 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:53.954 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:53.962 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:53.971 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:53.980 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:54.008 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:54.016 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T20:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:54.114 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:54.193 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:54.258 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:54.337 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-04-26T20:32:54.346 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-04-26T20:32:54.355 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:54.363 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:54.372 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:54.380 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:32:54.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:54.397 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:54.405 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:54.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T20:32:54.422 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:54.467 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:54.476 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:54.512 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:54.521 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-04-26T20:32:54.576 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-04-26T20:32:54.585 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:54.653 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:54.662 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:54.671 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:54.680 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:54.688 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:54.697 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:54.706 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T20:32:54.715 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:54.724 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:54.733 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:54.742 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:54.766 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-04-26T20:32:54.775 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-04-26T20:32:54.820 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:54.875 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:54.927 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:54.969 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:55.037 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:55.045 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:55.052 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:55.060 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T20:32:55.066 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:55.073 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:55.083 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:55.479 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:56.338 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-04-26T20:32:56.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-04-26T20:32:56.418 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:56.485 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:56.492 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:56.500 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:56.590 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:56.642 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:56.678 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:56.778 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:56.787 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:56.795 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:56.803 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:56.813 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:56.821 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-04-26T20:32:56.830 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-04-26T20:32:56.839 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:56.847 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:56.856 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:56.865 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:56.911 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:56.920 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:56.969 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:57.031 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:57.096 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:57.159 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:57.228 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:57.236 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:57.245 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-04-26T20:32:57.254 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-04-26T20:32:57.262 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:57.272 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:57.281 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:57.290 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T20:32:57.304 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:57.313 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:57.322 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:57.366 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:57.375 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:57.426 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:57.487 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:57.529 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:57.537 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-04-26T20:32:57.603 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-04-26T20:32:57.610 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:57.617 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:57.624 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:57.632 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:57.641 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:57.650 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:57.656 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:57.671 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:57.736 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:57.742 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:57.774 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:32:57.800 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:32:57.845 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:57.869 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:57.939 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:57.946 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-04-26T20:32:57.953 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-04-26T20:32:57.961 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:57.982 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:57.989 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:57.997 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:58.004 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:58.011 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-04-26T20:32:58.049 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-04-26T20:32:58.056 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:32:58.115 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:58.173 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:58.212 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-04-26T20:32:58.257 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:58.311 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:58.318 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:58.324 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:58.331 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:58.339 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:58.346 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:58.354 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:58.364 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-04-26T20:32:58.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-04-26T20:32:58.381 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:58.390 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:58.436 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:58.447 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:58.490 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:58.529 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:58.569 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:58.623 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:58.695 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:58.702 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:58.710 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:58.719 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:58.728 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-04-26T20:32:58.736 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-04-26T20:32:58.745 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:58.753 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:58.761 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:32:58.769 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:58.778 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:58.823 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:58.832 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:58.877 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:58.918 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:59.021 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:59.106 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:59.115 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:32:59.124 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:32:59.134 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:59.142 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:59.153 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:59.161 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:32:59.170 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:59.178 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:59.187 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:59.195 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:59.223 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:59.232 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:32:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:32:59.340 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:32:59.378 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:32:59.427 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:32:59.485 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:59.494 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:59.503 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:59.511 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:59.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:59.529 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:32:59.538 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:32:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:32:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:59.561 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:59.570 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:32:59.600 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:59.609 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:59.646 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:32:59.705 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:32:59.745 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:32:59.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:32:59.828 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:32:59.836 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:32:59.845 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:32:59.853 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:32:59.862 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:32:59.871 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:32:59.880 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:32:59.889 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:32:59.897 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:32:59.905 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:32:59.913 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:32:59.940 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:32:59.948 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:32:59.996 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:00.028 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:00.077 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:00.103 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:00.169 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:00.176 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:00.183 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:00.189 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:00.196 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:00.207 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:00.215 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:00.222 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:00.232 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:00.239 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:00.248 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:00.281 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:00.289 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:00.331 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:00.382 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:00.445 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:00.490 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:00.557 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:00.567 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:00.576 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:00.584 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-04-26T20:33:00.593 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-04-26T20:33:00.601 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:00.610 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:00.620 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:00.627 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:00.636 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:00.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:00.664 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:00.673 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:00.723 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:00.748 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:00.804 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:00.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:00.917 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-04-26T20:33:00.926 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-04-26T20:33:00.934 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:00.942 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:00.952 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:00.960 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:00.969 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:00.977 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:00.986 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:00.995 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:01.003 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:01.042 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:01.049 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:01.100 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:01.152 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-04-26T20:33:01.209 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-04-26T20:33:01.260 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:01.310 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:01.317 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:01.324 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T20:33:01.331 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:01.341 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:01.350 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:01.358 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:01.367 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:01.376 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:01.386 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:01.394 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:01.427 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:01.435 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:01.494 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:01.503 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-04-26T20:33:01.559 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:01.617 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:01.668 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:01.677 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:33:01.684 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:33:01.693 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:01.701 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:01.709 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:01.718 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:01.727 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:01.735 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:01.745 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:01.754 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:01.769 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:01.777 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:01.829 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:01.856 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:01.910 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:01.961 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:02.023 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:33:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:33:02.083 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:02.091 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:02.100 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:02.108 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:02.136 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:02.144 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:02.180 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:02.237 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:02.277 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:02.336 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:02.405 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:02.414 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:02.422 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:33:02.431 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:33:02.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:02.448 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:02.457 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:02.465 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:33:02.474 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:02.483 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:02.492 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:02.520 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:02.531 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:02.587 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:02.595 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:02.653 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:02.707 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:02.765 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:02.774 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:02.784 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:02.793 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:02.802 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:02.810 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:02.819 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:02.827 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:02.836 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:02.844 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:02.854 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:02.875 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:02.884 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:02.957 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:02.981 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:03.029 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:03.038 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:03.093 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:03.104 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:03.112 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:03.120 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:03.129 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:03.138 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:03.147 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-04-26T20:33:03.155 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:03.164 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:03.173 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:03.182 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:33:03.201 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:33:03.210 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:03.256 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:03.278 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:03.335 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:03.343 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:03.399 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:03.408 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:03.416 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:03.427 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:03.436 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:03.445 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:03.462 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:03.471 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:03.484 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:03.493 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:03.521 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:03.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:03.667 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:33:03.707 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:33:03.769 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:03.830 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:03.839 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:03.848 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:03.857 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:03.866 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:03.875 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:03.884 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:03.894 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:03.903 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:03.912 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:03.920 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:03.954 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:33:03.963 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:33:04.012 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:04.065 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:04.095 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:04.154 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:33:04.209 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:04.218 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:04.227 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:04.235 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:04.244 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:04.261 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:04.270 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:04.278 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:04.286 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:04.297 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:04.307 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:04.315 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:04.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:04.404 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:04.447 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:04.510 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:04.559 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:04.569 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:04.576 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:04.583 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:04.591 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:04.597 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:04.603 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:04.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:04.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:04.628 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:04.637 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:04.666 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:04.675 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-04-26T20:33:04.712 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-04-26T20:33:04.768 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:04.801 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:04.866 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:04.934 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-04-26T20:33:04.943 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:04.952 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:04.960 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:04.969 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T20:33:04.977 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:04.985 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:04.994 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:05.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:05.015 INFO:teuthology.orchestra.run.smithi131.stdout: "periodic_async_work": { 2024-04-26T20:33:05.024 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "periodic_async_work", 2024-04-26T20:33:05.046 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:05.055 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:05.121 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:05.183 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:05.192 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:05.248 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:05.257 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Periodically check for async work", 2024-04-26T20:33:05.269 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:05.287 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:05.296 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:05.304 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-04-26T20:33:05.313 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-04-26T20:33:05.324 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:05.335 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:05.343 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:05.352 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:33:05.361 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:05.403 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:05.429 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:05.459 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T20:33:05.480 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:05.544 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:05.553 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:05.562 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-04-26T20:33:05.588 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T20:33:05.595 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:05.604 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:05.611 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:05.619 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:05.627 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:05.636 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:05.657 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:05.667 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T20:33:05.747 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:05.802 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:05.847 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:05.878 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:05.934 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:05.943 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:05.951 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:05.959 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-04-26T20:33:05.968 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:05.977 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:05.986 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:05.994 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-04-26T20:33:06.003 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-04-26T20:33:06.012 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:33:06.021 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:06.038 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:06.046 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-04-26T20:33:06.089 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:06.142 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:06.250 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:06.325 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:06.334 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:06.342 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:06.351 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:06.359 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-04-26T20:33:06.368 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-04-26T20:33:06.376 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:06.384 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:06.392 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:06.401 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:06.410 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:06.418 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:06.427 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:06.500 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:06.555 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:06.589 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:06.639 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:06.692 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:06.701 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:33:06.710 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:33:06.718 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:06.726 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:06.734 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:06.742 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:33:06.751 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:06.759 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:06.767 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:06.776 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:06.796 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:06.805 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:06.849 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:06.913 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:06.946 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:33:06.953 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:33:07.017 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:07.023 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:07.032 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:07.041 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:07.051 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:07.059 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:07.068 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:07.077 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:07.084 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:07.093 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:07.102 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:07.117 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:07.126 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:07.170 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:07.227 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:07.235 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:07.297 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:33:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:33:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:33:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:33:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:33:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:08.160 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-04-26T20:33:08.499 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-04-26T20:33:08.508 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:08.516 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:08.524 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:08.532 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:08.540 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:08.550 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:08.559 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:08.567 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:08.576 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:08.598 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:08.609 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:08.670 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:08.679 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-04-26T20:33:08.730 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-04-26T20:33:08.738 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:08.799 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:08.809 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:08.817 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-04-26T20:33:08.826 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:08.833 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:08.842 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:08.850 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:08.857 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:08.865 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:08.873 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:08.881 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:08.895 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-04-26T20:33:08.903 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-04-26T20:33:08.945 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:08.968 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:09.042 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:09.087 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T20:33:09.171 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:09.180 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:09.188 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:09.196 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:09.206 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:09.213 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:09.221 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:09.228 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:09.237 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:09.245 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:09.253 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:33:09.302 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:09.310 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:09.369 INFO:teuthology.orchestra.run.smithi131.stdout: "modules": [ 2024-04-26T20:33:09.426 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-04-26T20:33:09.458 INFO:teuthology.orchestra.run.smithi131.stdout: "dashboard", 2024-04-26T20:33:09.466 INFO:teuthology.orchestra.run.smithi131.stdout: "iostat", 2024-04-26T20:33:09.558 INFO:teuthology.orchestra.run.smithi131.stdout: "nfs", 2024-04-26T20:33:09.566 INFO:teuthology.orchestra.run.smithi131.stdout: "restful" 2024-04-26T20:33:09.575 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:09.583 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-04-26T20:33:09.592 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:09.601 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-04-26T20:33:09.610 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:09.619 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:09.629 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:09.639 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:33:09.647 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:33:09.675 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:09.684 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:09.727 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:09.754 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:33:09.802 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:09.809 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:09.883 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:09.891 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T20:33:09.901 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:09.909 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:09.918 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:09.926 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:09.935 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:33:09.942 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:33:09.951 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:09.959 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:09.967 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:10.070 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:10.135 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:10.166 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:10.196 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:10.257 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:10.265 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:10.272 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:10.281 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:10.289 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:10.296 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:10.303 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:10.310 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:10.318 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:10.324 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:10.334 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:33:10.356 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:33:10.363 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:10.431 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:10.439 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:10.492 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:10.499 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:10.585 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:10.592 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:10.599 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:10.607 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:10.614 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:10.620 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:10.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:10.634 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:33:10.641 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:33:10.649 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:10.656 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:10.673 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:33:10.848 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:11.490 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:11.545 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:11.569 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:11.632 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:11.639 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:11.648 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:11.657 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:11.665 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:11.675 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:11.688 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:11.697 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:11.705 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:11.713 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:11.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:11.766 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:11.775 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:11.893 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:11.953 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:12.003 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:12.010 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:12.074 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:12.087 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:12.096 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:12.105 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:12.113 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:12.122 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:12.129 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:12.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:12.146 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-04-26T20:33:12.153 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-04-26T20:33:12.162 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:12.170 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:12.178 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:12.201 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:12.242 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:12.274 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:12.297 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:12.356 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-04-26T20:33:12.364 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:12.371 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:12.378 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:12.384 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:12.391 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-04-26T20:33:12.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-04-26T20:33:12.405 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:12.412 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:12.419 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:12.427 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-04-26T20:33:12.453 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:12.462 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:12.501 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:12.530 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-04-26T20:33:12.575 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:12.583 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:12.681 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:12.690 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:12.698 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-04-26T20:33:12.706 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-04-26T20:33:12.715 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:12.724 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:12.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:12.740 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:12.749 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:12.757 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:12.765 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:12.799 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-04-26T20:33:12.808 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:12.856 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:12.919 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:12.953 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:12.999 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-04-26T20:33:13.056 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-04-26T20:33:13.064 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:13.073 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:13.081 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:13.089 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:13.097 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:13.107 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:13.115 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:13.123 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-04-26T20:33:13.131 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:13.139 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:13.161 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:13.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:13.179 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-04-26T20:33:13.187 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-04-26T20:33:13.253 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:13.261 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:13.325 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:13.336 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-04-26T20:33:13.344 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:13.353 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:13.362 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:13.369 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-04-26T20:33:13.378 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:13.387 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:13.396 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:13.405 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:13.413 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-04-26T20:33:13.436 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-04-26T20:33:13.448 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:13.494 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:13.578 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:13.633 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:13.715 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:13.723 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:13.732 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-04-26T20:33:13.740 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:13.748 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:13.756 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:13.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:13.772 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-04-26T20:33:13.781 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-04-26T20:33:13.788 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:13.795 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:13.840 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:13.848 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:13.884 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:13.944 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:13.996 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:14.052 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T20:33:14.115 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:14.123 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:14.131 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:14.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:14.148 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-04-26T20:33:14.156 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-04-26T20:33:14.164 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:14.173 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:14.183 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:14.191 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:14.199 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:14.227 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:14.236 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:14.306 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-04-26T20:33:14.358 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:14.405 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:14.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:14.475 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:14.485 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:14.557 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:14.565 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:14.572 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-04-26T20:33:14.579 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:14.587 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:14.594 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:14.601 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-04-26T20:33:14.608 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-04-26T20:33:14.615 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:14.639 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:14.645 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:14.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:14.753 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:14.796 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:14.836 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:14.940 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T20:33:14.950 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:14.964 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:14.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:14.979 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-04-26T20:33:14.986 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-04-26T20:33:14.992 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:14.999 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:15.006 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:15.013 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-04-26T20:33:15.037 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:15.044 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:15.093 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:15.153 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T20:33:15.186 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T20:33:15.195 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:15.263 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:15.273 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:15.281 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-04-26T20:33:15.290 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-04-26T20:33:15.298 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:33:15.308 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:15.317 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:15.328 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:33:15.339 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-04-26T20:33:15.347 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-04-26T20:33:15.356 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:15.377 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T20:33:15.386 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T20:33:15.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:15.509 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:15.561 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:15.623 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-04-26T20:33:15.698 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-04-26T20:33:15.705 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:33:15.714 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:15.722 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:15.731 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-04-26T20:33:15.739 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:33:15.748 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-04-26T20:33:15.759 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:15.768 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T20:33:15.776 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:15.785 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:15.811 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:15.819 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:15.859 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-04-26T20:33:15.914 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-04-26T20:33:15.952 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:16.009 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:16.079 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:16.117 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T20:33:16.126 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:16.134 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:16.143 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:16.149 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T20:33:16.158 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T20:33:16.166 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:16.175 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:16.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:16.192 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-04-26T20:33:16.210 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-04-26T20:33:16.219 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:33:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:16.366 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-04-26T20:33:16.412 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-04-26T20:33:16.479 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-04-26T20:33:16.490 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:16.497 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-04-26T20:33:16.506 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T20:33:16.514 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:16.523 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:16.532 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:16.541 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-04-26T20:33:16.551 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-04-26T20:33:16.560 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:16.569 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:16.585 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:16.593 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-04-26T20:33:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:16.751 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:16.809 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T20:33:16.871 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T20:33:16.877 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:16.884 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:16.891 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:16.897 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-04-26T20:33:16.904 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-04-26T20:33:16.911 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:33:16.924 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:16.932 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:16.939 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:33:16.948 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-04-26T20:33:17.002 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-04-26T20:33:17.012 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:17.075 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T20:33:17.132 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T20:33:17.141 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:17.275 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:17.371 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:17.379 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:33:17.388 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:33:17.398 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:17.406 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:17.414 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:17.423 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:17.431 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:17.439 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:17.448 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:17.457 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:17.492 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:17.501 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:17.558 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:17.613 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:17.680 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:17.687 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:17.754 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:17.768 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:17.775 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:17.782 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:17.791 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:33:17.798 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:33:17.806 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:17.813 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:17.819 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:17.828 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:17.852 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:17.860 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:17.900 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:17.908 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:18.005 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:18.058 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:18.113 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:18.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:18.129 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:33:18.137 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:33:18.146 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:18.154 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:18.163 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:18.171 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:33:18.180 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:18.189 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:18.197 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:18.213 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:18.222 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:18.277 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:18.325 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:18.382 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:18.391 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:18.454 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:18.464 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:18.472 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:18.480 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:18.489 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:18.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:18.506 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:18.515 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:18.542 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:18.560 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:18.569 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:18.625 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:18.672 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:18.717 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:18.770 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:18.820 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:18.831 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:18.840 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:18.849 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-04-26T20:33:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-04-26T20:33:18.867 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:33:18.876 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:18.885 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:18.893 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:33:18.903 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:18.911 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:18.929 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:18.937 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T20:33:18.983 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:19.049 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:19.092 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:19.151 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:19.229 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-04-26T20:33:19.238 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-04-26T20:33:19.247 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:19.255 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:19.264 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:19.272 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-04-26T20:33:19.281 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:19.289 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:19.298 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:19.306 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-04-26T20:33:19.314 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-04-26T20:33:19.337 INFO:teuthology.orchestra.run.smithi131.stdout: "read", 2024-04-26T20:33:19.345 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap", 2024-04-26T20:33:19.399 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap-read" 2024-04-26T20:33:19.432 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:19.483 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-04-26T20:33:19.509 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:19.579 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:19.588 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:19.597 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:19.605 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-04-26T20:33:19.613 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-04-26T20:33:19.622 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:19.629 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:19.637 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:19.648 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:19.655 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:19.663 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:19.697 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:19.706 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T20:33:19.770 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:19.821 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:19.868 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:19.924 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:19.997 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:33:20.004 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:33:20.011 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:20.018 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:20.026 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:20.033 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:33:20.040 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:20.047 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:20.054 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:20.061 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T20:33:20.069 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:20.117 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:20.126 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:20.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:20.191 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-04-26T20:33:20.240 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-04-26T20:33:20.294 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:20.353 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:20.362 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:20.371 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:33:20.379 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:33:20.387 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:20.395 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:20.404 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T20:33:20.420 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T20:33:20.429 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:20.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:20.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:20.480 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-04-26T20:33:20.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-04-26T20:33:20.530 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:33:20.538 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:20.595 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:20.617 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:33:20.690 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:20.698 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:20.707 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:20.716 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T20:33:20.725 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:20.733 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:20.742 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:20.752 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:20.760 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:20.769 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:20.779 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:20.801 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-04-26T20:33:20.810 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:20.849 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:20.904 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:20.983 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-04-26T20:33:21.030 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-04-26T20:33:21.098 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:33:21.107 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:21.115 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:21.123 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-04-26T20:33:21.132 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:21.141 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:21.149 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:21.158 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T20:33:21.166 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:21.175 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:21.183 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:21.201 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:21.210 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-04-26T20:33:21.274 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-04-26T20:33:21.283 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:21.341 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:21.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-04-26T20:33:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T20:33:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:21.507 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-04-26T20:33:21.515 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-04-26T20:33:21.548 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:21.557 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-04-26T20:33:21.667 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:21.676 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:21.745 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:21.754 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T20:33:21.763 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:21.772 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:21.781 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:21.789 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:21.798 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-04-26T20:33:21.806 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-04-26T20:33:21.815 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:21.823 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:21.832 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:21.899 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:21.908 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:21.976 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:22.031 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:22.076 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T20:33:22.135 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T20:33:22.203 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:22.211 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:22.219 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:22.227 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-04-26T20:33:22.236 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-04-26T20:33:22.244 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:22.253 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:22.262 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:22.271 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:33:22.280 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:22.289 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:22.337 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:22.343 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T20:33:22.402 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:22.428 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:22.484 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:22.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:22.639 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-04-26T20:33:22.649 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T20:33:22.658 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:33:22.666 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:22.675 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:22.684 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-04-26T20:33:22.695 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:22.704 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:22.713 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:22.722 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T20:33:22.731 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:22.778 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:22.787 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:22.842 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:22.884 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm_log_destination": { 2024-04-26T20:33:22.934 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm_log_destination", 2024-04-26T20:33:22.943 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:23.007 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:23.016 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:23.034 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:23.043 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:23.051 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:23.060 INFO:teuthology.orchestra.run.smithi131.stdout: "file", 2024-04-26T20:33:23.067 INFO:teuthology.orchestra.run.smithi131.stdout: "file,syslog", 2024-04-26T20:33:23.076 INFO:teuthology.orchestra.run.smithi131.stdout: "syslog" 2024-04-26T20:33:23.085 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:23.094 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T20:33:23.130 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:23.139 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:23.234 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:23.290 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:23.333 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-04-26T20:33:23.388 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-04-26T20:33:23.453 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:23.461 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:23.471 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:23.480 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:23.488 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:23.496 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:23.505 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:23.513 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T20:33:23.522 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:23.530 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:23.537 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:23.590 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:23.599 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-04-26T20:33:23.665 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-04-26T20:33:23.729 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:23.784 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:23.793 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:23.854 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:23.861 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:23.868 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:23.875 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:23.882 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T20:33:23.889 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:23.896 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:23.905 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:23.914 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:23.922 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-04-26T20:33:23.931 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-04-26T20:33:23.957 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:23.965 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:24.020 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:24.198 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:24.269 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:24.276 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T20:33:24.285 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:24.294 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:24.301 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:24.310 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:24.317 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-04-26T20:33:24.326 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-04-26T20:33:24.334 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:24.342 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:24.351 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:24.391 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T20:33:24.400 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:24.482 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:24.549 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:24.600 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:33:24.653 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:24.744 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:24.753 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:24.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:24.771 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-04-26T20:33:24.780 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-04-26T20:33:24.788 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:24.805 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:24.813 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T20:33:24.822 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:24.830 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:24.869 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:24.878 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-04-26T20:33:24.931 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:24.987 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:25.040 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:25.048 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:25.119 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-04-26T20:33:25.128 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-04-26T20:33:25.136 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:25.145 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:25.153 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:25.163 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T20:33:25.172 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:25.182 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:25.191 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:25.200 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-04-26T20:33:25.208 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:25.241 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:25.250 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:25.289 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-04-26T20:33:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-04-26T20:33:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T20:33:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:33:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-04-26T20:33:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-04-26T20:33:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T20:33:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-04-26T20:33:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-04-26T20:33:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-04-26T20:33:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:26.181 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T20:33:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:26.858 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-04-26T20:33:27.054 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:27.068 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:27.078 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:27.091 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:27.099 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-04-26T20:33:27.107 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-04-26T20:33:27.115 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:27.122 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:27.130 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:27.139 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T20:33:27.172 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:27.181 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:27.189 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:27.295 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-04-26T20:33:27.378 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:27.387 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:27.460 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:27.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:27.478 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-04-26T20:33:27.486 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-04-26T20:33:27.493 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:27.501 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:27.508 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:27.517 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T20:33:27.525 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:27.534 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:27.543 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:27.574 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-04-26T20:33:27.583 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:27.648 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:27.697 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:27.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:27.792 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-04-26T20:33:27.881 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-04-26T20:33:27.890 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:27.898 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:27.907 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:27.915 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T20:33:27.924 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:27.932 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:27.940 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:27.948 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-04-26T20:33:27.957 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:27.965 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:28.019 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:28.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-04-26T20:33:28.133 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-04-26T20:33:28.189 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:28.228 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:28.290 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:28.299 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T20:33:28.307 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:28.315 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:28.324 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:28.334 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-04-26T20:33:28.343 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:28.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:28.361 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:28.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:28.378 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-04-26T20:33:28.441 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-04-26T20:33:28.451 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:28.567 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:28.635 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:28.700 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T20:33:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:28.823 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:28.832 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:28.842 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:33:28.854 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:28.864 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:28.872 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:28.880 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:28.888 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-04-26T20:33:28.897 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-04-26T20:33:28.905 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:28.914 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:28.956 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:28.965 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T20:33:29.077 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:29.128 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:29.169 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:29.217 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-04-26T20:33:29.274 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:29.282 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:29.291 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:29.299 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:29.309 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-04-26T20:33:29.317 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-04-26T20:33:29.325 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:29.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T20:33:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:29.453 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-04-26T20:33:29.511 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:29.561 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:29.568 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:29.640 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:29.646 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-04-26T20:33:29.653 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-04-26T20:33:29.660 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:29.667 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:29.674 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:29.681 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T20:33:29.687 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:29.694 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:29.701 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:29.707 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-04-26T20:33:29.747 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:29.754 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:29.824 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:29.881 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:29.938 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_samba": { 2024-04-26T20:33:29.989 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_samba", 2024-04-26T20:33:30.059 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:30.067 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:30.075 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:30.086 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T20:33:30.095 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:30.103 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:30.120 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Samba/SMB container image", 2024-04-26T20:33:30.129 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:30.137 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:30.145 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:30.167 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:30.175 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-04-26T20:33:30.224 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-04-26T20:33:30.253 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:30.314 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:30.377 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:30.434 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T20:33:30.442 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:30.452 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:30.460 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:30.469 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-04-26T20:33:30.477 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:30.484 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:30.493 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:30.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:30.511 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-04-26T20:33:30.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-04-26T20:33:30.544 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:30.552 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:30.618 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:30.676 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:30.727 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:30.801 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:30.866 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:30.876 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T20:33:30.885 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:30.894 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:30.902 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:30.911 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:30.919 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-04-26T20:33:30.927 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-04-26T20:33:30.936 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:30.944 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:30.953 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:30.999 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:33:31.008 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:31.063 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:31.072 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:31.131 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T20:33:31.189 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:31.235 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:31.244 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:31.254 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:31.263 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-04-26T20:33:31.271 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T20:33:31.279 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:31.287 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:31.295 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:31.305 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-04-26T20:33:31.313 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:31.322 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:31.344 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:31.353 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T20:33:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:31.425 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:31.489 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:31.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:31.600 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-04-26T20:33:31.608 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-04-26T20:33:31.617 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:31.625 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:31.633 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:31.661 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-04-26T20:33:31.669 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:31.678 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:31.700 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:31.773 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T20:33:31.791 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:31.798 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:31.804 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:31.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:31.913 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-04-26T20:33:31.983 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-04-26T20:33:32.052 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:32.059 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:32.068 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:32.076 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-04-26T20:33:32.085 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:32.093 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:32.102 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:32.112 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-04-26T20:33:32.121 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:32.130 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:32.138 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-04-26T20:33:32.218 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-04-26T20:33:32.277 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:32.335 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:32.386 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:32.442 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:32.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:32.457 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:32.465 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:32.474 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T20:33:32.483 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:32.492 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:32.500 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:32.509 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:32.517 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-04-26T20:33:32.526 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-04-26T20:33:32.549 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:32.558 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:32.609 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:32.673 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:33:32.718 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:32.763 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:32.829 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:32.837 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-04-26T20:33:32.845 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:32.854 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:32.863 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:32.871 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:32.880 INFO:teuthology.orchestra.run.smithi131.stdout: "grafana_dashboards_path": { 2024-04-26T20:33:32.888 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "grafana_dashboards_path", 2024-04-26T20:33:32.896 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:32.905 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:32.914 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:32.964 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T20:33:32.973 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:33.039 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:33.087 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:33.137 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T20:33:33.210 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:33.272 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:33.280 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:33.289 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-04-26T20:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-04-26T20:33:33.316 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:33.326 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:33.335 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:33.344 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:33:33.353 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:33.361 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:33.403 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:33.414 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-04-26T20:33:33.454 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:33.487 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:33.544 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:33.597 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:33.667 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-04-26T20:33:33.675 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-04-26T20:33:33.683 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:33.691 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:33.700 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:33.708 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:33.717 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:33.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:33.739 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:33.747 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T20:33:33.756 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:33.788 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:33.797 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:33.845 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:33.898 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-04-26T20:33:33.944 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-04-26T20:33:33.954 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:34.034 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:34.043 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:34.051 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:34.060 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:34.069 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:34.076 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:34.083 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T20:33:34.090 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:34.097 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:34.104 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:34.112 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:34.162 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:33:34.169 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:33:34.210 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:34.217 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:34.278 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:34.329 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:34.391 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:34.398 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:34.406 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:34.411 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:34.419 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:34.426 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:34.433 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:34.440 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:34.454 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:34.461 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:34.500 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:34.507 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:34.546 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:34.577 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:34.619 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-04-26T20:33:34.671 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-04-26T20:33:34.700 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:34.708 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:34.717 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:34.726 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:34.734 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:34.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:34.752 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:34.761 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T20:33:34.769 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:34.777 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:34.815 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:34.824 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:34.858 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:33:34.906 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:33:34.997 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:35.021 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:35.031 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:35.091 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:35.125 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:35.134 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:35.143 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:35.152 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T20:33:35.160 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:35.169 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:35.178 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:35.186 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:35.195 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:33:35.233 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:33:35.242 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:35.273 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:35.321 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:33:35.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:35.396 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:35.471 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:35.479 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:35.488 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:35.496 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:35.504 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:35.514 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:35.523 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:35.531 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:35.540 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:35.548 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:35.590 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:35.599 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:35.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:35.650 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:35.704 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:35.713 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:35.768 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:35.815 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:35.826 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:35.835 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:35.844 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:35.852 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:35.869 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:35.878 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:35.887 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:35.895 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:35.923 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T20:33:35.932 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T20:33:35.983 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:36.074 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:36.121 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:36.131 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:36.182 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:36.191 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:36.200 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T20:33:36.209 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:36.217 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:36.226 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:36.234 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:36.243 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T20:33:36.251 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T20:33:36.260 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:36.291 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:36.300 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:36.348 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-04-26T20:33:36.405 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:36.491 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:36.544 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:36.550 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T20:33:36.616 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:36.622 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:36.629 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:36.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:36.642 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-04-26T20:33:36.649 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-04-26T20:33:36.656 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:36.663 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:36.670 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:36.677 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:33:36.743 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:36.750 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:36.804 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:36.846 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-04-26T20:33:36.902 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:36.932 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:36.939 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:36.990 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:36.998 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-04-26T20:33:37.005 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-04-26T20:33:37.014 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:37.021 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:37.028 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:37.036 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:33:37.043 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:37.053 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:37.063 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:37.099 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T20:33:37.108 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:37.148 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:37.207 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:37.239 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:37.307 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-04-26T20:33:37.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-04-26T20:33:37.382 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:37.390 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:37.398 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:37.407 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:37.415 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:37.423 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:37.433 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:37.442 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-04-26T20:33:37.451 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:37.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:37.498 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:37.507 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:37.544 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-04-26T20:33:37.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-04-26T20:33:37.640 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:37.693 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:37.702 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:37.783 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-04-26T20:33:37.792 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:37.800 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:37.809 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:37.817 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-04-26T20:33:37.826 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-04-26T20:33:37.835 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:37.843 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T20:33:37.852 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:37.860 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:37.910 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:37.918 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:37.986 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-04-26T20:33:38.035 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-04-26T20:33:38.072 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:38.081 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:38.142 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:38.232 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-04-26T20:33:38.240 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:38.249 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:38.258 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:38.266 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T20:33:38.275 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:38.285 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:38.294 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:38.303 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:38.311 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-04-26T20:33:38.343 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-04-26T20:33:38.352 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:38.397 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:38.441 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:38.531 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T20:33:38.579 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:38.588 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:38.665 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:38.674 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T20:33:38.683 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:38.692 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:38.700 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:38.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:38.717 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-04-26T20:33:38.726 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-04-26T20:33:38.734 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:38.741 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:38.781 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:38.790 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:38.839 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:38.888 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:38.939 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:38.980 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T20:33:38.989 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:39.048 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:39.056 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:39.065 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:39.072 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-04-26T20:33:39.079 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-04-26T20:33:39.087 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:39.096 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:39.103 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:39.110 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:39.117 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:39.162 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:39.170 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:39.202 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T20:33:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:39.263 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:39.271 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:39.323 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:39.390 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-04-26T20:33:39.397 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-04-26T20:33:39.405 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:39.412 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:39.419 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:39.426 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:39.433 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:39.440 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:39.447 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:39.453 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T20:33:39.490 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:39.497 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:39.553 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:39.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:39.654 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-04-26T20:33:39.711 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-04-26T20:33:39.721 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:39.790 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:39.798 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:39.805 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:39.814 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:39.823 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:39.831 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:39.840 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T20:33:39.849 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:39.857 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:39.867 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:39.897 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:39.905 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-04-26T20:33:39.974 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-04-26T20:33:39.982 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:40.042 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:40.101 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:40.110 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:40.199 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:40.208 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:40.217 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:40.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T20:33:40.234 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:40.245 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:40.253 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:40.262 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:40.270 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-04-26T20:33:40.279 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-04-26T20:33:40.313 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:40.323 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:40.393 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:40.402 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-04-26T20:33:40.459 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:40.512 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:40.520 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:40.603 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-04-26T20:33:40.612 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:40.620 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:40.629 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:40.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:40.645 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-04-26T20:33:40.654 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-04-26T20:33:40.663 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:40.671 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:40.724 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:40.734 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:40.777 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:40.810 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:40.840 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T20:33:40.875 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:40.884 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:40.956 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:40.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:40.974 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-04-26T20:33:40.983 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-04-26T20:33:40.992 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:41.000 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:41.010 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:41.018 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:41.027 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:41.035 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:41.079 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T20:33:41.088 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:41.149 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:41.197 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:41.254 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-04-26T20:33:41.348 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-04-26T20:33:41.355 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:41.363 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:41.374 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:41.383 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:41.391 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:41.398 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:41.405 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:41.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T20:33:41.423 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:41.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:41.466 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:41.518 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:41.579 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-04-26T20:33:41.586 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-04-26T20:33:41.643 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:41.650 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:41.742 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:41.749 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:41.757 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:41.764 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:41.771 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:41.778 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T20:33:41.786 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:41.792 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:41.801 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:41.810 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:41.863 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-04-26T20:33:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-04-26T20:33:41.927 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:42.280 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:42.301 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:42.310 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:42.390 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:42.397 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:42.405 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T20:33:42.414 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:42.423 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:42.431 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:42.441 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:42.449 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-04-26T20:33:42.458 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-04-26T20:33:42.466 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:42.576 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:42.628 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:42.681 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:42.744 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T20:33:42.822 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:42.831 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:42.841 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:42.850 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:42.858 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:42.867 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:42.875 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:42.884 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-04-26T20:33:42.894 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:42.903 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:42.958 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:42.967 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:33:42.975 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:33:43.032 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:43.073 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:43.081 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:43.140 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:43.222 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:43.231 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:43.242 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:43.250 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:43.258 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:43.266 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:43.275 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:43.283 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:43.291 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:43.299 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:43.367 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:43.376 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:43.446 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:43.503 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:43.512 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:43.570 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:33:43.578 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:33:43.646 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:43.655 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:43.663 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:43.672 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:43.681 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:43.689 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:43.698 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:43.707 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:43.715 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:43.723 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:43.781 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:43.789 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:43.839 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:33:43.899 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:33:43.947 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:43.954 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:44.009 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:44.063 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:33:44.070 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:44.078 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:44.085 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:33:44.094 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:33:44.102 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:33:44.111 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:33:44.119 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:33:44.128 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:33:44.135 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:33:44.197 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:33:44.206 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:44.261 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:44.312 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:44.369 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:44.410 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:44.417 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:33:44.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:33:44.506 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:44.514 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:44.532 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:44.540 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:44.548 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:44.556 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:44.566 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:44.574 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:44.639 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:44.647 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:44.689 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:44.751 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-04-26T20:33:44.797 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-04-26T20:33:44.834 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:44.843 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:44.922 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:44.931 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-04-26T20:33:44.939 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:44.947 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:44.956 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:44.965 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T20:33:44.973 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:44.981 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:44.990 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:44.999 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:45.037 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-04-26T20:33:45.045 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-04-26T20:33:45.139 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:33:45.189 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:45.230 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:33:45.275 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-04-26T20:33:45.283 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:45.355 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:45.364 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:45.372 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T20:33:45.380 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:45.389 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:45.397 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:45.406 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:45.415 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:33:45.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:45.432 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:33:45.470 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-04-26T20:33:45.478 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:33:45.522 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:33:45.530 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:33:45.578 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T20:33:45.587 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T20:33:45.641 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:45.727 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:45.735 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:45.743 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:33:45.751 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:45.760 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:45.768 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:45.776 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:45.784 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:45.793 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:45.803 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:45.891 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:45.899 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T20:33:45.952 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T20:33:46.007 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:46.063 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:46.117 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:46.126 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:46.196 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:46.206 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:46.215 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:46.224 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:46.233 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:46.241 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:46.250 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:46.258 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:46.267 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T20:33:46.274 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T20:33:46.312 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:46.320 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:46.377 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:46.429 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:46.461 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:46.522 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:46.531 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:46.619 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:46.626 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:46.634 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:46.642 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:46.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:46.658 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-04-26T20:33:46.667 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T20:33:46.675 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:46.684 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:46.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:46.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:46.711 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:46.786 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:46.794 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:46.853 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:46.911 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:46.919 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:46.980 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:46.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:46.994 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T20:33:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T20:33:47.008 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:47.015 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:47.024 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:47.031 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:47.039 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:47.093 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:47.148 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:47.210 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:47.274 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:47.363 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:47.434 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:47.441 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:47.513 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T20:33:47.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T20:33:47.526 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:47.533 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:47.539 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:47.545 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:47.554 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:47.562 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:47.571 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:47.579 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:47.588 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:47.638 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:47.718 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:47.782 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:47.821 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T20:33:47.883 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T20:33:47.892 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:47.901 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:47.909 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:47.918 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:47.979 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:47.989 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:47.998 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:48.006 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:48.015 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:48.023 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:48.032 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:48.040 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:48.078 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T20:33:48.134 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T20:33:48.143 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:48.204 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:48.213 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:48.274 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:48.369 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:48.378 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:48.387 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:48.395 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:48.403 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:48.411 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:48.419 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:48.431 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:48.439 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T20:33:48.448 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T20:33:48.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:48.495 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:48.503 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:48.511 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:48.566 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:48.581 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:48.631 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:48.687 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:48.696 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:48.704 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:48.713 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:48.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:48.729 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T20:33:48.737 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T20:33:48.744 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:48.753 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:48.761 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:48.770 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:48.813 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:48.891 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:48.945 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:48.994 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:49.048 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:49.055 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:49.112 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:49.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:49.129 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T20:33:49.138 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T20:33:49.146 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:49.153 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:49.160 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:49.169 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:49.177 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:49.185 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:49.194 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:49.238 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:49.331 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:49.382 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:49.456 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:49.511 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:49.519 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T20:33:49.570 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T20:33:49.579 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:49.589 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:49.598 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:49.607 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:49.616 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:49.625 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:49.633 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:49.642 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:49.650 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:49.658 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:49.689 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:49.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:49.763 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T20:33:49.803 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T20:33:49.867 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:49.876 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:49.941 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:49.949 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:49.956 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:49.964 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:49.972 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:49.980 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:49.989 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:49.997 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:50.006 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:50.014 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:50.023 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T20:33:50.066 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T20:33:50.130 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:50.185 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:50.237 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:50.246 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:50.306 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:50.379 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:50.386 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:50.394 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:50.401 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:50.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:50.415 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:50.421 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:50.430 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T20:33:50.438 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T20:33:50.446 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:50.454 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:50.505 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:50.576 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:33:50.623 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:50.662 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:50.720 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:50.731 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:50.815 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:50.824 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:50.834 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:50.843 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:50.851 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T20:33:50.860 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T20:33:50.868 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:50.877 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:50.886 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:50.895 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:50.903 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:50.952 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:51.017 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:51.062 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:51.112 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:51.120 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:51.170 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:51.245 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:51.254 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-04-26T20:33:51.262 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-04-26T20:33:51.270 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:51.279 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:51.287 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:51.296 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:51.305 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:51.313 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:51.322 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:51.331 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:51.424 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:51.500 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:51.549 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:51.604 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:51.655 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T20:33:51.663 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T20:33:51.737 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:51.761 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:33:51.769 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:51.778 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:51.786 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:51.794 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:51.802 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:51.811 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:51.819 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:51.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:51.927 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T20:33:51.960 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T20:33:52.012 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:52.067 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:52.075 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:52.149 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:52.157 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:52.164 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:52.226 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:52.254 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:52.275 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:52.311 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:52.357 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:52.397 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:52.411 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T20:33:52.461 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T20:33:52.548 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:52.627 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:52.683 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:52.739 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:52.761 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:52.770 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:52.859 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:52.869 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:52.877 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:52.886 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:52.894 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:52.903 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:52.912 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T20:33:52.920 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T20:33:52.929 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:52.938 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:52.976 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:52.986 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:53.042 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:53.088 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:53.136 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:53.145 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:53.192 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:53.245 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:53.254 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:53.263 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:53.271 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T20:33:53.280 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T20:33:53.289 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:53.298 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:53.306 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:53.314 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:53.322 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:53.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:53.403 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:53.476 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:53.485 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:53.537 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:53.547 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:53.557 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:53.625 INFO:teuthology.orchestra.run.smithi131.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T20:33:53.632 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T20:33:53.641 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:53.649 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:53.659 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:53.682 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "[]", 2024-04-26T20:33:53.690 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:53.699 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:53.718 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:53.745 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:53.798 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:53.825 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:53.917 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:53.951 INFO:teuthology.orchestra.run.smithi131.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T20:33:53.976 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T20:33:53.984 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:54.051 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:54.059 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:54.068 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "{}", 2024-04-26T20:33:54.075 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:54.082 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:54.096 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:54.103 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:54.109 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:54.116 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:54.125 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:54.167 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:54.181 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T20:33:54.227 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T20:33:54.259 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:54.311 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:54.318 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:54.325 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:54.377 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:54.385 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:54.400 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:54.415 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:54.422 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:54.432 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:54.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:54.459 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T20:33:54.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T20:33:54.534 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:54.542 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:54.582 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:54.623 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:54.690 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:54.702 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:54.709 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:54.760 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:54.769 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:54.778 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:54.785 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:54.793 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:54.801 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T20:33:54.809 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T20:33:54.817 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:54.825 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:54.834 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:54.859 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:54.868 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:54.926 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:54.986 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:55.048 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:55.057 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:55.065 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:55.131 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:55.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:55.148 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T20:33:55.157 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T20:33:55.165 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:55.198 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:55.206 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:55.217 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:55.227 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:55.250 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:55.264 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:55.304 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:55.350 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:55.359 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:55.409 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:55.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:55.428 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T20:33:55.494 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T20:33:55.528 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:55.539 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:55.595 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:55.607 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:55.616 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:55.625 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:55.634 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:55.644 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:55.651 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:55.664 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:55.672 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:55.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:55.774 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T20:33:55.841 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T20:33:55.847 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:55.860 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:55.915 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:55.929 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:55.936 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:55.943 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:55.950 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:55.957 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:55.964 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:55.971 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:55.995 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:56.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:56.047 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T20:33:56.054 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T20:33:56.124 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:56.170 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:56.219 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:56.251 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:56.259 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:56.320 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:56.331 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:56.357 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:56.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:56.371 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:56.379 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:56.395 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:56.406 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T20:33:56.413 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T20:33:56.421 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:56.448 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:56.457 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:56.498 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:56.567 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:56.592 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:56.602 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:56.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:56.673 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:56.682 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:56.690 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:56.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:56.707 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T20:33:56.717 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T20:33:56.725 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:56.734 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:56.747 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:56.754 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:33:56.765 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:56.773 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:56.823 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:56.888 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:56.944 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:56.982 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:56.990 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:57.049 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:57.058 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T20:33:57.067 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T20:33:57.075 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:57.123 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:57.133 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:57.142 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:57.150 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:57.163 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:57.197 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:57.243 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:57.252 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:57.265 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:57.324 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:57.360 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:57.368 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T20:33:57.376 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T20:33:57.435 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:57.444 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:57.453 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:57.461 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T20:33:57.470 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:57.478 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:57.487 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:57.495 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:57.503 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:57.513 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:57.535 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:57.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:57.599 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T20:33:57.630 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T20:33:57.673 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:57.681 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:57.687 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:57.751 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:33:57.758 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:57.765 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:57.772 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:57.778 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:57.785 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:57.792 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:57.798 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:57.805 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:57.812 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T20:33:57.858 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T20:33:57.865 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:57.924 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:57.963 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:58.036 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-04-26T20:33:58.043 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:58.051 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:58.124 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:58.131 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:58.145 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:58.152 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:58.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:58.166 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T20:33:58.173 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T20:33:58.179 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:33:58.186 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:58.225 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:58.232 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-04-26T20:33:58.280 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:58.342 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:58.380 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:58.387 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:58.396 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:58.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:58.469 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:58.478 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:58.486 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T20:33:58.494 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T20:33:58.503 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:58.511 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:58.520 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:58.530 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:58.538 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:58.583 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:58.592 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:58.701 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:58.743 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:58.752 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:58.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:58.859 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T20:33:58.868 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T20:33:58.877 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:58.885 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:58.894 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:58.904 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-04-26T20:33:58.913 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:58.921 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:58.931 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:58.939 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:59.010 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:59.020 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:59.074 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:59.130 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:59.169 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T20:33:59.178 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T20:33:59.188 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:33:59.259 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:33:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:59.289 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:59.316 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:59.324 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:59.333 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:59.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:59.432 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T20:33:59.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T20:33:59.558 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:59.635 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:59.645 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:33:59.654 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:33:59.734 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:33:59.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:33:59.751 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:33:59.760 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:33:59.768 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:33:59.777 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:33:59.786 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:33:59.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:33:59.805 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T20:33:59.813 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T20:33:59.820 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:33:59.880 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:33:59.975 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:00.029 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:00.077 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:00.086 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:00.095 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:00.151 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:00.160 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:00.168 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:00.177 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:00.188 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:00.199 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T20:34:00.207 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T20:34:00.216 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:00.225 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:00.234 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:00.274 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:34:00.282 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:00.318 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:00.388 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:00.443 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:00.452 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:00.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:00.528 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:00.537 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:00.546 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T20:34:00.556 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T20:34:00.567 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:00.575 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:00.584 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:00.593 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:34:00.608 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:00.619 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:00.659 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:00.668 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:00.730 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:00.767 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:00.818 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:00.826 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:00.836 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T20:34:00.942 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T20:34:00.952 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:00.961 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:00.969 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:00.976 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:34:00.983 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:00.990 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:00.997 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:01.005 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:01.014 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:01.081 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:01.087 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:01.191 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:01.257 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-04-26T20:34:01.299 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-04-26T20:34:01.307 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:01.313 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:01.387 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:01.394 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:01.401 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:01.408 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:01.416 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:01.425 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:01.433 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:01.440 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:01.446 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:01.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:01.501 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-04-26T20:34:01.507 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-04-26T20:34:01.551 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:01.605 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:01.642 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:01.651 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:01.660 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:01.719 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:01.728 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:01.737 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:01.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:01.754 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:01.763 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:01.772 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:01.780 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-04-26T20:34:01.787 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-04-26T20:34:01.796 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:01.837 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:01.845 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:01.922 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:01.992 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:02.077 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-04-26T20:34:02.117 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:02.127 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:02.135 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:02.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:02.152 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-04-26T20:34:02.161 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-04-26T20:34:02.170 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:02.178 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:02.186 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:02.196 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-04-26T20:34:02.242 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:02.251 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:02.297 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:02.358 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:02.396 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:02.404 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:02.411 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:02.482 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:02.938 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-04-26T20:34:02.981 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-04-26T20:34:02.988 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:02.998 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:03.008 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:03.015 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:03.021 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:03.029 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:03.036 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:03.082 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:03.088 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:03.157 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:03.206 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:03.259 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:03.266 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:03.273 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:03.346 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:03.354 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:03.361 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:03.368 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:03.375 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:03.384 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:03.393 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:03.402 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:03.410 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:03.420 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:03.495 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:03.507 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:03.580 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:03.629 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:03.675 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:03.685 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:03.694 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:03.741 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:03.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:03.759 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:03.767 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:03.777 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:03.786 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:03.797 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:03.809 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:03.818 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:03.826 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:03.860 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:03.869 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:03.934 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:03.985 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:04.033 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:04.042 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:04.051 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:04.099 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:04.108 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:04.117 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:04.126 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:04.137 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:04.146 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:04.155 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:04.163 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:04.171 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:04.180 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:04.232 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:04.241 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:04.316 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:04.366 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:04.407 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:04.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:04.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:04.499 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:04.507 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:04.515 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:04.523 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:04.532 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:04.540 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:04.549 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:04.558 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:04.566 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:04.575 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:04.636 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:04.642 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:04.696 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:04.733 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:04.808 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:04.817 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:04.834 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:04.891 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-04-26T20:34:04.899 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-04-26T20:34:04.908 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:04.917 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:04.924 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:04.933 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:04.942 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:04.952 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:04.960 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:04.969 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-04-26T20:34:05.034 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:05.041 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:05.107 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:05.200 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-04-26T20:34:05.209 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T20:34:05.218 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:05.274 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:05.281 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:05.289 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:05.298 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:05.306 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:05.314 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:05.322 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:05.331 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:05.339 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:05.351 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:05.427 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:05.434 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:34:05.502 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:34:05.551 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:05.608 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:05.616 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:05.625 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-04-26T20:34:05.691 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:05.699 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:05.706 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:05.712 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:05.719 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:05.726 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:05.733 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:05.740 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:05.747 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:34:05.766 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:34:05.816 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:05.825 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:05.878 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:05.932 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-04-26T20:34:05.973 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:05.980 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:05.988 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:06.040 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:06.048 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:06.055 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:06.061 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:06.069 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:06.075 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-04-26T20:34:06.082 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-04-26T20:34:06.089 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:06.096 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:06.105 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:06.156 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:34:06.162 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:06.227 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:06.280 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:06.318 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:06.324 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:06.332 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:06.391 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:06.400 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:06.409 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-04-26T20:34:06.418 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-04-26T20:34:06.427 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:06.435 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:06.444 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:06.453 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-04-26T20:34:06.461 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:06.469 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:06.521 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:06.530 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:06.633 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:06.686 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:06.757 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:06.767 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:06.775 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-04-26T20:34:06.784 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-04-26T20:34:06.793 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:06.799 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:06.841 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:06.849 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-04-26T20:34:06.857 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:06.866 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:06.874 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:06.883 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:06.890 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-04-26T20:34:06.940 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:06.949 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:07.033 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:07.086 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:07.137 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:07.145 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:07.154 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-04-26T20:34:07.234 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-04-26T20:34:07.242 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:07.251 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:07.259 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:07.267 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-04-26T20:34:07.276 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-04-26T20:34:07.284 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-04-26T20:34:07.293 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:07.301 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-04-26T20:34:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-04-26T20:34:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-04-26T20:34:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-04-26T20:34:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-04-26T20:34:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:34:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-04-26T20:34:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:08.365 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:08.584 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:08.593 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:08.653 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:08.719 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:08.728 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:08.837 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:08.845 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:08.854 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:08.863 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:08.872 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:08.890 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:08.899 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:08.908 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:08.917 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:08.968 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:08.977 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:09.030 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:09.084 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:09.124 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:09.136 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:09.145 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:09.232 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:09.241 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:09.249 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:09.258 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:09.268 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:09.276 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:09.284 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:09.294 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:09.303 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:09.312 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:09.342 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:09.351 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:09.397 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:09.454 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:09.537 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:09.544 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:09.553 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:09.624 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:09.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:09.640 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:09.652 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:09.658 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:09.666 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:09.675 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:09.683 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:09.691 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:09.699 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:09.749 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:09.758 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:09.822 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:09.862 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:09.901 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:09.909 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:09.916 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-04-26T20:34:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-04-26T20:34:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:09.995 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:10.012 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-04-26T20:34:10.018 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:10.026 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:10.033 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:10.040 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T20:34:10.047 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:10.117 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:10.126 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:10.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-04-26T20:34:10.273 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-04-26T20:34:10.283 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:10.293 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:10.356 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:10.365 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-04-26T20:34:10.374 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:10.382 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:10.391 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:10.399 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T20:34:10.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:10.419 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:10.427 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:10.436 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:10.503 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-04-26T20:34:10.515 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-04-26T20:34:10.596 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:10.631 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-04-26T20:34:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:10.789 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:10.797 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:10.806 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-04-26T20:34:10.814 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:10.823 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:10.831 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:10.840 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:10.848 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-04-26T20:34:10.857 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-04-26T20:34:10.865 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:10.908 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:10.917 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-04-26T20:34:11.020 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:11.106 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:11.117 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:11.126 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T20:34:11.209 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:11.217 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:11.226 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:11.234 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:11.243 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-04-26T20:34:11.251 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-04-26T20:34:11.260 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:11.268 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:11.277 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:11.286 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:34:11.325 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:11.334 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:11.378 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:11.387 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T20:34:11.456 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:11.465 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:11.502 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:11.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:11.589 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:34:11.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:34:11.607 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:11.615 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:11.623 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:11.632 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:34:11.639 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:11.647 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:11.655 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:11.709 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T20:34:11.738 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:11.788 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:11.826 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:11.883 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:11.890 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-04-26T20:34:11.898 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-04-26T20:34:11.972 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:11.982 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:11.991 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:11.999 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-04-26T20:34:12.008 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:12.016 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:12.025 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:12.033 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T20:34:12.042 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:12.050 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:12.091 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:12.100 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:12.144 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:12.196 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:12.242 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:12.251 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-04-26T20:34:12.260 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:12.355 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:12.363 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:12.371 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:12.380 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:12.389 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:12.398 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:12.410 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:12.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:12.428 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:12.437 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:12.484 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:12.495 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:12.575 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:12.628 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:12.728 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:13.077 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:13.091 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:13.181 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:13.188 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:13.195 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:13.202 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:13.209 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:13.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:13.222 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:13.229 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:13.235 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:13.242 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:13.305 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:13.311 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:13.385 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:13.443 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:13.491 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:13.498 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:13.505 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:13.580 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:13.587 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:13.595 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:13.603 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:13.611 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:13.619 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:13.628 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:13.637 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:13.645 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:13.653 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:13.708 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:13.716 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:13.881 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:13.940 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:13.949 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:13.959 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:14.030 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:14.039 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:14.048 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:14.057 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:14.065 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:14.074 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:14.082 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:14.091 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:14.102 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:14.110 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:14.169 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:14.179 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:14.232 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:14.287 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:14.372 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:14.381 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:14.390 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:14.482 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:14.525 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:14.536 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:14.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:14.553 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-04-26T20:34:14.562 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-04-26T20:34:14.571 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:14.579 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:14.588 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-04-26T20:34:14.638 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:14.648 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:14.714 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:14.769 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:14.822 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:14.834 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:14.845 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:14.905 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:14.914 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-04-26T20:34:14.923 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-04-26T20:34:14.931 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:14.939 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:14.946 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:14.953 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-04-26T20:34:14.960 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:14.967 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:14.974 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:15.006 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:15.013 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:15.052 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:15.059 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:15.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:15.117 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:34:15.131 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:34:15.197 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:15.204 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:15.211 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:15.218 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-04-26T20:34:15.226 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:15.233 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:15.240 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:15.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:15.254 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:15.261 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:15.306 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:15.314 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:15.410 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:15.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:15.513 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:15.521 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-04-26T20:34:15.527 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-04-26T20:34:15.588 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-04-26T20:34:15.595 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:15.602 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-04-26T20:34:15.609 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-04-26T20:34:15.616 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:15.623 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:15.630 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:15.637 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-04-26T20:34:15.645 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:15.652 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:15.698 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:15.706 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T20:34:15.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:15.804 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:15.881 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:15.888 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:15.895 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-04-26T20:34:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-04-26T20:34:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-04-26T20:34:16.015 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:16.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:16.032 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:16.040 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T20:34:16.048 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:16.090 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:16.099 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:16.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:16.219 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-04-26T20:34:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-04-26T20:34:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:16.385 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:16.394 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:16.403 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:16.412 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:16.421 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:16.429 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-04-26T20:34:16.438 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:16.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:16.497 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:34:16.506 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:34:16.552 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-04-26T20:34:16.664 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-04-26T20:34:16.729 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:16.739 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:16.748 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T20:34:16.757 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:16.766 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:16.774 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:16.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:16.791 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:16.800 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:16.808 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:16.816 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:16.824 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:16.882 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:16.933 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:16.997 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:17.005 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:17.071 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:17.080 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:17.089 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:17.097 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:17.105 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:17.113 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:17.121 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:17.130 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:17.137 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:17.145 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:17.197 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:17.204 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:17.265 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:17.273 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:17.330 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:17.337 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:17.344 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:17.440 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:17.448 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:17.455 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:17.462 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:17.469 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:17.490 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:17.496 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:17.503 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:17.511 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:17.518 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:17.525 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:17.590 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:17.655 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:17.709 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:17.787 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:17.845 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:17.852 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:17.932 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:17.939 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:17.946 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:17.953 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:17.960 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:17.967 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:17.974 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:17.985 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:17.994 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:18.001 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:18.008 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:18.153 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:18.216 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:18.271 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:18.280 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:18.290 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:18.372 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:18.380 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:18.389 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:18.397 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:18.406 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:18.414 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:18.423 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:18.431 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:18.439 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:18.448 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-04-26T20:34:18.498 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-04-26T20:34:18.506 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:18.580 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:18.639 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:18.679 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:18.688 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:18.696 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:18.772 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:18.781 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-04-26T20:34:18.788 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:18.797 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:18.806 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:18.814 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:18.823 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-04-26T20:34:18.831 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-04-26T20:34:18.839 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:18.847 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:18.855 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:18.864 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-04-26T20:34:18.947 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:19.006 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:19.088 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:19.097 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-04-26T20:34:19.110 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:19.180 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:19.187 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:19.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:19.203 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-04-26T20:34:19.214 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-04-26T20:34:19.233 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:19.241 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:19.250 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:19.258 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-04-26T20:34:19.264 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:19.331 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:19.340 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T20:34:19.510 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:19.586 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:19.595 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:19.604 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:19.713 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-04-26T20:34:19.722 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-04-26T20:34:19.730 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:19.738 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:19.747 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:19.755 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:34:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:34:19.772 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-04-26T20:34:19.780 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:19.789 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T20:34:19.833 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:19.842 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:19.921 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:19.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:20.010 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-04-26T20:34:20.019 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-04-26T20:34:20.027 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:20.096 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:20.104 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:20.112 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:20.121 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:20.137 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:20.147 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:20.156 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-04-26T20:34:20.164 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:20.172 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:20.180 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:20.213 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:20.222 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-04-26T20:34:20.288 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-04-26T20:34:20.296 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:20.358 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:20.366 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:20.374 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-04-26T20:34:20.446 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:20.455 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:20.463 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:20.471 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T20:34:20.480 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:20.488 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:20.497 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:20.506 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:20.512 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:20.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:20.530 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:20.536 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-04-26T20:34:20.583 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:20.612 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:20.643 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:20.650 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:20.656 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:20.738 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:20.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:20.752 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:20.758 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:20.765 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:20.772 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:20.779 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:20.785 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:20.792 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:20.798 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:20.848 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:20.854 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:20.883 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:20.952 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:20.996 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:21.004 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:21.013 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:21.080 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:21.088 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:21.097 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:21.106 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:21.115 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:21.123 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:21.135 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:21.143 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:21.151 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:21.158 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:21.187 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:21.196 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:21.252 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:21.304 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:21.344 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:21.353 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:21.361 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:21.421 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:21.529 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:21.538 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:21.643 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:21.677 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:21.685 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:21.694 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:21.754 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:21.763 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:21.772 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:21.781 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:21.789 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:21.797 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:21.806 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:21.814 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:21.822 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:21.832 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:21.870 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:21.879 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:21.950 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:22.006 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:22.063 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:22.071 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:22.146 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:22.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:22.167 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:22.177 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-04-26T20:34:22.185 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:22.193 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:22.201 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:22.217 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:22.225 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:22.266 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:22.274 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:22.341 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:22.395 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:22.435 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:22.443 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:22.451 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:22.529 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:22.536 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:22.542 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:22.549 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:22.562 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:22.569 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:22.575 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:22.582 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:22.589 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:22.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:22.668 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:22.749 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:22.800 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:22.854 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:22.862 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:22.870 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:22.875 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:22.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:23.263 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:23.281 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:23.313 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:23.320 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:23.325 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:23.336 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:23.342 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:23.350 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:23.411 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:23.421 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:23.536 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:23.588 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:23.639 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:23.648 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:23.656 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:23.664 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:23.673 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:23.720 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:23.731 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:23.739 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:23.749 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:23.761 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:23.769 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:23.778 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:23.786 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:23.829 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:23.837 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:23.920 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:24.030 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:24.040 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:24.048 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:24.104 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:24.113 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:24.123 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:24.132 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:24.226 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:24.300 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-04-26T20:34:24.308 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:24.360 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:24.369 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:24.378 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-04-26T20:34:24.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-04-26T20:34:24.445 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:24.453 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:24.461 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:24.468 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-04-26T20:34:24.484 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:24.493 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:24.501 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:24.512 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T20:34:24.520 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:24.554 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:24.561 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:24.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:24.679 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-04-26T20:34:24.686 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-04-26T20:34:24.787 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:24.894 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:24.901 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:34:24.908 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-04-26T20:34:24.915 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:24.921 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:24.928 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T20:34:24.935 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:24.942 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:24.948 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:24.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:24.978 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:24.987 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:25.061 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:25.117 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:25.167 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:25.176 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:25.183 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:25.237 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:25.244 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:25.250 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:25.257 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:25.264 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:25.270 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:25.279 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:25.288 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:25.297 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:25.306 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:25.328 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:25.337 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-04-26T20:34:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-04-26T20:34:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-04-26T20:34:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-04-26T20:34:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:26.804 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-04-26T20:34:26.884 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T20:34:26.891 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:26.954 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:26.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:26.968 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:27.029 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:27.037 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:27.045 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:27.052 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:27.060 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:27.067 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:27.074 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:27.081 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:27.088 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:27.096 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:27.127 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:27.134 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:27.202 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:27.231 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:27.275 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:27.283 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:27.291 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:27.345 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:27.352 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:27.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:27.366 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:27.374 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:27.379 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:27.385 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:27.392 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:27.401 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:27.409 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:27.436 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:27.444 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:27.529 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:27.587 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:27.642 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:27.651 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:27.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:27.711 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:27.723 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:27.732 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:27.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:27.755 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:27.765 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:27.775 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:27.784 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:27.794 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:27.804 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:27.855 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:27.866 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:27.914 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:27.967 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:28.008 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:28.020 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:28.030 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:28.108 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:28.117 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:28.134 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:28.145 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:28.163 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:28.175 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:28.184 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:28.195 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:28.204 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:28.214 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:28.252 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:28.264 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:28.344 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:28.397 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:28.444 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:28.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:28.473 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:28.528 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:28.538 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-04-26T20:34:28.549 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-04-26T20:34:28.560 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:28.570 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:28.576 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:28.584 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:28.591 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:28.597 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:28.604 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:28.651 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-04-26T20:34:28.659 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T20:34:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:28.777 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:28.833 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:28.839 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-04-26T20:34:28.846 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-04-26T20:34:28.894 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:28.901 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:28.908 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:28.914 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-04-26T20:34:28.920 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:28.927 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:28.933 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:28.940 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-04-26T20:34:28.947 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:28.954 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:28.994 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:29.000 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:29.055 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-04-26T20:34:29.107 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-04-26T20:34:29.149 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:29.158 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:29.165 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:29.219 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-04-26T20:34:29.226 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:29.233 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:29.239 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:29.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T20:34:29.254 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:29.266 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:29.274 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:29.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:29.291 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-04-26T20:34:29.326 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-04-26T20:34:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:29.445 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:29.487 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:29.501 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:29.513 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:29.560 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:29.575 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-04-26T20:34:29.584 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:29.595 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:29.606 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:29.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:29.627 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-04-26T20:34:29.639 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-04-26T20:34:29.652 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:29.660 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:29.718 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:29.727 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-04-26T20:34:29.797 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:29.851 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:29.891 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:29.902 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T20:34:29.913 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T20:34:29.973 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:29.980 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:29.994 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:30.003 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:30.012 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:30.021 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:30.034 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-04-26T20:34:30.044 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:30.053 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:30.065 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:30.166 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:30.223 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:30.258 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:30.266 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:30.273 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:30.336 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:30.344 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:30.351 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:30.358 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:30.365 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:30.372 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:30.379 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:30.389 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:30.396 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:30.403 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:30.446 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:30.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:30.527 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:30.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:30.587 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:30.595 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:30.602 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:30.660 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:30.668 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:30.677 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:30.684 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:30.690 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:30.697 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:30.704 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:30.711 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:30.718 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:30.725 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:30.785 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:30.792 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:30.851 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:30.908 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:30.959 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:30.965 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:30.973 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:31.027 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:31.034 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:31.044 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:31.052 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:31.062 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:31.069 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:31.079 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:31.096 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:31.106 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:31.134 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:31.148 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:31.217 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:31.271 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:31.316 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:31.322 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:31.332 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:31.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:31.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:31.412 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:31.422 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:31.431 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:31.441 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:31.451 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:31.460 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:31.470 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:31.526 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:31.539 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:31.601 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:31.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:31.682 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:31.691 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-04-26T20:34:31.700 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:31.762 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:31.772 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:31.781 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:31.791 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:31.800 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:31.809 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:31.818 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:31.827 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:31.836 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:31.846 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:31.884 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:31.948 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:32.007 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:32.049 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:32.058 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:32.067 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:32.118 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:32.137 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:32.148 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:32.203 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:32.243 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:32.250 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:32.297 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:32.353 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:32.390 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:32.464 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:32.470 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:32.478 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:32.487 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:32.496 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:32.503 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:32.509 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:32.516 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:32.523 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:32.529 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:32.578 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:32.587 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:32.654 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:32.708 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:32.749 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:32.756 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:32.763 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:32.818 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:32.825 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:32.834 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:32.842 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:32.850 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:32.856 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:32.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:32.870 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:32.877 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:32.884 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:32.925 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:32.932 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:33.001 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:33.009 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:33.065 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:33.076 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:33.086 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:33.096 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:33.106 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:33.170 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:33.202 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:33.212 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:33.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:33.233 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:33.242 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-04-26T20:34:33.251 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:33.261 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:33.310 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:33.321 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:33.373 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:33.383 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:33.454 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:33.472 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:33.488 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:33.551 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:33.562 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:33.572 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:33.580 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:33.590 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:33.605 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:33.615 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:33.627 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:33.635 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:33.644 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:33.704 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:33.714 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:33.813 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:33.866 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:33.911 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:33.921 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:33.931 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:33.996 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:34.003 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:34.010 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:34.017 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:34.025 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:34.032 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:34.038 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:34.045 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:34.052 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:34.058 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:34.095 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:34.103 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:34.176 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:34.222 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:34.261 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:34.268 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:34.275 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:34.319 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:34.326 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:34.333 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:34.340 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:34.348 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:34.355 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:34.363 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:34.370 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:34.378 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:34.386 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:34.434 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:34.441 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:34.513 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:34.568 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:34.611 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:34.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:34.626 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:34.684 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:34.691 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:34.699 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:34.706 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:34.714 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:34.721 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:34.728 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:34.736 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:34.743 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:34.749 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:34.792 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:34.799 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:34.873 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:34.932 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:34.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:34.985 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:34.996 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-04-26T20:34:35.044 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:35.057 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:35.067 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:35.077 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-04-26T20:34:35.087 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-04-26T20:34:35.096 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:35.125 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:35.135 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:35.175 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:35.186 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:35.229 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T20:34:35.279 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:35.323 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:35.332 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:35.342 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:35.390 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:35.401 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:35.411 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:35.422 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:35.432 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:35.441 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:35.451 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:35.461 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:35.470 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:35.480 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:35.516 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:35.528 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:35.590 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:35.625 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:35.664 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:35.674 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:35.683 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:35.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:35.755 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:35.765 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:35.775 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:35.785 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:35.793 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:35.806 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:35.815 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:35.824 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:35.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:35.866 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:35.873 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:35.964 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:36.014 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:36.067 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:36.075 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:36.082 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:36.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:36.144 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:36.151 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:36.162 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:36.169 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:36.177 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:36.184 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:36.194 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:36.201 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:36.208 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:36.248 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:36.287 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:36.336 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:36.372 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:36.380 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:36.387 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:36.435 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:36.442 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:36.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:36.457 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:36.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:36.491 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:36.498 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:36.505 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:36.513 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:36.520 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:36.575 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:36.582 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:36.637 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:36.666 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:36.706 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:36.714 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:36.721 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:36.756 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:36.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:36.771 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-04-26T20:34:36.778 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-04-26T20:34:36.785 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:36.791 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:36.798 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:36.805 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:36.812 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:36.822 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:36.849 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:36.859 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-04-26T20:34:36.928 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-04-26T20:34:36.955 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-04-26T20:34:37.006 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:37.015 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-04-26T20:34:37.025 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:37.082 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:37.091 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:37.101 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:37.111 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:37.121 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:37.131 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:37.144 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-04-26T20:34:37.155 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:37.165 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:37.175 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:37.225 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:37.297 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:37.352 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:37.397 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:37.406 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:37.416 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:37.471 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:37.480 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:37.490 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:37.499 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:37.510 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:37.520 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:37.530 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:37.540 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:37.550 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:37.560 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:37.605 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:37.614 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:37.677 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:37.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:37.809 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:37.818 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:37.827 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:37.872 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:37.883 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:37.892 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:37.902 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:37.913 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:37.923 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:37.933 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:37.941 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:37.948 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:37.956 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:38.009 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:38.017 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:38.026 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:38.085 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:38.122 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:38.132 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:38.141 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:38.197 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:38.207 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:38.217 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:38.235 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:38.248 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:38.256 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:38.262 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:38.270 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:38.276 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:38.349 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:38.356 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:38.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:38.501 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:38.580 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:38.588 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:38.595 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:38.668 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:38.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:38.682 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:38.689 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:38.696 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:38.704 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:38.711 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:38.718 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:38.725 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:38.732 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:38.791 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:38.798 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:38.866 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:38.914 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:38.955 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:38.962 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-04-26T20:34:38.972 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:39.021 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:39.028 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:39.037 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:39.047 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:39.057 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:39.064 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:39.072 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:39.081 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:39.091 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:39.102 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:39.141 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:39.152 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:39.310 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:39.317 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:39.325 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:39.386 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:39.397 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:39.405 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:39.414 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:39.424 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:39.434 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:39.443 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:39.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:39.462 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:39.471 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:39.499 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:39.511 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:39.563 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:39.573 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:39.627 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:39.637 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:39.648 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:39.700 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:39.712 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:39.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:39.735 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:39.747 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:39.758 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:39.767 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:39.790 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:39.799 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:39.826 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:39.836 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:39.863 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:39.904 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:39.946 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:39.955 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:39.965 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:40.017 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:40.026 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:40.036 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:40.044 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:40.055 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:40.064 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:40.073 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:40.082 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:40.090 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:40.098 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:40.132 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:40.199 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:40.245 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:40.288 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:40.296 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:40.304 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:40.354 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:40.362 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:40.371 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:40.379 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:40.388 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:40.396 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:40.404 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:40.412 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-04-26T20:34:40.421 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:40.429 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:40.471 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:40.479 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:40.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:40.563 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:40.605 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:40.614 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:40.621 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:40.687 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:40.694 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:40.701 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:40.708 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:40.715 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:40.722 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:40.729 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:40.738 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:40.746 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:40.782 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:40.792 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:40.849 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:40.905 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:40.970 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:40.979 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:40.989 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:41.036 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:41.046 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:41.054 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:41.064 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:41.073 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:41.083 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:41.093 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:41.101 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:41.112 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:41.156 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:41.166 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:41.221 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:41.229 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:41.288 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:41.296 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:41.304 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:41.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:41.357 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:41.365 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:41.374 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:41.384 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:41.393 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:41.402 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:41.411 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:41.420 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:41.429 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:41.456 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:41.466 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:41.520 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:41.549 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:41.593 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:41.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:41.611 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:41.653 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:41.662 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:41.671 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:41.679 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:41.688 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:41.697 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:41.705 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:41.714 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:41.723 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:41.733 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:41.765 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:41.773 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:41.819 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:41.862 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:34:41.904 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:34:41.913 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:41.921 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:41.984 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:41.992 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:34:42.001 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:42.010 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:42.019 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:42.028 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:42.036 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:42.044 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:42.053 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:42.062 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:42.093 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-04-26T20:34:42.101 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-04-26T20:34:42.152 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:34:42.167 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:42.228 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-04-26T20:34:42.236 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-04-26T20:34:42.286 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:42.295 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:42.304 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-04-26T20:34:42.313 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T20:34:42.322 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:42.331 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:42.340 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:42.348 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:42.356 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:42.365 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:42.411 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-04-26T20:34:42.421 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:42.479 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:42.539 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:42.579 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-04-26T20:34:42.588 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-04-26T20:34:42.597 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:42.658 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:42.667 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:42.675 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:42.684 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:42.692 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:42.700 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:42.708 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T20:34:42.717 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:42.726 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:42.735 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:42.786 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:42.794 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-04-26T20:34:42.852 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-04-26T20:34:42.878 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:42.935 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:42.947 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:42.956 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:34:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:43.056 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:43.064 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:43.075 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:43.084 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:43.095 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:43.105 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:43.115 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:43.125 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:43.173 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:43.183 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:43.237 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:43.291 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:43.346 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:43.353 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:43.363 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:43.502 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:43.528 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:43.537 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:43.547 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:43.582 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:43.590 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:43.598 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:43.607 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:43.618 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:43.629 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:43.693 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:43.700 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:43.761 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:43.788 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:43.840 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:43.847 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:43.856 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:43.913 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:43.920 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:43.929 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:43.939 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:43.948 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:43.957 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:43.966 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:43.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:43.994 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:44.026 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:44.034 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:44.105 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:44.160 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:44.200 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:44.207 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:44.214 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:44.270 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:44.277 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:44.289 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:44.296 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:44.303 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:44.310 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:44.317 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:44.326 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:44.334 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:44.343 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:44.384 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:44.393 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:44.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:44.498 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:44.542 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:44.552 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:44.560 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:44.616 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:44.625 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:44.634 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:44.642 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:44.652 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:44.661 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:44.671 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:44.680 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:44.689 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:44.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:44.750 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-04-26T20:34:44.759 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-04-26T20:34:44.817 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:44.870 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:44.923 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:44.932 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-04-26T20:34:44.940 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:44.991 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:45.001 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:45.009 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-04-26T20:34:45.018 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:45.026 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:45.036 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:45.044 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-04-26T20:34:45.061 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-04-26T20:34:45.069 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:34:45.113 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:45.123 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:45.182 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-04-26T20:34:45.233 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:45.279 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:45.289 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:45.301 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-04-26T20:34:45.349 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:45.359 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:45.368 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:45.378 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:45.388 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:45.397 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:45.407 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:45.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-04-26T20:34:45.426 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:45.436 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:45.463 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:45.472 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-04-26T20:34:45.524 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-04-26T20:34:45.580 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:45.628 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:45.637 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:45.647 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:34:45.697 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:45.707 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:45.715 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:45.724 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:45.734 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:45.743 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:45.754 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:45.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:45.771 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-04-26T20:34:45.779 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-04-26T20:34:45.816 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:45.823 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:45.886 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:34:45.990 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:45.999 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:46.009 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:46.058 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T20:34:46.067 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T20:34:46.075 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:46.081 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:46.090 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:46.099 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:46.108 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:46.118 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:46.125 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:46.135 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:46.172 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:46.184 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:46.238 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:46.266 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:46.311 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:46.318 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:46.420 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:46.427 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:46.434 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:46.442 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:46.448 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:46.456 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:46.463 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:46.470 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:46.478 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:46.485 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:46.534 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:46.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:46.606 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:46.646 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:46.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:46.710 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:46.717 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:46.769 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:46.776 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:46.784 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:46.791 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:46.798 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:46.806 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:46.813 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:46.820 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:46.826 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:46.833 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:46.840 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:46.897 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:46.942 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:46.992 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:47.041 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:47.051 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:47.060 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:47.126 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:47.135 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:47.142 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:47.151 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:47.160 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:47.169 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:47.178 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:47.186 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:47.195 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:47.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:47.212 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:47.271 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:47.325 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:47.373 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:47.424 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:47.434 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:47.444 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:47.494 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:47.514 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:47.523 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:47.535 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:47.546 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:47.555 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:47.565 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:47.575 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-04-26T20:34:47.585 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-04-26T20:34:47.595 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:47.635 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:47.644 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:47.706 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:47.762 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:47.816 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:47.824 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:47.834 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:47.890 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:47.899 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:47.908 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:47.917 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:47.927 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T20:34:47.936 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T20:34:47.945 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:47.955 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:47.963 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:47.972 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-04-26T20:34:48.018 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:48.029 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:48.077 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:48.133 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:48.191 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:48.199 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:48.208 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:48.266 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:48.278 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-04-26T20:34:48.287 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-04-26T20:34:48.297 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:34:48.305 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:48.315 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:48.323 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-04-26T20:34:48.332 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:48.340 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:48.349 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:48.401 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:48.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:48.474 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:48.560 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:48.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:48.677 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:34:48.695 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:34:48.758 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:48.766 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:48.772 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:48.779 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-04-26T20:34:48.786 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:48.792 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:48.799 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:48.806 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T20:34:48.813 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:48.821 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:48.885 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:48.892 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:48.965 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:34:49.019 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:34:49.061 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:49.068 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:49.075 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:49.140 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-04-26T20:34:49.149 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:49.159 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:49.167 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:49.175 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T20:34:49.185 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:49.194 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:49.204 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:49.214 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:49.223 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-04-26T20:34:49.259 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-04-26T20:34:49.267 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:49.307 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:49.314 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:49.377 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-04-26T20:34:49.385 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:49.395 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:49.448 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:49.458 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:49.468 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:49.477 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:49.488 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:49.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:49.506 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-04-26T20:34:49.516 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-04-26T20:34:49.526 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:49.535 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:49.583 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:49.591 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-04-26T20:34:49.638 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:49.671 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:49.715 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:49.724 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-04-26T20:34:49.733 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-04-26T20:34:49.775 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:49.794 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:49.804 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:49.813 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:49.823 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:49.833 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-04-26T20:34:49.841 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-04-26T20:34:49.851 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:49.861 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:49.870 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:49.880 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-04-26T20:34:49.935 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-04-26T20:34:49.993 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-04-26T20:34:50.041 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:50.050 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:50.059 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:50.108 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:50.116 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:50.125 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:50.133 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:50.151 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:50.159 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-04-26T20:34:50.167 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:50.175 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:50.183 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:50.220 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:50.229 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:50.291 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:50.348 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:50.390 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:50.399 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:50.407 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:50.490 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:50.498 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:50.507 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:50.515 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:50.523 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:50.533 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:50.541 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:50.549 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:50.561 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:50.568 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:50.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:50.626 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:50.693 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:50.799 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:50.807 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:50.814 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:50.878 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:50.885 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:50.892 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:50.900 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:50.907 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:50.913 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:50.920 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:50.927 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:50.934 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:50.941 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:50.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:50.990 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:51.036 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:51.075 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:51.140 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:51.149 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:51.216 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:51.226 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:51.235 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:51.244 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:51.254 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:51.264 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:51.274 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:51.281 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:51.291 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:51.300 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:51.343 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:51.352 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:51.403 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:51.463 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:51.549 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:51.561 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:51.570 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:51.624 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:51.634 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:51.645 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:51.655 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:51.665 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:51.674 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:51.684 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:51.694 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:51.704 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:51.737 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:51.794 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-04-26T20:34:51.860 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-04-26T20:34:51.915 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:34:51.925 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:51.935 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:51.974 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-04-26T20:34:51.983 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:51.993 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:52.001 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:52.011 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:52.021 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:52.030 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:52.040 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:52.048 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:52.058 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-04-26T20:34:52.095 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T20:34:52.106 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:52.166 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:52.224 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:52.274 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:52.285 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:52.294 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:52.332 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:52.340 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:52.349 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:52.357 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:52.365 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:52.374 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:52.385 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-04-26T20:34:52.394 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-04-26T20:34:52.402 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:52.410 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:52.458 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:52.466 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:52.502 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:52.562 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:52.606 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:52.616 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:52.625 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:52.664 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:52.673 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:52.682 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:52.690 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:52.697 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:52.705 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:52.713 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-04-26T20:34:52.720 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:52.727 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:52.734 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:52.775 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-04-26T20:34:52.782 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-04-26T20:34:52.846 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:52.901 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:52.911 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:52.963 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:34:52.971 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:53.025 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:53.033 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:53.040 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:53.050 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:53.060 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:53.070 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:53.080 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:53.090 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-04-26T20:34:53.100 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-04-26T20:34:53.110 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:53.187 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:53.197 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:53.243 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:53.297 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:53.366 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:53.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:53.386 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:53.485 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:53.499 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:53.508 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:53.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:53.533 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:53.543 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:53.554 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:53.565 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:53.577 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:53.589 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:53.638 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:53.702 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:53.753 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:53.824 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:53.834 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:53.897 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:53.907 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:53.916 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:53.927 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:53.938 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:53.950 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:53.961 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:53.976 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:53.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:53.998 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:54.054 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:54.068 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:54.079 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:54.145 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:54.156 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:54.167 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:54.214 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:54.261 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:54.271 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:54.281 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:54.291 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:54.301 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:54.311 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:54.320 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:54.330 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:54.341 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:54.349 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:54.409 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:54.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:54.429 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:54.479 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:54.524 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:54.533 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:54.545 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:54.606 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:54.615 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:54.624 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:54.633 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:54.642 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:54.653 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:54.661 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:54.669 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:54.679 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:54.688 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:54.728 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:54.738 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:54.747 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:54.802 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:54.860 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:54.870 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:54.880 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:54.928 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:54.938 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:54.948 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:54.957 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:54.967 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:54.976 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:54.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:54.995 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-04-26T20:34:55.005 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-04-26T20:34:55.014 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:55.041 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:55.050 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:55.060 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:55.116 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:55.163 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:55.174 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:55.183 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:55.219 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:55.228 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:55.236 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:55.244 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:55.252 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-04-26T20:34:55.261 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-04-26T20:34:55.269 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:55.277 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:55.285 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:55.292 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:55.315 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:55.393 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:55.448 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:55.457 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:55.464 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:55.542 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:55.552 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:55.562 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:55.575 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:55.585 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-04-26T20:34:55.592 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:55.602 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:55.612 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:55.624 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:55.633 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:55.685 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:55.695 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:55.706 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:55.779 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:55.830 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:55.843 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:55.852 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:55.921 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:55.933 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:55.944 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:55.956 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:55.968 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:55.982 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:55.994 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:56.005 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:56.017 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:56.028 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:56.062 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:56.075 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:56.085 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:56.167 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:56.215 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:56.227 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:56.238 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:56.286 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:56.301 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:56.336 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:56.347 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:56.359 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:56.374 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:56.386 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:56.395 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:56.403 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:56.415 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:56.449 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:56.460 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:56.571 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:56.583 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:56.594 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:56.649 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:56.663 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:56.672 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:56.681 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:56.692 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:56.704 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:56.716 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:56.724 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:56.737 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:56.749 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:56.790 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:56.802 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:56.816 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:56.868 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:56.913 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:56.922 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:56.930 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:56.942 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:56.986 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:56.997 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:57.008 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:57.020 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:57.031 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:57.043 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:57.067 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:57.078 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:57.103 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:57.113 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:57.121 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:57.175 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-04-26T20:34:57.186 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:57.198 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:57.247 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:57.285 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:57.299 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:57.310 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:57.321 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:57.332 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:57.343 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:57.353 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:57.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:57.373 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:57.384 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:57.403 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:57.413 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:57.422 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:57.484 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:57.542 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:57.552 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:57.561 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:57.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:57.627 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:57.635 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:57.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:57.652 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:57.660 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:57.668 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:57.676 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:57.684 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:57.692 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:57.712 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:57.723 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:57.731 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:57.776 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:57.821 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:57.828 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:57.836 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:57.906 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:57.915 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:57.923 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:57.931 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:57.939 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:57.949 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:57.957 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:57.982 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:57.998 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:58.005 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:58.014 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:58.073 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:58.121 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:58.130 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:58.141 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:58.190 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:58.200 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:58.210 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:58.222 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:58.234 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:58.242 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:34:58.252 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:34:58.262 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:58.272 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:58.283 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:58.303 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:58.313 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:58.322 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:58.375 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:58.430 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:58.439 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:58.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:58.506 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:58.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:58.525 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-04-26T20:34:58.536 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-04-26T20:34:58.545 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:58.555 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:58.564 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:34:58.573 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-04-26T20:34:58.583 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:58.593 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:58.619 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T20:34:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:58.696 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:58.763 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:58.770 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:58.779 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:34:58.818 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:58.828 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:34:58.838 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-04-26T20:34:58.848 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:34:58.857 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:34:58.867 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:34:58.876 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:34:58.887 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:34:58.896 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:58.906 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:34:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:59.042 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:59.130 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:59.140 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:34:59.150 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:34:59.160 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:34:59.170 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:34:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:34:59.250 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:34:59.262 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:34:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:59.320 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:59.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:59.385 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:34:59.396 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:34:59.408 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:34:59.465 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:59.539 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:34:59.564 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:59.613 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:59.624 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:34:59.635 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:34:59.650 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:34:59.664 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:34:59.673 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:34:59.684 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:34:59.693 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:34:59.707 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:34:59.716 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:34:59.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:34:59.754 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:34:59.765 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:34:59.831 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:34:59.880 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:34:59.892 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:34:59.903 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:00.001 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:00.013 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:00.025 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:00.033 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:00.045 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:00.056 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:00.067 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:00.078 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:00.087 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:00.097 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:00.160 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:00.170 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:00.182 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:00.237 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:00.306 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:00.319 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:00.329 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:00.413 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:00.423 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:00.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:00.442 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:00.452 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:00.462 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:00.472 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:00.481 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:00.491 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-04-26T20:35:00.501 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-04-26T20:35:00.543 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:00.551 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:00.560 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:00.641 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:00.829 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:00.842 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:00.853 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:00.935 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:00.948 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:00.957 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:00.967 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:00.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:00.989 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-04-26T20:35:01.001 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-04-26T20:35:01.010 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:01.021 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:01.033 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:01.083 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-04-26T20:35:01.095 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:01.106 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:01.193 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:01.257 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:01.267 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:01.305 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:01.314 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:01.324 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-04-26T20:35:01.333 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-04-26T20:35:01.343 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:01.354 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:01.365 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:01.374 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:01.384 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:01.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:01.428 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:01.437 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:01.445 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:01.505 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:01.554 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:01.564 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:01.573 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-04-26T20:35:01.665 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-04-26T20:35:01.675 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:35:01.685 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:01.696 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:01.705 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:01.714 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:01.724 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:01.732 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:01.740 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:01.748 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:01.777 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:01.787 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:01.797 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:01.806 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-04-26T20:35:01.865 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-04-26T20:35:01.875 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-04-26T20:35:01.885 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:01.957 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:01.969 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:01.978 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:01.987 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:01.997 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:02.007 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:02.016 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:02.026 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:02.087 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-04-26T20:35:02.096 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-04-26T20:35:02.108 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:02.168 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:02.228 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:02.237 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:02.246 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:02.299 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:02.308 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:02.318 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:02.327 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:02.337 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:02.347 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:02.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:02.367 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-04-26T20:35:02.376 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-04-26T20:35:02.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:02.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:02.404 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:02.414 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:02.459 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:02.505 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:02.515 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:02.524 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:02.591 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:02.601 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:02.609 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:02.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:02.628 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-04-26T20:35:02.636 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-04-26T20:35:02.646 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:02.657 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:02.666 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:02.690 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:35:02.727 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:02.737 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:02.791 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:02.846 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:02.855 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:02.864 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:02.925 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:02.934 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-04-26T20:35:02.943 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-04-26T20:35:02.953 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:35:02.962 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:02.974 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:02.985 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:02.997 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-04-26T20:35:03.008 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-04-26T20:35:03.020 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:03.074 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:03.082 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:03.093 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:03.166 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:03.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:03.233 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-04-26T20:35:03.244 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-04-26T20:35:03.284 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:03.296 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:03.307 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:03.321 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:03.333 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:03.344 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:03.358 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:03.372 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:03.384 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:03.396 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:03.435 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:03.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-04-26T20:35:03.525 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-04-26T20:35:03.579 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:03.590 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:03.601 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:03.651 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:03.658 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:03.665 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:03.678 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:03.688 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:03.698 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:03.710 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:03.720 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:03.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:03.740 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-04-26T20:35:03.785 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-04-26T20:35:03.795 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:03.805 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:03.862 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:03.921 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:03.933 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:03.944 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:03.984 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:03.993 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:04.004 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:04.014 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:04.026 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:04.037 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:04.048 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:04.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:04.068 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:04.080 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-04-26T20:35:04.127 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:04.138 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:04.149 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:04.220 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-04-26T20:35:04.271 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-04-26T20:35:04.283 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:04.294 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:04.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:04.362 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:04.377 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:04.390 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:04.402 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:04.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T20:35:04.424 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:04.438 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:04.449 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:04.459 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:04.525 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-04-26T20:35:04.533 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-04-26T20:35:04.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:04.603 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:04.654 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:04.664 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:04.674 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:04.719 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:04.729 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:04.739 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-04-26T20:35:04.748 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:04.758 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:04.768 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:04.778 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:04.786 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:04.795 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:04.804 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:04.835 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:04.844 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:04.854 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:04.903 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:04.954 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:04.961 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:04.969 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:05.011 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:05.021 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:05.030 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:05.040 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:05.050 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:05.060 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:05.070 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:05.079 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:05.089 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:05.098 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:05.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:05.239 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:05.304 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:05.314 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:05.324 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:05.377 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:05.386 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:05.396 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:05.405 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:05.415 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:05.424 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:05.434 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:05.444 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:05.453 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:05.462 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:05.492 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:05.501 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:05.511 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:05.564 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:05.612 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:05.622 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:05.633 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:05.683 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:05.693 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:05.703 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:05.714 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:05.723 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:05.731 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:05.738 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:05.746 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:05.754 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:05.762 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:05.812 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:05.838 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:05.846 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:05.915 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:05.971 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:05.979 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:05.988 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:06.034 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:06.042 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:06.049 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:06.057 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:06.065 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:06.073 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:06.081 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:06.088 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:06.098 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:06.108 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:06.117 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:06.128 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:06.151 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-04-26T20:35:06.228 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:06.279 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:06.290 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:06.301 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:06.350 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:06.365 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:06.376 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:06.386 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:06.397 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:06.409 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:06.420 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:06.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:06.446 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:06.458 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:06.485 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:06.496 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:06.510 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:06.589 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:06.645 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:06.657 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:06.669 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:06.721 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:06.733 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:06.745 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:06.756 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:06.768 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:06.777 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:06.797 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:06.809 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:06.825 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:06.858 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:06.870 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:06.881 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:06.953 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:07.012 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:07.024 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:07.033 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:07.083 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:07.094 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:07.105 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:07.117 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:07.129 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:07.141 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:07.152 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:07.162 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:07.173 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:07.184 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:07.218 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:07.233 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:07.241 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-04-26T20:35:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:08.505 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:08.513 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:08.521 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:08.529 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:08.545 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:08.553 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:08.561 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:08.600 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:08.607 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:08.617 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:08.695 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:08.756 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:08.766 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:08.777 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:08.825 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:08.835 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:08.845 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:08.854 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:08.864 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:08.875 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:08.885 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:08.895 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:08.905 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:08.913 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:08.950 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:08.960 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:08.995 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:09.099 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:09.166 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:09.184 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:09.272 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:09.282 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:09.292 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:09.301 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:09.311 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:09.320 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:09.342 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:09.352 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:09.362 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:09.409 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:09.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:09.428 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:09.469 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-04-26T20:35:09.517 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:09.527 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:09.537 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:09.598 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-04-26T20:35:09.608 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-04-26T20:35:09.618 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:09.629 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:09.639 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:09.654 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T20:35:09.669 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:09.680 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:09.692 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:09.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:09.734 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:09.744 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:09.752 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:09.842 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:09.885 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:35:09.896 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:35:09.904 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:35:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:09.991 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:10.003 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-04-26T20:35:10.017 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:10.029 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:10.041 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:10.053 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:10.065 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:10.076 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:10.088 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:10.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:10.165 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:10.173 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:10.257 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:10.300 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:10.310 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:10.322 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:10.376 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:10.388 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:10.400 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:10.412 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:10.423 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:10.434 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:10.447 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:10.454 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:10.471 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:10.513 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:10.523 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:10.533 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:10.651 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:10.775 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:10.782 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:10.789 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:10.799 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:10.808 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:10.815 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:10.825 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:10.834 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:10.844 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:10.853 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:10.884 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:10.894 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:10.903 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:11.026 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:11.036 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:11.048 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:11.055 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:11.063 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:11.124 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:11.134 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:11.141 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:11.148 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:11.157 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:11.166 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:11.178 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:11.190 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:11.258 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:11.270 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:11.279 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:11.344 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:11.436 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:11.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:11.456 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:11.515 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:11.526 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:11.535 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:11.545 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:11.556 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:11.565 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:11.574 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:11.583 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:11.601 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:11.611 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:11.637 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:11.650 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:11.660 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:11.724 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:11.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:11.793 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:11.802 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-04-26T20:35:11.852 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:11.862 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:11.871 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:11.881 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-04-26T20:35:11.890 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-04-26T20:35:11.900 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:11.910 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:11.920 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:11.930 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:35:11.940 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:11.969 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:11.979 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:11.988 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T20:35:12.049 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:12.100 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:12.108 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:12.117 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:12.169 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-04-26T20:35:12.181 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-04-26T20:35:12.190 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:12.200 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:12.210 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:12.221 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:35:12.231 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:12.275 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:12.285 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:12.295 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T20:35:12.333 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:12.342 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:12.351 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:12.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:12.457 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-04-26T20:35:12.468 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-04-26T20:35:12.477 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:12.522 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:12.542 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:12.552 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:35:12.562 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:12.572 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:12.582 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:12.592 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T20:35:12.600 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:12.610 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:12.619 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:12.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:12.646 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-04-26T20:35:12.657 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-04-26T20:35:12.714 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:12.808 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:12.818 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:12.826 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:12.865 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:12.873 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:12.881 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:12.887 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T20:35:12.896 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:12.903 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:12.911 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:12.919 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:12.927 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-04-26T20:35:12.936 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-04-26T20:35:12.962 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:12.971 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:12.979 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:13.037 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:13.082 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:13.090 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:13.098 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:13.140 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T20:35:13.148 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:13.157 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:13.165 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:13.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:13.181 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-04-26T20:35:13.190 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-04-26T20:35:13.198 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:13.206 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:13.218 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:13.260 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:13.268 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:13.278 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:13.335 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:13.391 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:13.401 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:13.412 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:13.457 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:13.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:13.480 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-04-26T20:35:13.491 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-04-26T20:35:13.507 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:13.518 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:13.530 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:13.541 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:13.554 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:13.565 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:13.593 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:13.605 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:13.616 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:13.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:13.732 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:13.744 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:13.756 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-04-26T20:35:13.797 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-04-26T20:35:13.808 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:13.820 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:13.828 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T20:35:13.851 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:13.861 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:13.872 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:13.885 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:13.896 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:13.930 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:13.941 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:13.953 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:14.039 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-04-26T20:35:14.100 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-04-26T20:35:14.106 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:14.116 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:14.167 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:14.178 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:14.190 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:14.204 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:14.216 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:14.227 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:14.238 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:14.250 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:14.262 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:14.273 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:14.299 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:35:14.308 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:35:14.319 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:35:14.381 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:14.424 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:14.435 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-04-26T20:35:14.446 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-04-26T20:35:14.507 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:14.519 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:14.530 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:14.544 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:14.556 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:14.567 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:14.579 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:14.587 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-04-26T20:35:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-04-26T20:35:14.608 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:35:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:14.638 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:14.652 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-04-26T20:35:14.722 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:14.774 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:14.786 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:14.797 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:14.848 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:14.859 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:14.870 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:14.881 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:14.893 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-04-26T20:35:14.904 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-04-26T20:35:14.915 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:14.926 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:14.938 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:14.946 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:14.962 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:14.973 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:14.987 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:15.050 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:15.107 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:15.117 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:15.127 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:15.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:15.188 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-04-26T20:35:15.200 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-04-26T20:35:15.208 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:15.216 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:15.224 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:15.231 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:15.238 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:15.246 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:15.254 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:15.282 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:15.290 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:15.298 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:15.370 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:15.424 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:15.433 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:15.440 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:15.532 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:15.560 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:15.568 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:15.577 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:15.585 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:15.592 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:15.600 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:15.608 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:15.616 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:15.624 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:15.649 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:15.658 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:15.666 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:15.709 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:15.748 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:15.756 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:15.766 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:15.805 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:15.815 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:15.824 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:15.835 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:15.845 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:15.866 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:15.876 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:15.885 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:15.895 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:15.922 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:15.932 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:15.942 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:16.082 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:16.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:16.101 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:16.167 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:16.177 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:16.186 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:16.195 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:16.204 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:16.213 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:16.225 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:16.234 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:16.243 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:16.253 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:16.274 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:16.363 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:16.406 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:16.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:16.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:16.466 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:16.476 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:16.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:16.494 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:16.504 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:16.515 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:16.524 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:16.533 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:16.543 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:16.583 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:16.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:16.660 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:16.752 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:16.764 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:16.778 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:16.839 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-04-26T20:35:16.848 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-04-26T20:35:16.860 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:16.871 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:16.882 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:16.893 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:16.901 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:16.910 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:16.921 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:16.933 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:16.974 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:16.998 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:17.064 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:17.111 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-04-26T20:35:17.123 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-04-26T20:35:17.134 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:17.180 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:17.189 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:17.198 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:17.209 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:17.220 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:17.232 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:17.246 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:17.269 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:17.281 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:17.299 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:17.310 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-04-26T20:35:17.321 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-04-26T20:35:17.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:17.457 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:17.467 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:17.477 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T20:35:17.522 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:17.532 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:17.541 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:17.551 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:17.561 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:17.571 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:17.581 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:17.591 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:17.603 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:17.610 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:17.647 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:17.656 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-04-26T20:35:17.666 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:17.726 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:17.781 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:17.792 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:17.847 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:17.858 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:17.867 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:17.877 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:17.884 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:17.892 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:17.901 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:17.911 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:17.920 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:17.933 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:17.965 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:17.974 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:17.987 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:18.047 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:18.109 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:18.121 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:18.134 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:18.199 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:18.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:18.225 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:18.237 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:18.248 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:18.259 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:18.274 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:18.285 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:18.296 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:18.307 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:18.337 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:18.349 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:18.361 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:18.427 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:18.464 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:18.476 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:18.487 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:18.539 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:18.550 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:18.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:18.576 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:18.625 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:18.637 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:18.647 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:18.657 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:18.666 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:18.675 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:18.705 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:18.715 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:18.725 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:18.790 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:18.857 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:18.867 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:18.877 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:18.952 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:18.965 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:18.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:18.985 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:18.994 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:19.003 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:19.014 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:19.024 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:19.034 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:19.044 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:19.079 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:19.089 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:19.101 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:19.165 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:19.215 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:19.224 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:19.242 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:19.288 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:19.299 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:19.308 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:19.318 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-04-26T20:35:19.328 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:19.340 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:19.349 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:19.360 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:19.369 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:19.379 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:19.408 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:19.418 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:19.429 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:19.589 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:19.597 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:19.606 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:19.655 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:19.664 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:19.673 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:19.682 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:19.690 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:19.700 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:19.708 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:19.718 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:19.728 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:19.738 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:19.777 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:19.785 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:19.794 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:19.855 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:19.889 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:19.897 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:19.904 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:19.955 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:19.964 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:19.973 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:19.981 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:19.992 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:20.004 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:20.013 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:20.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:20.031 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:20.038 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:20.077 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:20.087 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:20.096 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:20.160 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:20.205 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:20.215 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:20.225 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:20.271 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:20.281 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:20.291 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:20.304 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:20.312 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:20.319 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:20.342 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:20.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:20.362 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:20.385 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:20.395 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:20.405 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:20.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:20.531 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:20.544 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:20.554 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:20.621 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:20.633 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:20.648 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:20.656 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:20.669 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:20.679 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:20.688 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:20.696 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:20.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:20.715 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-04-26T20:35:20.756 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-04-26T20:35:20.768 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:35:20.779 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:20.837 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:20.880 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-04-26T20:35:20.889 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:20.903 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:20.968 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:20.979 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T20:35:20.990 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:21.002 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:21.017 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:21.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:21.042 INFO:teuthology.orchestra.run.smithi131.stdout: "periodic_async_work": { 2024-04-26T20:35:21.056 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "periodic_async_work", 2024-04-26T20:35:21.071 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:21.082 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:21.110 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:21.122 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:21.131 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:21.190 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:21.239 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:21.250 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Periodically check for async work", 2024-04-26T20:35:21.262 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:21.312 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:21.324 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:21.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:21.346 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-04-26T20:35:21.357 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-04-26T20:35:21.365 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:35:21.374 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:21.385 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:21.397 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-04-26T20:35:21.408 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T20:35:21.524 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:21.575 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:21.586 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:21.646 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-04-26T20:35:21.656 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T20:35:21.666 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:21.678 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:21.688 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:21.716 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-04-26T20:35:21.725 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:21.735 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:21.746 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:21.755 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T20:35:21.775 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:21.785 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:21.794 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:21.849 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:21.897 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:21.907 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:21.916 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:21.954 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-04-26T20:35:21.963 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-04-26T20:35:21.973 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-04-26T20:35:21.983 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-04-26T20:35:21.992 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-04-26T20:35:22.003 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-04-26T20:35:22.013 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-04-26T20:35:22.023 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:22.031 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:22.039 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-04-26T20:35:22.059 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:22.067 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:22.074 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:22.127 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:22.185 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:22.192 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:22.200 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:22.261 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:22.268 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-04-26T20:35:22.275 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-04-26T20:35:22.282 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:22.288 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:22.296 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:22.302 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:22.310 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:22.317 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:22.324 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:22.361 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:22.368 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:22.375 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:22.437 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:22.489 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:22.496 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-04-26T20:35:22.503 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-04-26T20:35:22.547 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-04-26T20:35:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:22.563 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:22.570 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-04-26T20:35:22.577 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:22.584 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:22.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:22.598 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:22.606 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:22.613 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:22.643 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:22.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:22.659 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-04-26T20:35:22.717 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-04-26T20:35:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:22.769 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:22.823 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:22.831 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:22.838 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:22.847 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:22.855 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:22.864 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:22.872 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:22.883 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:22.892 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:22.903 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:22.944 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:22.953 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:22.962 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:23.033 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:23.089 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:23.098 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:23.106 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-04-26T20:35:23.160 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-04-26T20:35:23.169 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:23.177 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:23.185 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:23.194 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:23.203 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:23.212 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:23.220 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:23.229 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:23.237 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:23.273 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:23.282 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:23.291 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:23.368 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-04-26T20:35:23.404 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-04-26T20:35:23.413 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:23.422 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:23.483 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:23.492 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-04-26T20:35:23.500 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:23.509 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:23.518 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-04-26T20:35:23.527 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-04-26T20:35:23.535 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-04-26T20:35:23.544 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-04-26T20:35:23.552 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-04-26T20:35:23.561 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-04-26T20:35:23.594 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-04-26T20:35:23.601 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:23.609 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:23.666 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:23.722 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:23.730 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:23.739 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:23.789 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-04-26T20:35:23.799 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-04-26T20:35:23.808 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-04-26T20:35:23.816 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:23.824 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-04-26T20:35:23.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-04-26T20:35:23.842 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:23.850 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:23.858 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:23.866 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:23.893 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:23.901 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:23.910 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:24.015 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-04-26T20:35:24.024 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-04-26T20:35:24.034 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-04-26T20:35:24.102 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:24.111 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:24.120 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:24.128 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:24.135 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-04-26T20:35:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-04-26T20:35:24.261 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-04-26T20:35:24.313 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:24.321 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:24.329 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-04-26T20:35:24.362 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:24.369 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:24.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:24.384 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:24.391 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:24.398 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:24.405 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:24.439 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:24.446 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-04-26T20:35:24.455 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-04-26T20:35:24.489 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-04-26T20:35:24.496 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-04-26T20:35:24.503 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-04-26T20:35:24.559 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T20:35:24.613 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-04-26T20:35:24.623 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-04-26T20:35:24.630 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-04-26T20:35:24.692 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-04-26T20:35:24.700 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-04-26T20:35:24.707 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-04-26T20:35:24.714 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-04-26T20:35:24.721 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:24.742 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:24.748 INFO:teuthology.orchestra.run.smithi131.stdout: "services": { 2024-04-26T20:35:24.756 INFO:teuthology.orchestra.run.smithi131.stdout: "dashboard": "https://172.21.15.131:8443/" 2024-04-26T20:35:24.788 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "always_on_modules": { 2024-04-26T20:35:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "octopus": [ 2024-04-26T20:35:24.859 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-04-26T20:35:24.895 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-04-26T20:35:24.903 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-04-26T20:35:24.909 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-04-26T20:35:24.977 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-04-26T20:35:24.984 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-04-26T20:35:24.990 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-04-26T20:35:24.998 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-04-26T20:35:25.006 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-04-26T20:35:25.015 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-04-26T20:35:25.024 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:25.033 INFO:teuthology.orchestra.run.smithi131.stdout: "pacific": [ 2024-04-26T20:35:25.042 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-04-26T20:35:25.051 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-04-26T20:35:25.075 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-04-26T20:35:25.085 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-04-26T20:35:25.093 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-04-26T20:35:25.159 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-04-26T20:35:25.215 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-04-26T20:35:25.224 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-04-26T20:35:25.233 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-04-26T20:35:25.290 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-04-26T20:35:25.299 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:25.308 INFO:teuthology.orchestra.run.smithi131.stdout: "quincy": [ 2024-04-26T20:35:25.316 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-04-26T20:35:25.325 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-04-26T20:35:25.333 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-04-26T20:35:25.341 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-04-26T20:35:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-04-26T20:35:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-04-26T20:35:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-04-26T20:35:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-04-26T20:35:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-04-26T20:35:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-04-26T20:35:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "reef": [ 2024-04-26T20:35:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-04-26T20:35:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-04-26T20:35:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-04-26T20:35:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-04-26T20:35:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-04-26T20:35:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-04-26T20:35:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-04-26T20:35:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-04-26T20:35:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-04-26T20:35:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-04-26T20:35:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-04-26T20:35:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "squid": [ 2024-04-26T20:35:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-04-26T20:35:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-04-26T20:35:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-04-26T20:35:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-04-26T20:35:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-04-26T20:35:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-04-26T20:35:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-04-26T20:35:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-04-26T20:35:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-04-26T20:35:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-04-26T20:35:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:35:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "last_failure_osd_epoch": 3, 2024-04-26T20:35:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "active_clients": [ 2024-04-26T20:35:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-04-26T20:35:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-04-26T20:35:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-04-26T20:35:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-04-26T20:35:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3818652562 2024-04-26T20:35:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:35:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "libcephsqlite", 2024-04-26T20:35:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-04-26T20:35:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-04-26T20:35:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-04-26T20:35:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 245853723 2024-04-26T20:35:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:35:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-04-26T20:35:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-04-26T20:35:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-04-26T20:35:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-04-26T20:35:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 4133589329 2024-04-26T20:35:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:35:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-04-26T20:35:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-04-26T20:35:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-04-26T20:35:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-04-26T20:35:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 731526281 2024-04-26T20:35:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:35:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-04-26T20:35:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-04-26T20:35:26.804 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-26T20:35:26.850 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-04-26T20:35:26.859 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-04-26T20:35:26.867 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 4099508339 2024-04-26T20:35:26.876 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:26.885 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:35:26.893 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:26.902 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-26T20:35:26.910 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:35:26.926 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T20:35:26.935 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T20:35:27.033 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:54 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2868635479' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T20:35:27.042 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:55 smithi148 ceph-mon[32281]: pgmap v1007: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.050 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:57 smithi148 ceph-mon[32281]: pgmap v1008: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.108 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:29:59 smithi148 ceph-mon[32281]: pgmap v1009: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:27.155 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:00 smithi148 ceph-mon[32281]: overall HEALTH_OK 2024-04-26T20:35:27.164 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:01 smithi148 ceph-mon[32281]: pgmap v1010: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:27.172 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:02 smithi148 ceph-mon[32281]: pgmap v1011: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.226 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:04 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:27.236 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:05 smithi148 ceph-mon[32281]: pgmap v1012: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.244 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:05 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.252 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:05 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.262 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.270 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.278 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.287 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:06 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.306 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:06 smithi148 ceph-mon[32281]: pgmap v1013: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:08 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:08 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:08 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:08 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.408 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:08 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.452 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:08 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.461 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:09 smithi148 ceph-mon[32281]: pgmap v1014: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:27.471 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.529 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.538 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:27.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:27.555 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:27.564 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:10 smithi148 ceph-mon[32281]: pgmap v1015: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:27.573 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:13 smithi148 ceph-mon[32281]: pgmap v1016: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:15 smithi148 ceph-mon[32281]: pgmap v1017: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.590 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:17 smithi148 ceph-mon[32281]: pgmap v1018: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:19 smithi148 ceph-mon[32281]: pgmap v1019: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:27.608 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:21 smithi148 ceph-mon[32281]: pgmap v1020: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:27.633 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:23 smithi148 ceph-mon[32281]: pgmap v1021: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.641 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:25 smithi148 ceph-mon[32281]: pgmap v1022: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.650 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:27 smithi148 ceph-mon[32281]: pgmap v1023: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.718 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:29 smithi148 ceph-mon[32281]: pgmap v1024: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:31 smithi148 ceph-mon[32281]: pgmap v1025: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:27.780 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:33 smithi148 ceph-mon[32281]: pgmap v1026: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.790 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:35 smithi148 ceph-mon[32281]: pgmap v1027: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.864 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:37 smithi148 ceph-mon[32281]: pgmap v1028: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.873 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:39 smithi148 ceph-mon[32281]: pgmap v1029: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:27.882 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:41 smithi148 ceph-mon[32281]: pgmap v1030: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:27.890 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:43 smithi148 ceph-mon[32281]: pgmap v1031: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.898 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:45 smithi148 ceph-mon[32281]: pgmap v1032: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.907 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:47 smithi148 ceph-mon[32281]: pgmap v1033: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:49 smithi148 ceph-mon[32281]: pgmap v1034: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:27.923 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:51 smithi148 ceph-mon[32281]: pgmap v1035: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:27.931 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:53 smithi148 ceph-mon[32281]: pgmap v1036: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.939 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:55 smithi148 ceph-mon[32281]: pgmap v1037: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:27.965 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:57 smithi148 ceph-mon[32281]: pgmap v1038: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:27.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:30:59 smithi148 ceph-mon[32281]: pgmap v1039: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:27.979 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:01 smithi148 ceph-mon[32281]: pgmap v1040: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.039 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:03 smithi148 ceph-mon[32281]: pgmap v1041: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:05 smithi148 ceph-mon[32281]: pgmap v1042: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.078 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:07 smithi148 ceph-mon[32281]: pgmap v1043: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.086 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:09 smithi148 ceph-mon[32281]: pgmap v1044: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:28.135 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:28.144 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:28.152 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:10 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:28.160 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:11 smithi148 ceph-mon[32281]: pgmap v1045: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.169 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:28.177 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:28.186 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:28.194 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:28.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:11 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:28.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:13 smithi148 ceph-mon[32281]: pgmap v1046: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.241 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:14 smithi148 ceph-mon[32281]: pgmap v1047: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.248 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:17 smithi148 ceph-mon[32281]: pgmap v1048: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.255 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:19 smithi148 ceph-mon[32281]: pgmap v1049: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:28.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:21 smithi148 ceph-mon[32281]: pgmap v1050: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.354 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:23 smithi148 ceph-mon[32281]: pgmap v1051: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:25 smithi148 ceph-mon[32281]: pgmap v1052: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:27 smithi148 ceph-mon[32281]: pgmap v1053: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:29 smithi148 ceph-mon[32281]: pgmap v1054: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:28.446 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:31 smithi148 ceph-mon[32281]: pgmap v1055: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.453 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:33 smithi148 ceph-mon[32281]: pgmap v1056: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:35 smithi148 ceph-mon[32281]: pgmap v1057: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.466 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:37 smithi148 ceph-mon[32281]: pgmap v1058: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.474 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:39 smithi148 ceph-mon[32281]: pgmap v1059: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.480 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:41 smithi148 ceph-mon[32281]: pgmap v1060: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:35:28.489 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:43 smithi148 ceph-mon[32281]: pgmap v1061: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:28.496 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:45 smithi148 ceph-mon[32281]: pgmap v1062: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.502 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:47 smithi148 ceph-mon[32281]: pgmap v1063: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:28.537 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:49 smithi148 ceph-mon[32281]: pgmap v1064: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:51 smithi148 ceph-mon[32281]: pgmap v1065: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:53 smithi148 ceph-mon[32281]: pgmap v1066: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:55 smithi148 ceph-mon[32281]: pgmap v1067: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.644 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:57 smithi148 ceph-mon[32281]: pgmap v1068: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.652 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:31:59 smithi148 ceph-mon[32281]: pgmap v1069: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:28.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:01 smithi148 ceph-mon[32281]: pgmap v1070: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.717 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:03 smithi148 ceph-mon[32281]: pgmap v1071: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.725 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:05 smithi148 ceph-mon[32281]: pgmap v1072: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:07 smithi148 ceph-mon[32281]: pgmap v1073: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.740 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:09 smithi148 ceph-mon[32281]: pgmap v1074: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:28.747 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:11 smithi148 ceph-mon[32281]: pgmap v1075: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:12 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:28.761 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:13 smithi148 ceph-mon[32281]: pgmap v1076: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.768 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:28.775 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:28.782 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:28.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:15 smithi148 ceph-mon[32281]: pgmap v1077: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:28.863 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:17 smithi148 ceph-mon[32281]: pgmap v1078: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.870 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:19 smithi148 ceph-mon[32281]: pgmap v1079: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:28.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:21 smithi148 ceph-mon[32281]: pgmap v1080: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:28.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:23 smithi148 ceph-mon[32281]: pgmap v1081: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:28.991 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:25 smithi148 ceph-mon[32281]: pgmap v1082: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.000 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:27 smithi148 ceph-mon[32281]: pgmap v1083: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.050 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:29 smithi148 ceph-mon[32281]: pgmap v1084: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.059 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:31 smithi148 ceph-mon[32281]: pgmap v1085: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:29.067 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:33 smithi148 ceph-mon[32281]: pgmap v1086: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.075 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:35 smithi148 ceph-mon[32281]: pgmap v1087: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.084 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:37 smithi148 ceph-mon[32281]: pgmap v1088: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.093 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:39 smithi148 ceph-mon[32281]: pgmap v1089: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.101 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:41 smithi148 ceph-mon[32281]: pgmap v1090: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:29.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:43 smithi148 ceph-mon[32281]: pgmap v1091: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:45 smithi148 ceph-mon[32281]: pgmap v1092: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.127 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:47 smithi148 ceph-mon[32281]: pgmap v1093: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.198 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:49 smithi148 ceph-mon[32281]: pgmap v1094: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.207 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:51 smithi148 ceph-mon[32281]: pgmap v1095: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:29.216 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:53 smithi148 ceph-mon[32281]: pgmap v1096: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.274 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:55 smithi148 ceph-mon[32281]: pgmap v1097: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:57 smithi148 ceph-mon[32281]: pgmap v1098: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:32:59 smithi148 ceph-mon[32281]: pgmap v1099: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:01 smithi148 ceph-mon[32281]: pgmap v1100: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:29.401 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:02 smithi148 ceph-mon[32281]: pgmap v1101: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:05 smithi148 ceph-mon[32281]: pgmap v1102: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.418 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:07 smithi148 ceph-mon[32281]: pgmap v1103: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.427 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:09 smithi148 ceph-mon[32281]: pgmap v1104: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.436 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:10 smithi148 ceph-mon[32281]: pgmap v1105: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:29.456 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:13 smithi148 ceph-mon[32281]: pgmap v1106: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.465 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:13 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:29.474 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:15 smithi148 ceph-mon[32281]: pgmap v1107: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.483 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:29.539 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:29.548 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:29.557 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:17 smithi148 ceph-mon[32281]: pgmap v1108: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.622 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:19 smithi148 ceph-mon[32281]: pgmap v1109: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.708 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:21 smithi148 ceph-mon[32281]: pgmap v1110: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:29.718 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:23 smithi148 ceph-mon[32281]: pgmap v1111: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.727 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:25 smithi148 ceph-mon[32281]: pgmap v1112: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.801 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:27 smithi148 ceph-mon[32281]: pgmap v1113: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:29.810 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:29 smithi148 ceph-mon[32281]: pgmap v1114: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.818 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:31 smithi148 ceph-mon[32281]: pgmap v1115: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:35:29.827 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:33 smithi148 ceph-mon[32281]: pgmap v1116: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:29.837 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:35 smithi148 ceph-mon[32281]: pgmap v1117: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.847 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:37 smithi148 ceph-mon[32281]: pgmap v1118: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:39 smithi148 ceph-mon[32281]: pgmap v1119: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.864 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:41 smithi148 ceph-mon[32281]: pgmap v1120: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:29.873 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:43 smithi148 ceph-mon[32281]: pgmap v1121: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.882 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:45 smithi148 ceph-mon[32281]: pgmap v1122: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:29.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:47 smithi148 ceph-mon[32281]: pgmap v1123: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:29.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:49 smithi148 ceph-mon[32281]: pgmap v1124: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:29.933 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:51 smithi148 ceph-mon[32281]: pgmap v1125: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.000 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:53 smithi148 ceph-mon[32281]: pgmap v1126: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.009 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:55 smithi148 ceph-mon[32281]: pgmap v1127: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:57 smithi148 ceph-mon[32281]: pgmap v1128: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:33:59 smithi148 ceph-mon[32281]: pgmap v1129: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.087 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:01 smithi148 ceph-mon[32281]: pgmap v1130: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.096 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:03 smithi148 ceph-mon[32281]: pgmap v1131: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.106 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:05 smithi148 ceph-mon[32281]: pgmap v1132: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:07 smithi148 ceph-mon[32281]: pgmap v1133: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.123 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:08 smithi148 ceph-mon[32281]: pgmap v1134: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:11 smithi148 ceph-mon[32281]: pgmap v1135: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.140 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:13 smithi148 ceph-mon[32281]: pgmap v1136: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.156 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:15 smithi148 ceph-mon[32281]: pgmap v1137: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.164 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:15 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:30.171 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:30.179 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:30.186 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:30.193 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:17 smithi148 ceph-mon[32281]: pgmap v1138: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:19 smithi148 ceph-mon[32281]: pgmap v1139: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.277 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:21 smithi148 ceph-mon[32281]: pgmap v1140: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.284 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:23 smithi148 ceph-mon[32281]: pgmap v1141: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.292 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:25 smithi148 ceph-mon[32281]: pgmap v1142: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:27 smithi148 ceph-mon[32281]: pgmap v1143: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.383 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:29 smithi148 ceph-mon[32281]: pgmap v1144: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:31 smithi148 ceph-mon[32281]: pgmap v1145: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:33 smithi148 ceph-mon[32281]: pgmap v1146: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.404 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:35 smithi148 ceph-mon[32281]: pgmap v1147: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:37 smithi148 ceph-mon[32281]: pgmap v1148: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.418 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:39 smithi148 ceph-mon[32281]: pgmap v1149: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.425 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:41 smithi148 ceph-mon[32281]: pgmap v1150: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:43 smithi148 ceph-mon[32281]: pgmap v1151: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:45 smithi148 ceph-mon[32281]: pgmap v1152: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.463 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:47 smithi148 ceph-mon[32281]: pgmap v1153: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.470 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:49 smithi148 ceph-mon[32281]: pgmap v1154: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.477 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:51 smithi148 ceph-mon[32281]: pgmap v1155: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.534 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:53 smithi148 ceph-mon[32281]: pgmap v1156: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.578 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:55 smithi148 ceph-mon[32281]: pgmap v1157: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.586 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:57 smithi148 ceph-mon[32281]: pgmap v1158: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:34:59 smithi148 ceph-mon[32281]: pgmap v1159: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.650 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:01 smithi148 ceph-mon[32281]: pgmap v1160: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:03 smithi148 ceph-mon[32281]: pgmap v1161: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.665 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:05 smithi148 ceph-mon[32281]: pgmap v1162: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.672 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:07 smithi148 ceph-mon[32281]: pgmap v1163: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.679 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:09 smithi148 ceph-mon[32281]: pgmap v1164: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.686 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:11 smithi148 ceph-mon[32281]: pgmap v1165: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.693 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:13 smithi148 ceph-mon[32281]: pgmap v1166: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.700 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:15 smithi148 ceph-mon[32281]: pgmap v1167: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.707 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:16 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:30.714 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:17 smithi148 ceph-mon[32281]: pgmap v1168: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.757 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:30.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:30.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:30.805 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:19 smithi148 ceph-mon[32281]: pgmap v1169: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:21 smithi148 ceph-mon[32281]: pgmap v1170: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:30.866 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:23 smithi148 ceph-mon[32281]: pgmap v1171: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.875 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:25 smithi148 ceph-mon[32281]: pgmap v1172: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:54 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2868635479' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T20:35:30.959 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:55 smithi157 ceph-mon[32703]: pgmap v1007: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:30.968 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:57 smithi157 ceph-mon[32703]: pgmap v1008: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:30.977 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:29:59 smithi157 ceph-mon[32703]: pgmap v1009: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:30.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:00 smithi157 ceph-mon[32703]: overall HEALTH_OK 2024-04-26T20:35:30.994 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:01 smithi157 ceph-mon[32703]: pgmap v1010: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:31.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:02 smithi157 ceph-mon[32703]: pgmap v1011: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:04 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:31.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:05 smithi157 ceph-mon[32703]: pgmap v1012: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:31.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:05 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:05 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:06 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:06 smithi157 ceph-mon[32703]: pgmap v1013: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:08 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.298 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:08 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.309 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:08 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.316 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:08 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.325 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:08 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.336 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:08 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:09 smithi157 ceph-mon[32703]: pgmap v1014: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:31.353 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.370 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:31.428 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:31.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:10 smithi157 ceph-mon[32703]: pgmap v1015: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:31.446 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:13 smithi157 ceph-mon[32703]: pgmap v1016: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:15 smithi157 ceph-mon[32703]: pgmap v1017: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:31.546 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:17 smithi157 ceph-mon[32703]: pgmap v1018: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:19 smithi157 ceph-mon[32703]: pgmap v1019: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:31.568 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:21 smithi157 ceph-mon[32703]: pgmap v1020: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:31.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:23 smithi157 ceph-mon[32703]: pgmap v1021: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.650 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:25 smithi157 ceph-mon[32703]: pgmap v1022: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:31.660 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:27 smithi157 ceph-mon[32703]: pgmap v1023: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.669 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:29 smithi157 ceph-mon[32703]: pgmap v1024: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:31.678 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:31 smithi157 ceph-mon[32703]: pgmap v1025: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:31.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:33 smithi157 ceph-mon[32703]: pgmap v1026: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:35 smithi157 ceph-mon[32703]: pgmap v1027: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:31.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:37 smithi157 ceph-mon[32703]: pgmap v1028: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:39 smithi157 ceph-mon[32703]: pgmap v1029: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:31.723 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:41 smithi157 ceph-mon[32703]: pgmap v1030: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:31.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:43 smithi157 ceph-mon[32703]: pgmap v1031: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.772 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:45 smithi157 ceph-mon[32703]: pgmap v1032: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:31.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:47 smithi157 ceph-mon[32703]: pgmap v1033: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.859 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:49 smithi157 ceph-mon[32703]: pgmap v1034: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:31.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:51 smithi157 ceph-mon[32703]: pgmap v1035: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:31.912 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:53 smithi157 ceph-mon[32703]: pgmap v1036: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:31.920 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:55 smithi157 ceph-mon[32703]: pgmap v1037: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.000 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:57 smithi157 ceph-mon[32703]: pgmap v1038: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:30:59 smithi157 ceph-mon[32703]: pgmap v1039: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:01 smithi157 ceph-mon[32703]: pgmap v1040: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:03 smithi157 ceph-mon[32703]: pgmap v1041: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:05 smithi157 ceph-mon[32703]: pgmap v1042: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:07 smithi157 ceph-mon[32703]: pgmap v1043: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:09 smithi157 ceph-mon[32703]: pgmap v1044: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:32.065 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:32.073 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:10 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:32.098 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:11 smithi157 ceph-mon[32703]: pgmap v1045: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.107 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:32.116 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:32.160 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:32.190 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:32.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:11 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:32.208 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:13 smithi157 ceph-mon[32703]: pgmap v1046: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.267 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:14 smithi157 ceph-mon[32703]: pgmap v1047: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:17 smithi157 ceph-mon[32703]: pgmap v1048: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.283 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:19 smithi157 ceph-mon[32703]: pgmap v1049: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:21 smithi157 ceph-mon[32703]: pgmap v1050: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.300 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:23 smithi157 ceph-mon[32703]: pgmap v1051: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.309 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:25 smithi157 ceph-mon[32703]: pgmap v1052: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:27 smithi157 ceph-mon[32703]: pgmap v1053: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.327 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:29 smithi157 ceph-mon[32703]: pgmap v1054: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.335 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:31 smithi157 ceph-mon[32703]: pgmap v1055: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:33 smithi157 ceph-mon[32703]: pgmap v1056: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:35 smithi157 ceph-mon[32703]: pgmap v1057: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:37 smithi157 ceph-mon[32703]: pgmap v1058: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.385 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:39 smithi157 ceph-mon[32703]: pgmap v1059: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.457 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:41 smithi157 ceph-mon[32703]: pgmap v1060: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:35:32.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:43 smithi157 ceph-mon[32703]: pgmap v1061: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:32.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:45 smithi157 ceph-mon[32703]: pgmap v1062: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:47 smithi157 ceph-mon[32703]: pgmap v1063: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:32.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:49 smithi157 ceph-mon[32703]: pgmap v1064: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.574 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:51 smithi157 ceph-mon[32703]: pgmap v1065: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.580 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:53 smithi157 ceph-mon[32703]: pgmap v1066: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.588 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:55 smithi157 ceph-mon[32703]: pgmap v1067: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.595 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:57 smithi157 ceph-mon[32703]: pgmap v1068: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:31:59 smithi157 ceph-mon[32703]: pgmap v1069: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.610 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:01 smithi157 ceph-mon[32703]: pgmap v1070: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.617 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:03 smithi157 ceph-mon[32703]: pgmap v1071: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:05 smithi157 ceph-mon[32703]: pgmap v1072: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.633 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:07 smithi157 ceph-mon[32703]: pgmap v1073: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.660 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:09 smithi157 ceph-mon[32703]: pgmap v1074: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.667 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:11 smithi157 ceph-mon[32703]: pgmap v1075: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.674 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:12 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:32.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:13 smithi157 ceph-mon[32703]: pgmap v1076: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.757 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:32.764 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:32.771 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:32.816 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:15 smithi157 ceph-mon[32703]: pgmap v1077: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.823 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:17 smithi157 ceph-mon[32703]: pgmap v1078: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.830 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:19 smithi157 ceph-mon[32703]: pgmap v1079: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:21 smithi157 ceph-mon[32703]: pgmap v1080: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.844 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:23 smithi157 ceph-mon[32703]: pgmap v1081: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.885 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:25 smithi157 ceph-mon[32703]: pgmap v1082: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.893 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:27 smithi157 ceph-mon[32703]: pgmap v1083: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:29 smithi157 ceph-mon[32703]: pgmap v1084: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.907 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:31 smithi157 ceph-mon[32703]: pgmap v1085: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:32.914 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:33 smithi157 ceph-mon[32703]: pgmap v1086: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.931 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:35 smithi157 ceph-mon[32703]: pgmap v1087: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:32.941 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:37 smithi157 ceph-mon[32703]: pgmap v1088: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:32.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:39 smithi157 ceph-mon[32703]: pgmap v1089: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:32.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:41 smithi157 ceph-mon[32703]: pgmap v1090: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.019 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:43 smithi157 ceph-mon[32703]: pgmap v1091: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:45 smithi157 ceph-mon[32703]: pgmap v1092: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:47 smithi157 ceph-mon[32703]: pgmap v1093: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:49 smithi157 ceph-mon[32703]: pgmap v1094: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:33.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:51 smithi157 ceph-mon[32703]: pgmap v1095: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.100 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:53 smithi157 ceph-mon[32703]: pgmap v1096: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.109 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:55 smithi157 ceph-mon[32703]: pgmap v1097: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:57 smithi157 ceph-mon[32703]: pgmap v1098: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:32:59 smithi157 ceph-mon[32703]: pgmap v1099: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:33.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:01 smithi157 ceph-mon[32703]: pgmap v1100: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.146 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:02 smithi157 ceph-mon[32703]: pgmap v1101: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.155 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:05 smithi157 ceph-mon[32703]: pgmap v1102: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.163 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:07 smithi157 ceph-mon[32703]: pgmap v1103: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:09 smithi157 ceph-mon[32703]: pgmap v1104: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:33.204 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:10 smithi157 ceph-mon[32703]: pgmap v1105: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.212 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:13 smithi157 ceph-mon[32703]: pgmap v1106: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.248 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:13 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:33.294 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:15 smithi157 ceph-mon[32703]: pgmap v1107: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:33.312 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:33.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:33.375 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:17 smithi157 ceph-mon[32703]: pgmap v1108: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.383 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:19 smithi157 ceph-mon[32703]: pgmap v1109: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:33.391 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:21 smithi157 ceph-mon[32703]: pgmap v1110: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.400 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:23 smithi157 ceph-mon[32703]: pgmap v1111: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.409 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:25 smithi157 ceph-mon[32703]: pgmap v1112: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:27 smithi157 ceph-mon[32703]: pgmap v1113: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:33.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:29 smithi157 ceph-mon[32703]: pgmap v1114: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:31 smithi157 ceph-mon[32703]: pgmap v1115: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:35:33.444 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:33 smithi157 ceph-mon[32703]: pgmap v1116: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:33.464 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:35 smithi157 ceph-mon[32703]: pgmap v1117: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.473 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:37 smithi157 ceph-mon[32703]: pgmap v1118: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.482 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:39 smithi157 ceph-mon[32703]: pgmap v1119: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:33.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:41 smithi157 ceph-mon[32703]: pgmap v1120: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:43 smithi157 ceph-mon[32703]: pgmap v1121: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.594 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:45 smithi157 ceph-mon[32703]: pgmap v1122: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:47 smithi157 ceph-mon[32703]: pgmap v1123: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.668 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:49 smithi157 ceph-mon[32703]: pgmap v1124: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:33.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:51 smithi157 ceph-mon[32703]: pgmap v1125: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:53 smithi157 ceph-mon[32703]: pgmap v1126: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:55 smithi157 ceph-mon[32703]: pgmap v1127: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:57 smithi157 ceph-mon[32703]: pgmap v1128: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.712 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:33:59 smithi157 ceph-mon[32703]: pgmap v1129: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:33.720 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:01 smithi157 ceph-mon[32703]: pgmap v1130: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:03 smithi157 ceph-mon[32703]: pgmap v1131: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:05 smithi157 ceph-mon[32703]: pgmap v1132: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.745 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:07 smithi157 ceph-mon[32703]: pgmap v1133: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.765 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:08 smithi157 ceph-mon[32703]: pgmap v1134: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:11 smithi157 ceph-mon[32703]: pgmap v1135: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:33.782 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:13 smithi157 ceph-mon[32703]: pgmap v1136: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:15 smithi157 ceph-mon[32703]: pgmap v1137: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:33.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:15 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:33.903 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:33.912 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:33.978 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:33.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:17 smithi157 ceph-mon[32703]: pgmap v1138: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:33.997 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:19 smithi157 ceph-mon[32703]: pgmap v1139: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:21 smithi157 ceph-mon[32703]: pgmap v1140: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.013 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:23 smithi157 ceph-mon[32703]: pgmap v1141: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:25 smithi157 ceph-mon[32703]: pgmap v1142: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:27 smithi157 ceph-mon[32703]: pgmap v1143: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:29 smithi157 ceph-mon[32703]: pgmap v1144: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:34.047 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:31 smithi157 ceph-mon[32703]: pgmap v1145: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:33 smithi157 ceph-mon[32703]: pgmap v1146: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.094 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:35 smithi157 ceph-mon[32703]: pgmap v1147: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.102 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:37 smithi157 ceph-mon[32703]: pgmap v1148: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.110 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:39 smithi157 ceph-mon[32703]: pgmap v1149: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:34.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:41 smithi157 ceph-mon[32703]: pgmap v1150: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.210 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:43 smithi157 ceph-mon[32703]: pgmap v1151: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.219 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:45 smithi157 ceph-mon[32703]: pgmap v1152: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:47 smithi157 ceph-mon[32703]: pgmap v1153: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.283 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:49 smithi157 ceph-mon[32703]: pgmap v1154: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:34.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:51 smithi157 ceph-mon[32703]: pgmap v1155: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:53 smithi157 ceph-mon[32703]: pgmap v1156: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:55 smithi157 ceph-mon[32703]: pgmap v1157: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.315 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:57 smithi157 ceph-mon[32703]: pgmap v1158: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:34:59 smithi157 ceph-mon[32703]: pgmap v1159: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:34.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:01 smithi157 ceph-mon[32703]: pgmap v1160: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:03 smithi157 ceph-mon[32703]: pgmap v1161: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.347 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:05 smithi157 ceph-mon[32703]: pgmap v1162: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.355 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:07 smithi157 ceph-mon[32703]: pgmap v1163: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:09 smithi157 ceph-mon[32703]: pgmap v1164: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:34.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:11 smithi157 ceph-mon[32703]: pgmap v1165: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.395 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:13 smithi157 ceph-mon[32703]: pgmap v1166: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.463 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:15 smithi157 ceph-mon[32703]: pgmap v1167: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:16 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:34.512 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:17 smithi157 ceph-mon[32703]: pgmap v1168: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.519 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:34.566 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:34.573 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.580 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:19 smithi157 ceph-mon[32703]: pgmap v1169: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:34.586 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:21 smithi157 ceph-mon[32703]: pgmap v1170: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.594 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:23 smithi157 ceph-mon[32703]: pgmap v1171: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.600 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:25 smithi157 ceph-mon[32703]: pgmap v1172: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.610 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:54 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2868635479' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T20:35:34.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:55 smithi131 ceph-mon[27919]: pgmap v1007: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.626 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:57 smithi131 ceph-mon[27919]: pgmap v1008: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.633 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:29:59 smithi131 ceph-mon[27919]: pgmap v1009: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:34.673 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:00 smithi131 ceph-mon[27919]: overall HEALTH_OK 2024-04-26T20:35:34.680 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:01 smithi131 ceph-mon[27919]: pgmap v1010: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:34.687 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:02 smithi131 ceph-mon[27919]: pgmap v1011: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.750 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:04 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:34.801 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:05 smithi131 ceph-mon[27919]: pgmap v1012: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:34.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:05 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.816 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:05 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.918 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.927 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.936 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.946 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:06 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.954 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:06 smithi131 ceph-mon[27919]: pgmap v1013: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:34.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:08 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:08 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.981 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:08 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:08 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:34.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:08 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.034 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:08 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.042 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:09 smithi131 ceph-mon[27919]: pgmap v1014: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:35.050 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:35.196 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:35.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.262 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:10 smithi131 ceph-mon[27919]: pgmap v1015: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:35.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:13 smithi131 ceph-mon[27919]: pgmap v1016: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:15 smithi131 ceph-mon[27919]: pgmap v1017: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:17 smithi131 ceph-mon[27919]: pgmap v1018: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.296 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:19 smithi131 ceph-mon[27919]: pgmap v1019: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:35.306 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:21 smithi131 ceph-mon[27919]: pgmap v1020: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:35.315 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:23 smithi131 ceph-mon[27919]: pgmap v1021: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.324 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:25 smithi131 ceph-mon[27919]: pgmap v1022: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.334 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:27 smithi131 ceph-mon[27919]: pgmap v1023: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.343 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:29 smithi131 ceph-mon[27919]: pgmap v1024: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:31 smithi131 ceph-mon[27919]: pgmap v1025: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:35.418 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:33 smithi131 ceph-mon[27919]: pgmap v1026: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.429 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:35 smithi131 ceph-mon[27919]: pgmap v1027: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:37 smithi131 ceph-mon[27919]: pgmap v1028: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:39 smithi131 ceph-mon[27919]: pgmap v1029: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:35.565 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:41 smithi131 ceph-mon[27919]: pgmap v1030: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:35.574 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:43 smithi131 ceph-mon[27919]: pgmap v1031: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:45 smithi131 ceph-mon[27919]: pgmap v1032: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.624 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:47 smithi131 ceph-mon[27919]: pgmap v1033: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.646 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:49 smithi131 ceph-mon[27919]: pgmap v1034: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:35.657 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:51 smithi131 ceph-mon[27919]: pgmap v1035: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:35.665 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:53 smithi131 ceph-mon[27919]: pgmap v1036: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.673 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:55 smithi131 ceph-mon[27919]: pgmap v1037: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.682 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:57 smithi131 ceph-mon[27919]: pgmap v1038: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.691 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:30:59 smithi131 ceph-mon[27919]: pgmap v1039: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:35.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:01 smithi131 ceph-mon[27919]: pgmap v1040: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:35.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:03 smithi131 ceph-mon[27919]: pgmap v1041: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.739 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:05 smithi131 ceph-mon[27919]: pgmap v1042: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:07 smithi131 ceph-mon[27919]: pgmap v1043: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:09 smithi131 ceph-mon[27919]: pgmap v1044: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:35.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:35.850 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.861 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:10 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.870 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:11 smithi131 ceph-mon[27919]: pgmap v1045: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:35.907 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.915 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.923 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:35.932 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:35.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:11 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:35.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:13 smithi131 ceph-mon[27919]: pgmap v1046: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.958 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:14 smithi131 ceph-mon[27919]: pgmap v1047: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:35.967 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:17 smithi131 ceph-mon[27919]: pgmap v1048: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:35.976 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:19 smithi131 ceph-mon[27919]: pgmap v1049: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:35.984 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:21 smithi131 ceph-mon[27919]: pgmap v1050: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:23 smithi131 ceph-mon[27919]: pgmap v1051: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.023 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:25 smithi131 ceph-mon[27919]: pgmap v1052: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.031 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:27 smithi131 ceph-mon[27919]: pgmap v1053: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.088 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:29 smithi131 ceph-mon[27919]: pgmap v1054: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.139 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:31 smithi131 ceph-mon[27919]: pgmap v1055: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.148 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:33 smithi131 ceph-mon[27919]: pgmap v1056: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.155 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:35 smithi131 ceph-mon[27919]: pgmap v1057: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.199 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:37 smithi131 ceph-mon[27919]: pgmap v1058: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:39 smithi131 ceph-mon[27919]: pgmap v1059: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:41 smithi131 ceph-mon[27919]: pgmap v1060: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:35:36.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:43 smithi131 ceph-mon[27919]: pgmap v1061: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:36.233 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:45 smithi131 ceph-mon[27919]: pgmap v1062: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.242 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:47 smithi131 ceph-mon[27919]: pgmap v1063: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:36.250 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:49 smithi131 ceph-mon[27919]: pgmap v1064: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.258 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:51 smithi131 ceph-mon[27919]: pgmap v1065: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:53 smithi131 ceph-mon[27919]: pgmap v1066: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:55 smithi131 ceph-mon[27919]: pgmap v1067: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.306 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:57 smithi131 ceph-mon[27919]: pgmap v1068: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.315 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:31:59 smithi131 ceph-mon[27919]: pgmap v1069: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.323 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:01 smithi131 ceph-mon[27919]: pgmap v1070: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.386 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:03 smithi131 ceph-mon[27919]: pgmap v1071: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.435 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:05 smithi131 ceph-mon[27919]: pgmap v1072: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.443 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:07 smithi131 ceph-mon[27919]: pgmap v1073: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.452 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:09 smithi131 ceph-mon[27919]: pgmap v1074: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.529 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:11 smithi131 ceph-mon[27919]: pgmap v1075: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:12 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:36.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:13 smithi131 ceph-mon[27919]: pgmap v1076: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.554 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:36.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:36.569 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:36.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:15 smithi131 ceph-mon[27919]: pgmap v1077: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.584 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:17 smithi131 ceph-mon[27919]: pgmap v1078: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.591 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:19 smithi131 ceph-mon[27919]: pgmap v1079: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:21 smithi131 ceph-mon[27919]: pgmap v1080: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:23 smithi131 ceph-mon[27919]: pgmap v1081: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:25 smithi131 ceph-mon[27919]: pgmap v1082: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.648 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:27 smithi131 ceph-mon[27919]: pgmap v1083: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:29 smithi131 ceph-mon[27919]: pgmap v1084: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:31 smithi131 ceph-mon[27919]: pgmap v1085: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.782 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:33 smithi131 ceph-mon[27919]: pgmap v1086: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.790 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:35 smithi131 ceph-mon[27919]: pgmap v1087: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:37 smithi131 ceph-mon[27919]: pgmap v1088: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.833 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:39 smithi131 ceph-mon[27919]: pgmap v1089: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.841 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:41 smithi131 ceph-mon[27919]: pgmap v1090: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.851 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:43 smithi131 ceph-mon[27919]: pgmap v1091: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.861 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:45 smithi131 ceph-mon[27919]: pgmap v1092: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.869 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:47 smithi131 ceph-mon[27919]: pgmap v1093: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.878 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:49 smithi131 ceph-mon[27919]: pgmap v1094: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:51 smithi131 ceph-mon[27919]: pgmap v1095: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:36.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:53 smithi131 ceph-mon[27919]: pgmap v1096: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.926 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:55 smithi131 ceph-mon[27919]: pgmap v1097: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:36.935 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:57 smithi131 ceph-mon[27919]: pgmap v1098: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:36.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:32:59 smithi131 ceph-mon[27919]: pgmap v1099: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:36.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:01 smithi131 ceph-mon[27919]: pgmap v1100: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:02 smithi131 ceph-mon[27919]: pgmap v1101: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.060 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:05 smithi131 ceph-mon[27919]: pgmap v1102: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.069 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:07 smithi131 ceph-mon[27919]: pgmap v1103: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:09 smithi131 ceph-mon[27919]: pgmap v1104: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:37.117 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:10 smithi131 ceph-mon[27919]: pgmap v1105: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:13 smithi131 ceph-mon[27919]: pgmap v1106: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.134 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:13 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:37.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:15 smithi131 ceph-mon[27919]: pgmap v1107: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:37.161 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:37.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:37.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:17 smithi131 ceph-mon[27919]: pgmap v1108: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.186 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:19 smithi131 ceph-mon[27919]: pgmap v1109: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:37.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:21 smithi131 ceph-mon[27919]: pgmap v1110: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.233 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:23 smithi131 ceph-mon[27919]: pgmap v1111: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.242 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:25 smithi131 ceph-mon[27919]: pgmap v1112: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.251 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:27 smithi131 ceph-mon[27919]: pgmap v1113: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:37.295 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:29 smithi131 ceph-mon[27919]: pgmap v1114: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.352 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:31 smithi131 ceph-mon[27919]: pgmap v1115: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T20:35:37.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:33 smithi131 ceph-mon[27919]: pgmap v1116: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T20:35:37.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:35 smithi131 ceph-mon[27919]: pgmap v1117: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.408 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:37 smithi131 ceph-mon[27919]: pgmap v1118: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:39 smithi131 ceph-mon[27919]: pgmap v1119: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:37.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:41 smithi131 ceph-mon[27919]: pgmap v1120: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.433 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:43 smithi131 ceph-mon[27919]: pgmap v1121: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.446 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:45 smithi131 ceph-mon[27919]: pgmap v1122: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:47 smithi131 ceph-mon[27919]: pgmap v1123: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:49 smithi131 ceph-mon[27919]: pgmap v1124: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:37.472 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:51 smithi131 ceph-mon[27919]: pgmap v1125: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.481 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:53 smithi131 ceph-mon[27919]: pgmap v1126: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:55 smithi131 ceph-mon[27919]: pgmap v1127: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.529 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:57 smithi131 ceph-mon[27919]: pgmap v1128: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.536 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:33:59 smithi131 ceph-mon[27919]: pgmap v1129: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:37.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:01 smithi131 ceph-mon[27919]: pgmap v1130: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:03 smithi131 ceph-mon[27919]: pgmap v1131: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:05 smithi131 ceph-mon[27919]: pgmap v1132: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.636 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:07 smithi131 ceph-mon[27919]: pgmap v1133: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:08 smithi131 ceph-mon[27919]: pgmap v1134: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:11 smithi131 ceph-mon[27919]: pgmap v1135: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.705 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:13 smithi131 ceph-mon[27919]: pgmap v1136: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:15 smithi131 ceph-mon[27919]: pgmap v1137: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:15 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:37.731 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:37.740 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:37.749 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:37.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:17 smithi131 ceph-mon[27919]: pgmap v1138: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:19 smithi131 ceph-mon[27919]: pgmap v1139: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.776 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:21 smithi131 ceph-mon[27919]: pgmap v1140: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.821 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:23 smithi131 ceph-mon[27919]: pgmap v1141: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:25 smithi131 ceph-mon[27919]: pgmap v1142: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:37.839 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:27 smithi131 ceph-mon[27919]: pgmap v1143: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.912 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:29 smithi131 ceph-mon[27919]: pgmap v1144: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:37.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:31 smithi131 ceph-mon[27919]: pgmap v1145: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:37.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:33 smithi131 ceph-mon[27919]: pgmap v1146: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:37.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:35 smithi131 ceph-mon[27919]: pgmap v1147: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.025 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:37 smithi131 ceph-mon[27919]: pgmap v1148: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.033 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:39 smithi131 ceph-mon[27919]: pgmap v1149: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.042 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:41 smithi131 ceph-mon[27919]: pgmap v1150: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.050 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:43 smithi131 ceph-mon[27919]: pgmap v1151: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.059 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:45 smithi131 ceph-mon[27919]: pgmap v1152: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:47 smithi131 ceph-mon[27919]: pgmap v1153: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:49 smithi131 ceph-mon[27919]: pgmap v1154: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:51 smithi131 ceph-mon[27919]: pgmap v1155: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:53 smithi131 ceph-mon[27919]: pgmap v1156: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.101 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:55 smithi131 ceph-mon[27919]: pgmap v1157: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:57 smithi131 ceph-mon[27919]: pgmap v1158: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.166 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:34:59 smithi131 ceph-mon[27919]: pgmap v1159: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:01 smithi131 ceph-mon[27919]: pgmap v1160: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.233 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:03 smithi131 ceph-mon[27919]: pgmap v1161: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:05 smithi131 ceph-mon[27919]: pgmap v1162: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.293 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:07 smithi131 ceph-mon[27919]: pgmap v1163: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.301 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:09 smithi131 ceph-mon[27919]: pgmap v1164: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:11 smithi131 ceph-mon[27919]: pgmap v1165: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.351 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:13 smithi131 ceph-mon[27919]: pgmap v1166: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:15 smithi131 ceph-mon[27919]: pgmap v1167: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.367 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:16 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:38.377 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:17 smithi131 ceph-mon[27919]: pgmap v1168: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.385 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:38.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:38.402 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:35:38.410 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:19 smithi131 ceph-mon[27919]: pgmap v1169: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:21 smithi131 ceph-mon[27919]: pgmap v1170: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.446 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:23 smithi131 ceph-mon[27919]: pgmap v1171: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:25 smithi131 ceph-mon[27919]: pgmap v1172: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:27 smithi131 ceph-mon[27919]: pgmap v1173: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.503 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:29 smithi131 ceph-mon[27919]: pgmap v1174: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:31 smithi131 ceph-mon[27919]: pgmap v1175: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:32 smithi131 ceph-mon[27919]: pgmap v1176: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:35:38.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:35 smithi131 ceph-mon[27919]: pgmap v1177: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.626 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:37 smithi131 ceph-mon[27919]: pgmap v1178: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.636 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:27 smithi148 ceph-mon[32281]: pgmap v1173: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.645 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:29 smithi148 ceph-mon[32281]: pgmap v1174: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:31 smithi148 ceph-mon[32281]: pgmap v1175: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.667 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:32 smithi148 ceph-mon[32281]: pgmap v1176: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.675 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:35 smithi148 ceph-mon[32281]: pgmap v1177: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:37 smithi148 ceph-mon[32281]: pgmap v1178: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.693 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:27 smithi157 ceph-mon[32703]: pgmap v1173: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:29 smithi157 ceph-mon[32703]: pgmap v1174: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:38.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:31 smithi157 ceph-mon[32703]: pgmap v1175: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:38.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:32 smithi157 ceph-mon[32703]: pgmap v1176: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:35 smithi157 ceph-mon[32703]: pgmap v1177: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:38.794 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:37 smithi157 ceph-mon[32703]: pgmap v1178: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:38.846 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:35:38.856 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:35:38.864 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 770, 2024-04-26T20:35:38.894 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1138, 2024-04-26T20:35:38.904 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 167182, 2024-04-26T20:35:38.912 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 689219, 2024-04-26T20:35:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 10, 2024-04-26T20:35:38.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-04-26T20:35:38.940 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.261802963, 2024-04-26T20:35:38.949 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.126340966, 2024-04-26T20:35:38.958 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.108911336, 2024-04-26T20:35:38.965 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.015464334, 2024-04-26T20:35:38.975 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:35:38.993 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1136, 2024-04-26T20:35:39.001 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.151760984, 2024-04-26T20:35:39.009 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000133592 2024-04-26T20:35:39.053 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:39.104 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:35:39.113 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1845, 2024-04-26T20:35:39.122 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001182174, 2024-04-26T20:35:39.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000640 2024-04-26T20:35:39.185 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:39.192 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:35:39.200 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:35:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:39.217 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:35:39.226 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 538, 2024-04-26T20:35:39.235 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 526, 2024-04-26T20:35:39.243 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 460719, 2024-04-26T20:35:39.251 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 365721, 2024-04-26T20:35:39.276 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 16, 2024-04-26T20:35:39.285 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-04-26T20:35:39.293 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.158214844, 2024-04-26T20:35:39.351 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.055164452, 2024-04-26T20:35:39.404 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.076985002, 2024-04-26T20:35:39.411 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.022303610, 2024-04-26T20:35:39.419 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:35:39.458 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 526, 2024-04-26T20:35:39.467 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.062879050, 2024-04-26T20:35:39.476 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000119541 2024-04-26T20:35:39.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:39.495 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:35:39.502 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 49, 2024-04-26T20:35:39.511 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000014981, 2024-04-26T20:35:39.518 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000305 2024-04-26T20:35:39.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:39.534 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 457056, 2024-04-26T20:35:39.560 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 290368 2024-04-26T20:35:39.567 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:39.574 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:35:39.636 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 40, 2024-04-26T20:35:39.696 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 385, 2024-04-26T20:35:39.709 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 12791, 2024-04-26T20:35:39.718 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 732751, 2024-04-26T20:35:39.793 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 13, 2024-04-26T20:35:39.800 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-04-26T20:35:39.807 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.136544558, 2024-04-26T20:35:39.818 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.094551485, 2024-04-26T20:35:39.827 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.032399901, 2024-04-26T20:35:39.835 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000441102, 2024-04-26T20:35:39.843 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:35:39.851 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 383, 2024-04-26T20:35:39.860 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.050322492, 2024-04-26T20:35:39.869 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000131390 2024-04-26T20:35:39.890 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:39.899 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:35:39.907 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 52, 2024-04-26T20:35:39.953 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000021513, 2024-04-26T20:35:39.994 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000413 2024-04-26T20:35:40.003 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:40.012 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 9408, 2024-04-26T20:35:40.051 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 675024 2024-04-26T20:35:40.060 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:40.069 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-04-26T20:35:40.079 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-04-26T20:35:40.087 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-04-26T20:35:40.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:40.104 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-04-26T20:35:40.112 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-04-26T20:35:40.121 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-04-26T20:35:40.130 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 46, 2024-04-26T20:35:40.161 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001821830, 2024-04-26T20:35:40.169 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000039605 2024-04-26T20:35:40.177 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:40.233 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:40.292 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-04-26T20:35:40.301 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-04-26T20:35:40.310 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-04-26T20:35:40.354 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-04-26T20:35:40.363 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000007585, 2024-04-26T20:35:40.372 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000003792 2024-04-26T20:35:40.380 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:40.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:40.396 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-04-26T20:35:40.405 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 6, 2024-04-26T20:35:40.413 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 3, 2024-04-26T20:35:40.421 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-04-26T20:35:40.429 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3, 2024-04-26T20:35:40.451 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001657062, 2024-04-26T20:35:40.459 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000552354 2024-04-26T20:35:40.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:40.542 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-04-26T20:35:40.550 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 3, 2024-04-26T20:35:40.559 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-04-26T20:35:40.568 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-04-26T20:35:40.613 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-04-26T20:35:40.622 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-04-26T20:35:40.630 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-04-26T20:35:40.639 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 14, 2024-04-26T20:35:40.647 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-04-26T20:35:40.656 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 14, 2024-04-26T20:35:40.664 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-04-26T20:35:40.674 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 14, 2024-04-26T20:35:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-04-26T20:35:40.691 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 2, 2024-04-26T20:35:40.709 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 2, 2024-04-26T20:35:40.720 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 2, 2024-04-26T20:35:40.729 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 8, 2024-04-26T20:35:40.769 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 3, 2024-04-26T20:35:40.817 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-04-26T20:35:40.826 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 2, 2024-04-26T20:35:40.834 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-04-26T20:35:40.909 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-04-26T20:35:40.918 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-04-26T20:35:40.927 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-04-26T20:35:40.935 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-04-26T20:35:40.944 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-04-26T20:35:40.953 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-04-26T20:35:40.961 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-04-26T20:35:40.969 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-04-26T20:35:40.978 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-04-26T20:35:40.987 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-04-26T20:35:41.028 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-04-26T20:35:41.037 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 4, 2024-04-26T20:35:41.046 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 234, 2024-04-26T20:35:41.100 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-04-26T20:35:41.159 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T20:35:41.169 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-04-26T20:35:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T20:35:41.226 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-04-26T20:35:41.235 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-04-26T20:35:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T20:35:41.252 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T20:35:41.260 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-04-26T20:35:41.269 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-04-26T20:35:41.276 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 2, 2024-04-26T20:35:41.285 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 3, 2024-04-26T20:35:41.293 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T20:35:41.301 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T20:35:41.334 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T20:35:41.343 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 4, 2024-04-26T20:35:41.350 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-04-26T20:35:41.413 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 1, 2024-04-26T20:35:41.484 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 3 2024-04-26T20:35:41.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:41.503 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-04-26T20:35:41.543 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-04-26T20:35:41.551 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-04-26T20:35:41.559 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-04-26T20:35:41.568 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 3, 2024-04-26T20:35:41.575 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-04-26T20:35:41.582 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-04-26T20:35:41.589 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-04-26T20:35:41.596 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-04-26T20:35:41.602 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-04-26T20:35:41.609 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-04-26T20:35:41.616 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-04-26T20:35:41.623 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-04-26T20:35:41.629 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-04-26T20:35:41.656 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-04-26T20:35:41.709 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-04-26T20:35:41.718 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-04-26T20:35:41.727 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-04-26T20:35:41.776 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-04-26T20:35:41.784 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-04-26T20:35:41.793 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T20:35:41.802 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-04-26T20:35:41.810 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-04-26T20:35:41.819 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-04-26T20:35:41.828 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-04-26T20:35:41.836 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-04-26T20:35:41.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:41.853 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-04-26T20:35:41.862 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-04-26T20:35:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 4, 2024-04-26T20:35:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-04-26T20:35:41.945 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-04-26T20:35:42.030 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 4, 2024-04-26T20:35:42.039 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-04-26T20:35:42.048 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-04-26T20:35:42.092 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 2, 2024-04-26T20:35:42.107 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-04-26T20:35:42.115 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-04-26T20:35:42.123 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-04-26T20:35:42.132 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-04-26T20:35:42.140 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-04-26T20:35:42.148 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-04-26T20:35:42.156 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-04-26T20:35:42.164 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4195593, 2024-04-26T20:35:42.172 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-04-26T20:35:42.201 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-04-26T20:35:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3, 2024-04-26T20:35:42.218 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.005929183, 2024-04-26T20:35:42.275 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001976394 2024-04-26T20:35:42.378 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:42.386 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-04-26T20:35:42.394 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:42.469 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-04-26T20:35:42.478 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 17, 2024-04-26T20:35:42.486 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 17, 2024-04-26T20:35:42.495 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-04-26T20:35:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 15, 2024-04-26T20:35:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 16, 2024-04-26T20:35:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-04-26T20:35:42.528 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 14, 2024-04-26T20:35:42.536 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 14, 2024-04-26T20:35:42.544 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-04-26T20:35:42.552 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 2, 2024-04-26T20:35:42.643 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 3, 2024-04-26T20:35:42.652 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 1, 2024-04-26T20:35:42.698 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 51200, 2024-04-26T20:35:42.717 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223608 2024-04-26T20:35:42.805 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:42.813 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-04-26T20:35:42.867 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 8, 2024-04-26T20:35:42.877 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-04-26T20:35:42.886 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 6, 2024-04-26T20:35:42.895 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 36, 2024-04-26T20:35:42.904 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-04-26T20:35:42.913 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-04-26T20:35:42.922 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:42.930 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:42.939 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:42.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:42.976 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-04-26T20:35:42.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-04-26T20:35:42.994 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001403924, 2024-04-26T20:35:43.046 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000701962 2024-04-26T20:35:43.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.134 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-04-26T20:35:43.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.192 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.201 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.222 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-04-26T20:35:43.231 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.239 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.247 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.255 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.290 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-04-26T20:35:43.298 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.317 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.326 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.408 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-04-26T20:35:43.462 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.471 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.480 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.542 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.551 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-04-26T20:35:43.559 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-04-26T20:35:43.568 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000253138, 2024-04-26T20:35:43.576 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000253138 2024-04-26T20:35:43.585 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.593 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-04-26T20:35:43.602 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.610 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.659 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.666 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-04-26T20:35:43.675 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.740 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.796 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.804 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.815 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-04-26T20:35:43.864 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.871 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.878 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.887 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.896 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-04-26T20:35:43.905 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.914 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.921 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:43.931 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:43.940 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-04-26T20:35:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:43.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:43.992 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.049 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.107 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-04-26T20:35:44.116 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.125 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.188 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.197 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.206 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-04-26T20:35:44.215 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.223 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.231 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.239 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.248 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-04-26T20:35:44.257 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.265 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.300 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.308 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.318 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-04-26T20:35:44.377 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.436 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.446 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.498 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-04-26T20:35:44.507 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.515 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.529 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.540 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.548 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-04-26T20:35:44.557 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.565 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.573 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.608 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-04-26T20:35:44.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.625 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.686 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.781 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-04-26T20:35:44.790 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.838 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.847 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.865 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-04-26T20:35:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.888 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.897 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:44.905 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-04-26T20:35:44.913 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:44.969 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:44.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:44.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.061 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-04-26T20:35:45.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.170 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.230 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.239 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-04-26T20:35:45.248 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.257 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.267 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.275 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.284 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-04-26T20:35:45.292 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.301 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.310 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.347 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.356 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-04-26T20:35:45.365 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.426 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.502 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.511 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.520 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-04-26T20:35:45.605 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.615 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.623 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.640 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-04-26T20:35:45.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.658 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.667 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.675 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.683 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-04-26T20:35:45.751 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.760 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.768 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.829 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:45.892 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-04-26T20:35:45.901 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-04-26T20:35:45.909 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-04-26T20:35:45.958 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:45.967 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:45.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:45.983 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:45.992 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:46.001 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-04-26T20:35:46.009 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 2, 2024-04-26T20:35:46.018 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-04-26T20:35:46.026 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-04-26T20:35:46.037 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 2, 2024-04-26T20:35:46.074 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-04-26T20:35:46.083 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 1, 2024-04-26T20:35:46.093 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-04-26T20:35:46.153 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 1481, 2024-04-26T20:35:46.206 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-04-26T20:35:46.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:46.225 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-04-26T20:35:46.263 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:35:46.269 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-04-26T20:35:46.278 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:35:46.286 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:35:46.295 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:35:46.304 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:35:46.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:35:46.322 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:35:46.330 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:35:46.339 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:35:46.375 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:35:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:35:46.393 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:35:46.428 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:35:46.460 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:35:46.494 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-04-26T20:35:46.503 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:35:46.544 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-04-26T20:35:46.553 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:35:46.567 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:35:46.575 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:35:46.584 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:35:46.593 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:35:46.601 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:35:46.610 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:35:46.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-04-26T20:35:46.627 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:35:46.662 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:35:46.670 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:35:46.680 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-04-26T20:35:46.720 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-04-26T20:35:46.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-04-26T20:35:46.782 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:35:46.791 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:35:46.872 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:35:46.880 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:35:46.890 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 265287, 2024-04-26T20:35:46.898 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 97, 2024-04-26T20:35:46.907 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:35:46.916 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-04-26T20:35:46.926 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-04-26T20:35:46.935 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-04-26T20:35:46.943 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:35:46.951 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-04-26T20:35:46.975 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:35:46.983 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-04-26T20:35:46.992 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 9184, 2024-04-26T20:35:47.034 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 177, 2024-04-26T20:35:47.090 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:35:47.099 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:35:47.108 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-04-26T20:35:47.209 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-04-26T20:35:47.218 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 68429, 2024-04-26T20:35:47.226 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 293, 2024-04-26T20:35:47.235 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-04-26T20:35:47.244 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-04-26T20:35:47.257 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-04-26T20:35:47.266 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-04-26T20:35:47.274 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:47.283 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-04-26T20:35:47.292 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-04-26T20:35:47.328 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-04-26T20:35:47.338 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 40, 2024-04-26T20:35:47.347 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 11906, 2024-04-26T20:35:47.408 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 30, 2024-04-26T20:35:47.465 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 40, 2024-04-26T20:35:47.474 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-04-26T20:35:47.483 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 40, 2024-04-26T20:35:47.492 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.254994349, 2024-04-26T20:35:47.501 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006374858 2024-04-26T20:35:47.510 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:47.546 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-04-26T20:35:47.554 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-04-26T20:35:47.563 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 40, 2024-04-26T20:35:47.572 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 64 2024-04-26T20:35:47.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:47.590 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 40, 2024-04-26T20:35:47.599 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-04-26T20:35:47.624 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 40, 2024-04-26T20:35:47.633 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-04-26T20:35:47.643 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-04-26T20:35:47.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-04-26T20:35:47.773 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-04-26T20:35:47.780 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-04-26T20:35:47.789 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 3, 2024-04-26T20:35:47.838 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 6, 2024-04-26T20:35:47.845 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-04-26T20:35:47.854 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-04-26T20:35:47.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-04-26T20:35:47.870 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-04-26T20:35:47.879 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-04-26T20:35:47.888 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-04-26T20:35:47.896 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-04-26T20:35:47.904 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-04-26T20:35:47.912 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-04-26T20:35:47.932 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-04-26T20:35:47.941 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-04-26T20:35:47.950 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-04-26T20:35:48.017 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-04-26T20:35:48.073 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-04-26T20:35:48.082 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-04-26T20:35:48.091 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-04-26T20:35:48.120 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T20:35:48.129 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-04-26T20:35:48.138 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-04-26T20:35:48.146 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-04-26T20:35:48.155 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-04-26T20:35:48.164 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-04-26T20:35:48.173 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-04-26T20:35:48.181 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-04-26T20:35:48.190 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-04-26T20:35:48.197 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-04-26T20:35:48.227 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-04-26T20:35:48.237 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-04-26T20:35:48.245 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-04-26T20:35:48.300 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-04-26T20:35:48.365 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-04-26T20:35:48.374 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-04-26T20:35:48.385 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-04-26T20:35:48.447 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-04-26T20:35:48.455 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-04-26T20:35:48.464 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-04-26T20:35:48.473 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 97, 2024-04-26T20:35:48.482 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-04-26T20:35:48.491 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-04-26T20:35:48.499 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 9, 2024-04-26T20:35:48.508 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 14, 2024-04-26T20:35:48.516 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 5, 2024-04-26T20:35:48.525 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-04-26T20:35:48.553 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 18, 2024-04-26T20:35:48.561 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-04-26T20:35:48.570 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-04-26T20:35:48.638 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:48.697 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-04-26T20:35:48.707 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-04-26T20:35:48.715 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-04-26T20:35:48.755 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-04-26T20:35:48.764 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-04-26T20:35:48.773 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:48.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-04-26T20:35:48.791 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-04-26T20:35:48.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:48.810 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-04-26T20:35:48.819 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-04-26T20:35:48.827 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T20:35:48.836 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-04-26T20:35:48.874 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-04-26T20:35:48.882 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-04-26T20:35:48.891 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-04-26T20:35:48.942 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-04-26T20:35:48.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:49.007 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:35:49.015 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:35:49.063 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-04-26T20:35:49.071 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:35:49.079 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 1344, 2024-04-26T20:35:49.087 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 489318, 2024-04-26T20:35:49.095 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:35:49.106 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 1344, 2024-04-26T20:35:49.114 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:35:49.124 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:35:49.133 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1344, 2024-04-26T20:35:49.142 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 489318, 2024-04-26T20:35:49.175 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:35:49.184 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:49.192 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:49.259 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:49.343 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:49.352 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:49.360 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-04-26T20:35:49.397 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:35:49.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-04-26T20:35:49.411 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:35:49.418 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-04-26T20:35:49.427 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-04-26T20:35:49.436 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:35:49.445 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-04-26T20:35:49.454 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 40, 2024-04-26T20:35:49.462 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 11906, 2024-04-26T20:35:49.498 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 40, 2024-04-26T20:35:49.508 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 11906, 2024-04-26T20:35:49.517 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:35:49.526 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:49.583 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:49.639 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:49.648 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:49.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:49.766 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-04-26T20:35:49.773 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:35:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-04-26T20:35:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:35:49.800 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-04-26T20:35:49.809 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-04-26T20:35:49.818 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:35:49.827 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-04-26T20:35:49.836 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 40, 2024-04-26T20:35:49.882 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 40, 2024-04-26T20:35:49.890 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 40, 2024-04-26T20:35:49.899 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 40, 2024-04-26T20:35:49.907 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:35:49.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:50.024 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:50.033 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:50.041 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:50.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:50.104 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-04-26T20:35:50.113 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:35:50.121 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-04-26T20:35:50.130 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:35:50.139 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-04-26T20:35:50.148 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-04-26T20:35:50.156 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:35:50.165 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-04-26T20:35:50.190 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:35:50.199 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:35:50.207 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-04-26T20:35:50.216 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-04-26T20:35:50.308 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:35:50.391 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:50.399 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:50.406 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:50.481 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:50.488 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:50.495 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x555b5c918500": { 2024-04-26T20:35:50.502 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:35:50.510 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-04-26T20:35:50.517 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:35:50.527 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 4, 2024-04-26T20:35:50.534 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1289, 2024-04-26T20:35:50.542 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:35:50.565 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 4, 2024-04-26T20:35:50.574 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:35:50.582 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:35:50.591 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3, 2024-04-26T20:35:50.647 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1289, 2024-04-26T20:35:50.697 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:35:50.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:50.715 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:50.780 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:50.788 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:35:50.800 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-26T20:35:50.809 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-26T20:35:50.818 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T20:35:50.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T20:35:50.836 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:39 smithi148 ceph-mon[32281]: pgmap v1179: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:50.881 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:41 smithi148 ceph-mon[32281]: pgmap v1180: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:50.889 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:43 smithi148 ceph-mon[32281]: pgmap v1181: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:50.898 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:45 smithi148 ceph-mon[32281]: pgmap v1182: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:50.906 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:47 smithi148 ceph-mon[32281]: pgmap v1183: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:51.001 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:49 smithi148 ceph-mon[32281]: pgmap v1184: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:51.047 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:39 smithi157 ceph-mon[32703]: pgmap v1179: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:51.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:41 smithi157 ceph-mon[32703]: pgmap v1180: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:51.065 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:43 smithi157 ceph-mon[32703]: pgmap v1181: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:51.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:45 smithi157 ceph-mon[32703]: pgmap v1182: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:51.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:47 smithi157 ceph-mon[32703]: pgmap v1183: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:51.138 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:49 smithi157 ceph-mon[32703]: pgmap v1184: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:51.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:39 smithi131 ceph-mon[27919]: pgmap v1179: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:51.156 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:41 smithi131 ceph-mon[27919]: pgmap v1180: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:51.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:43 smithi131 ceph-mon[27919]: pgmap v1181: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:51.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:45 smithi131 ceph-mon[27919]: pgmap v1182: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:51.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:47 smithi131 ceph-mon[27919]: pgmap v1183: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:51.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:49 smithi131 ceph-mon[27919]: pgmap v1184: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:51.669 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:35:51.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:51 smithi148 ceph-mon[32281]: pgmap v1185: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:51.824 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:51 smithi131 ceph-mon[27919]: pgmap v1185: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:51.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:51 smithi157 ceph-mon[32703]: pgmap v1185: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:35:53.780 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:53 smithi131 ceph-mon[27919]: pgmap v1186: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:53.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:53 smithi148 ceph-mon[32281]: pgmap v1186: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:53.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:53 smithi157 ceph-mon[32703]: pgmap v1186: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:54.148 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:35:54.158 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:35:54.166 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 468, 2024-04-26T20:35:54.175 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 464, 2024-04-26T20:35:54.184 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 415216, 2024-04-26T20:35:54.192 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 286800, 2024-04-26T20:35:54.201 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-04-26T20:35:54.209 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 0, 2024-04-26T20:35:54.217 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.165265516, 2024-04-26T20:35:54.226 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.074867808, 2024-04-26T20:35:54.234 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.058204620, 2024-04-26T20:35:54.324 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.022538956, 2024-04-26T20:35:54.333 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:35:54.342 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 464, 2024-04-26T20:35:54.434 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.048099123, 2024-04-26T20:35:54.510 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000103661 2024-04-26T20:35:54.517 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:54.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:35:54.569 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:54.578 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:54.584 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:54.590 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:54.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 415216, 2024-04-26T20:35:54.604 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 286800 2024-04-26T20:35:54.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:54.617 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:35:54.625 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 2, 2024-04-26T20:35:54.649 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 354, 2024-04-26T20:35:54.695 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 320, 2024-04-26T20:35:54.701 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 219488, 2024-04-26T20:35:54.708 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-04-26T20:35:54.774 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-04-26T20:35:54.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.068456539, 2024-04-26T20:35:54.839 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.056063256, 2024-04-26T20:35:54.848 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.003968602, 2024-04-26T20:35:54.894 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:35:54.903 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:35:54.912 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 352, 2024-04-26T20:35:54.921 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.038556583, 2024-04-26T20:35:54.931 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000109535 2024-04-26T20:35:54.939 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:54.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:35:54.957 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:54.965 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:55.005 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:55.015 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:55.023 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T20:35:55.051 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 219488 2024-04-26T20:35:55.100 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:55.146 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:35:55.154 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 0, 2024-04-26T20:35:55.163 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2, 2024-04-26T20:35:55.244 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 0, 2024-04-26T20:35:55.253 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 0, 2024-04-26T20:35:55.263 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 2, 2024-04-26T20:35:55.271 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 1, 2024-04-26T20:35:55.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.000925416, 2024-04-26T20:35:55.289 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T20:35:55.297 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.000767497, 2024-04-26T20:35:55.306 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:35:55.314 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:35:55.338 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:55.346 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:55.354 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:55.363 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:55.425 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:35:55.483 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:55.492 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:55.501 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:55.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:55.553 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:35:55.561 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:35:55.569 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:55.578 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-04-26T20:35:55.586 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 0, 2024-04-26T20:35:55.594 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-04-26T20:35:55.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:55.611 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-04-26T20:35:55.641 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:35:55.650 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-04-26T20:35:55.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:35:55.719 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:35:55.770 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:35:55.821 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:35:55.830 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:35:55.838 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:35:55.897 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:35:55.906 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:35:55.914 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:35:55.921 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:35:55.928 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:35:55.935 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:35:55.942 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:35:55.949 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-04-26T20:35:55.957 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:35:55.982 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-04-26T20:35:55.990 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:35:55.997 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:35:56.048 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:35:56.101 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:35:56.162 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:35:56.170 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:35:56.177 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:35:56.228 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-04-26T20:35:56.235 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:35:56.242 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:35:56.250 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:35:56.257 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-04-26T20:35:56.264 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-04-26T20:35:56.271 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-04-26T20:35:56.278 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:35:56.285 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:35:56.307 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:35:56.314 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:35:56.321 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 149735, 2024-04-26T20:35:56.388 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 30, 2024-04-26T20:35:56.443 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:35:56.495 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-04-26T20:35:56.502 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-04-26T20:35:56.509 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-04-26T20:35:56.544 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:35:56.551 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-04-26T20:35:56.558 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:35:56.566 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-04-26T20:35:56.573 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 0, 2024-04-26T20:35:56.579 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 0, 2024-04-26T20:35:56.586 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:35:56.593 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:35:56.600 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-04-26T20:35:56.624 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-04-26T20:35:56.631 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 928, 2024-04-26T20:35:56.639 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2, 2024-04-26T20:35:56.648 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-04-26T20:35:56.698 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-04-26T20:35:56.746 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-04-26T20:35:56.755 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-04-26T20:35:56.764 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:35:56.811 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:35:56.820 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:35:56.829 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-04-26T20:35:56.838 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:35:56.847 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 466, 2024-04-26T20:35:56.855 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 346634, 2024-04-26T20:35:56.863 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:35:56.872 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 466, 2024-04-26T20:35:56.881 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:35:56.897 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:35:56.905 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 466, 2024-04-26T20:35:56.914 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 346634, 2024-04-26T20:35:56.922 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:35:56.957 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:35:57.024 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:35:57.033 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:35:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:57.085 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:35:57.094 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:35:57.103 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:55 smithi148 ceph-mon[32281]: pgmap v1187: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:57.112 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:55 smithi157 ceph-mon[32703]: pgmap v1187: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:57.122 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T20:35:57.131 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T20:35:57.139 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T20:35:57.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T20:35:57.157 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:55 smithi131 ceph-mon[27919]: pgmap v1187: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:35:57.639 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:35:57.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:57 smithi131 ceph-mon[27919]: pgmap v1188: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:57.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:57 smithi157 ceph-mon[32703]: pgmap v1188: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:58.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:57 smithi148 ceph-mon[32281]: pgmap v1188: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:35:59.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:35:59 smithi131 ceph-mon[27919]: pgmap v1189: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:35:59.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:35:59 smithi157 ceph-mon[32703]: pgmap v1189: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:00.067 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:36:00.076 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:36:00.151 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 520, 2024-04-26T20:36:00.160 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 988, 2024-04-26T20:36:00.170 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 94588, 2024-04-26T20:36:00.179 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 359567, 2024-04-26T20:36:00.187 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-04-26T20:36:00.195 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-04-26T20:36:00.239 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.220320233, 2024-04-26T20:36:00.247 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.101285097, 2024-04-26T20:36:00.258 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.080850689, 2024-04-26T20:36:00.312 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.015893163, 2024-04-26T20:36:00.321 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:00.329 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 986, 2024-04-26T20:36:00.337 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.125025432, 2024-04-26T20:36:00.431 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000126800 2024-04-26T20:36:00.458 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:00.467 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:00.483 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1466, 2024-04-26T20:36:00.518 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000909609, 2024-04-26T20:36:00.525 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000620 2024-04-26T20:36:00.532 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:00.539 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:36:00.546 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:36:00.553 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:00.561 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:36:00.567 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 629, 2024-04-26T20:36:00.584 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 618, 2024-04-26T20:36:00.614 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 489745, 2024-04-26T20:36:00.621 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 338443, 2024-04-26T20:36:00.628 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 7, 2024-04-26T20:36:00.655 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-04-26T20:36:00.712 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.183552616, 2024-04-26T20:36:00.774 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.075230446, 2024-04-26T20:36:00.781 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.080533862, 2024-04-26T20:36:00.788 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.023476178, 2024-04-26T20:36:00.843 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:00.850 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 620, 2024-04-26T20:36:00.860 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.091681152, 2024-04-26T20:36:00.867 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000147872 2024-04-26T20:36:00.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:00.882 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:00.890 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 209, 2024-04-26T20:36:00.897 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000089248, 2024-04-26T20:36:00.905 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000427 2024-04-26T20:36:00.942 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:00.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 451120, 2024-04-26T20:36:00.957 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 291504 2024-04-26T20:36:01.007 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:01.065 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:36:01.112 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 36, 2024-04-26T20:36:01.120 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 387, 2024-04-26T20:36:01.127 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 10085, 2024-04-26T20:36:01.168 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 887832, 2024-04-26T20:36:01.175 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 20, 2024-04-26T20:36:01.184 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 8, 2024-04-26T20:36:01.194 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.131248944, 2024-04-26T20:36:01.203 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.088434183, 2024-04-26T20:36:01.212 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.031701497, 2024-04-26T20:36:01.220 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000209604, 2024-04-26T20:36:01.229 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:01.238 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 383, 2024-04-26T20:36:01.270 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.046480859, 2024-04-26T20:36:01.278 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000121359 2024-04-26T20:36:01.286 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:01.296 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:01.351 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 47, 2024-04-26T20:36:01.393 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000017315, 2024-04-26T20:36:01.402 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000368 2024-04-26T20:36:01.410 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:01.468 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 7248, 2024-04-26T20:36:01.477 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 767456 2024-04-26T20:36:01.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:01.494 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-04-26T20:36:01.503 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-04-26T20:36:01.511 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-04-26T20:36:01.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:01.528 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-04-26T20:36:01.538 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-04-26T20:36:01.562 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-04-26T20:36:01.570 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 37, 2024-04-26T20:36:01.579 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002627457, 2024-04-26T20:36:01.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000071012 2024-04-26T20:36:01.718 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:01.774 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:01.783 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-04-26T20:36:01.792 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-04-26T20:36:01.833 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-04-26T20:36:01.842 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-04-26T20:36:01.851 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000007950, 2024-04-26T20:36:01.859 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000003975 2024-04-26T20:36:01.868 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:01.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:01.883 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-04-26T20:36:01.891 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 1, 2024-04-26T20:36:01.900 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-04-26T20:36:01.930 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-04-26T20:36:01.939 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:01.947 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:01.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:02.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-04-26T20:36:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 1, 2024-04-26T20:36:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-04-26T20:36:02.118 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-04-26T20:36:02.126 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-04-26T20:36:02.135 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-04-26T20:36:02.143 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-04-26T20:36:02.151 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 10, 2024-04-26T20:36:02.159 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-04-26T20:36:02.167 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 10, 2024-04-26T20:36:02.176 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-04-26T20:36:02.184 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-04-26T20:36:02.206 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-04-26T20:36:02.215 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-04-26T20:36:02.223 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-04-26T20:36:02.230 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-04-26T20:36:02.289 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-04-26T20:36:02.332 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-04-26T20:36:02.340 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-04-26T20:36:02.347 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-04-26T20:36:02.393 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-04-26T20:36:02.400 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-04-26T20:36:02.406 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-04-26T20:36:02.414 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-04-26T20:36:02.421 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-04-26T20:36:02.428 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-04-26T20:36:02.436 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-04-26T20:36:02.443 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-04-26T20:36:02.450 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-04-26T20:36:02.472 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-04-26T20:36:02.480 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-04-26T20:36:02.487 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 1, 2024-04-26T20:36:02.516 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 4, 2024-04-26T20:36:02.545 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 234, 2024-04-26T20:36:02.583 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-04-26T20:36:02.591 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T20:36:02.598 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-04-26T20:36:02.634 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T20:36:02.642 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-04-26T20:36:02.649 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-04-26T20:36:02.656 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T20:36:02.663 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T20:36:02.670 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-04-26T20:36:02.677 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-04-26T20:36:02.683 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T20:36:02.690 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-04-26T20:36:02.714 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T20:36:02.720 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T20:36:02.728 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T20:36:02.763 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-04-26T20:36:02.803 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-04-26T20:36:02.841 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-04-26T20:36:02.848 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-04-26T20:36:02.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:02.905 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-04-26T20:36:02.912 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-04-26T20:36:02.919 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-04-26T20:36:02.926 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-04-26T20:36:02.934 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 1, 2024-04-26T20:36:02.941 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-04-26T20:36:02.948 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-04-26T20:36:02.954 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-04-26T20:36:02.961 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-04-26T20:36:02.993 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-04-26T20:36:03.000 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-04-26T20:36:03.007 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-04-26T20:36:03.037 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-04-26T20:36:03.081 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-04-26T20:36:03.118 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-04-26T20:36:03.130 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-04-26T20:36:03.138 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-04-26T20:36:03.196 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-04-26T20:36:03.204 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-04-26T20:36:03.214 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-04-26T20:36:03.223 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T20:36:03.231 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-04-26T20:36:03.241 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-04-26T20:36:03.249 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-04-26T20:36:03.258 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-04-26T20:36:03.267 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-04-26T20:36:03.275 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:03.288 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-04-26T20:36:03.297 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-04-26T20:36:03.305 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 3, 2024-04-26T20:36:03.364 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-04-26T20:36:03.398 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-04-26T20:36:03.407 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 3, 2024-04-26T20:36:03.416 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-04-26T20:36:03.459 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-04-26T20:36:03.468 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 2, 2024-04-26T20:36:03.477 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-04-26T20:36:03.486 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-04-26T20:36:03.494 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-04-26T20:36:03.503 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-04-26T20:36:03.512 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-04-26T20:36:03.521 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-04-26T20:36:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-04-26T20:36:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4195202, 2024-04-26T20:36:03.554 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-04-26T20:36:03.563 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-04-26T20:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-04-26T20:36:03.643 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.003858834, 2024-04-26T20:36:03.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001929417 2024-04-26T20:36:03.711 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:03.720 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-04-26T20:36:03.777 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:03.786 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-04-26T20:36:03.795 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 13, 2024-04-26T20:36:03.803 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 13, 2024-04-26T20:36:03.812 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-04-26T20:36:03.820 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 11, 2024-04-26T20:36:03.829 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 12, 2024-04-26T20:36:03.837 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-04-26T20:36:03.846 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 10, 2024-04-26T20:36:03.888 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 10, 2024-04-26T20:36:03.895 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-04-26T20:36:03.903 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-04-26T20:36:03.962 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-04-26T20:36:04.041 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-04-26T20:36:04.085 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 50560, 2024-04-26T20:36:04.094 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223608 2024-04-26T20:36:04.102 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.181 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-04-26T20:36:04.190 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 1, 2024-04-26T20:36:04.199 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-04-26T20:36:04.208 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 1, 2024-04-26T20:36:04.217 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 18, 2024-04-26T20:36:04.226 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-04-26T20:36:04.235 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-04-26T20:36:04.243 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.251 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.294 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.309 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-04-26T20:36:04.317 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.366 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.396 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.404 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.410 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-04-26T20:36:04.473 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.480 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.487 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.494 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.501 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-04-26T20:36:04.508 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.515 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.523 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.530 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.608 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-04-26T20:36:04.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.622 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.670 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.768 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-04-26T20:36:04.775 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.782 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.839 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.853 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-04-26T20:36:04.860 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.867 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.873 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.881 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.888 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-04-26T20:36:04.897 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:04.919 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:04.928 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:04.937 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:04.947 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-04-26T20:36:05.003 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.060 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.069 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.077 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.117 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-04-26T20:36:05.126 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-04-26T20:36:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.187 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.196 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.204 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-04-26T20:36:05.221 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.274 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:35:59 smithi148 ceph-mon[32281]: pgmap v1189: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:05.342 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:01 smithi157 ceph-mon[32703]: pgmap v1190: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:05.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:03 smithi157 ceph-mon[32703]: pgmap v1191: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:05.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:01 smithi148 ceph-mon[32281]: pgmap v1190: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:05.468 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:03 smithi148 ceph-mon[32281]: pgmap v1191: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:05.535 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.543 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.551 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.559 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-04-26T20:36:05.568 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.575 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.584 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.592 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.600 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-04-26T20:36:05.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.653 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.661 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.670 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.726 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-04-26T20:36:05.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.787 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.797 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.855 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-04-26T20:36:05.864 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.881 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.889 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.898 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-04-26T20:36:05.906 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:05.914 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:05.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:05.962 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:05.969 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-04-26T20:36:05.994 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.030 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.061 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.068 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.075 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-04-26T20:36:06.117 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.126 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.133 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.148 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-04-26T20:36:06.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.163 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.170 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.224 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-04-26T20:36:06.231 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.239 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.246 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.296 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.304 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-04-26T20:36:06.345 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.355 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.408 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.416 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.423 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-04-26T20:36:06.430 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.438 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.446 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.460 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-04-26T20:36:06.489 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.552 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.564 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.571 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.610 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-04-26T20:36:06.640 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.679 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.744 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.791 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-04-26T20:36:06.799 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.811 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.818 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.825 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.832 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-04-26T20:36:06.840 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.847 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:06.854 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:06.886 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:06.895 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-04-26T20:36:06.902 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:06.965 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:07.021 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:07.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:07.069 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-04-26T20:36:07.076 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:07.136 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:07.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:07.151 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:07.160 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-04-26T20:36:07.168 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-04-26T20:36:07.177 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-04-26T20:36:07.186 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:07.195 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:07.204 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:07.283 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:07.292 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:07.301 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-04-26T20:36:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-04-26T20:36:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-04-26T20:36:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-04-26T20:36:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-04-26T20:36:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-04-26T20:36:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-04-26T20:36:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-04-26T20:36:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 1268, 2024-04-26T20:36:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-04-26T20:36:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-04-26T20:36:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:36:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-04-26T20:36:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:36:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:36:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:36:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:36:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:36:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:36:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:36:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:36:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:36:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:36:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:36:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:36:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:36:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-04-26T20:36:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:36:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-04-26T20:36:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:36:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:36:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:36:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:36:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:36:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:36:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:36:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-04-26T20:36:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:36:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:36:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:36:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-04-26T20:36:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-04-26T20:36:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-04-26T20:36:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:36:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:36:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:36:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:36:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 232231, 2024-04-26T20:36:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 83, 2024-04-26T20:36:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:36:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-04-26T20:36:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-04-26T20:36:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-04-26T20:36:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:36:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-04-26T20:36:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:36:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-04-26T20:36:08.499 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 9184, 2024-04-26T20:36:08.506 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 177, 2024-04-26T20:36:08.513 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:36:08.520 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:36:08.527 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-04-26T20:36:08.534 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-04-26T20:36:08.542 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 50872, 2024-04-26T20:36:08.549 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 198, 2024-04-26T20:36:08.594 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-04-26T20:36:08.601 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-04-26T20:36:08.609 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-04-26T20:36:08.668 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-04-26T20:36:08.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:08.763 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-04-26T20:36:08.801 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-04-26T20:36:08.809 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-04-26T20:36:08.866 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 37, 2024-04-26T20:36:08.875 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 11110, 2024-04-26T20:36:08.885 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 33, 2024-04-26T20:36:08.895 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 37, 2024-04-26T20:36:08.904 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-04-26T20:36:08.914 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 37, 2024-04-26T20:36:08.924 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.315992976, 2024-04-26T20:36:08.933 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.008540350 2024-04-26T20:36:08.943 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:08.986 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-04-26T20:36:08.996 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-04-26T20:36:09.005 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 37, 2024-04-26T20:36:09.040 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 59 2024-04-26T20:36:09.107 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:09.116 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 37, 2024-04-26T20:36:09.152 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-04-26T20:36:09.162 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 37, 2024-04-26T20:36:09.232 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-04-26T20:36:09.242 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-04-26T20:36:09.251 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-04-26T20:36:09.260 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-04-26T20:36:09.269 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-04-26T20:36:09.277 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 2, 2024-04-26T20:36:09.286 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 6, 2024-04-26T20:36:09.295 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-04-26T20:36:09.305 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-04-26T20:36:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-04-26T20:36:09.324 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-04-26T20:36:09.361 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-04-26T20:36:09.412 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-04-26T20:36:09.463 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-04-26T20:36:09.470 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-04-26T20:36:09.509 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-04-26T20:36:09.517 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-04-26T20:36:09.566 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-04-26T20:36:09.573 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-04-26T20:36:09.581 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-04-26T20:36:09.589 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-04-26T20:36:09.596 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-04-26T20:36:09.604 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-04-26T20:36:09.613 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T20:36:09.622 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-04-26T20:36:09.631 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-04-26T20:36:09.668 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-04-26T20:36:09.705 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-04-26T20:36:09.747 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-04-26T20:36:09.755 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-04-26T20:36:09.793 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-04-26T20:36:09.802 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-04-26T20:36:09.863 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-04-26T20:36:09.873 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-04-26T20:36:09.882 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-04-26T20:36:09.891 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-04-26T20:36:09.900 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-04-26T20:36:09.909 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-04-26T20:36:09.917 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-04-26T20:36:09.925 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-04-26T20:36:09.933 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-04-26T20:36:09.969 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-04-26T20:36:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-04-26T20:36:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 97, 2024-04-26T20:36:10.013 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-04-26T20:36:10.123 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-04-26T20:36:10.132 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 8, 2024-04-26T20:36:10.177 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 11, 2024-04-26T20:36:10.186 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 3, 2024-04-26T20:36:10.232 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-04-26T20:36:10.241 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 14, 2024-04-26T20:36:10.251 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-04-26T20:36:10.259 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-04-26T20:36:10.268 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:10.277 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-04-26T20:36:10.285 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-04-26T20:36:10.294 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-04-26T20:36:10.302 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-04-26T20:36:10.349 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-04-26T20:36:10.358 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:10.366 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-04-26T20:36:10.392 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-04-26T20:36:10.434 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:10.443 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-04-26T20:36:10.486 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-04-26T20:36:10.493 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T20:36:10.577 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-04-26T20:36:10.589 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-04-26T20:36:10.596 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-04-26T20:36:10.604 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-04-26T20:36:10.611 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-04-26T20:36:10.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:10.628 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:36:10.635 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:36:10.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-04-26T20:36:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:36:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 1181, 2024-04-26T20:36:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 458449, 2024-04-26T20:36:10.790 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:10.862 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 1181, 2024-04-26T20:36:10.870 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:36:10.956 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:36:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1181, 2024-04-26T20:36:11.030 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 458449, 2024-04-26T20:36:11.036 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:36:11.045 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:11.054 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:11.063 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:11.072 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:11.081 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:11.090 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-04-26T20:36:11.099 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:36:11.173 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-04-26T20:36:11.183 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:36:11.257 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-04-26T20:36:11.327 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-04-26T20:36:11.419 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:11.429 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:11.515 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 37, 2024-04-26T20:36:11.529 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 11110, 2024-04-26T20:36:11.615 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 37, 2024-04-26T20:36:11.624 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 11110, 2024-04-26T20:36:11.632 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:36:11.641 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:11.658 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:11.668 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:11.676 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:11.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:11.707 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-04-26T20:36:11.803 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:36:11.811 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-04-26T20:36:11.820 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:36:11.828 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-04-26T20:36:11.910 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-04-26T20:36:11.921 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:11.969 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:11.980 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 37, 2024-04-26T20:36:12.038 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 37, 2024-04-26T20:36:12.047 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 37, 2024-04-26T20:36:12.059 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 37, 2024-04-26T20:36:12.068 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:36:12.085 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:12.094 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:12.103 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:12.111 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:12.121 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:12.166 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-04-26T20:36:12.175 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:36:12.184 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-04-26T20:36:12.192 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:36:12.251 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-04-26T20:36:12.261 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-04-26T20:36:12.315 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:12.324 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:12.359 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:36:12.368 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:36:12.376 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-04-26T20:36:12.384 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-04-26T20:36:12.393 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:36:12.401 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:12.409 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:12.419 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:12.429 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:12.476 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:12.485 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5562b8780500": { 2024-04-26T20:36:12.496 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:36:12.554 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-04-26T20:36:12.595 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:36:12.602 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 3, 2024-04-26T20:36:12.644 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 898, 2024-04-26T20:36:12.652 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 3, 2024-04-26T20:36:12.700 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:36:12.709 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:36:12.719 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2, 2024-04-26T20:36:12.727 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 898, 2024-04-26T20:36:12.736 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:36:12.745 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:12.753 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:12.762 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:12.810 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:12.818 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:36:12.879 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-26T20:36:12.920 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-26T20:36:12.927 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T20:36:12.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T20:36:12.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:01 smithi131 ceph-mon[27919]: pgmap v1190: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:13.024 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:03 smithi131 ceph-mon[27919]: pgmap v1191: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:13.031 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:05 smithi131 ceph-mon[27919]: pgmap v1192: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:13.038 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:07 smithi131 ceph-mon[27919]: pgmap v1193: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:13.045 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:09 smithi131 ceph-mon[27919]: pgmap v1194: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:13.052 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:11 smithi131 ceph-mon[27919]: pgmap v1195: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:13.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:05 smithi157 ceph-mon[32703]: pgmap v1192: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:13.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:07 smithi157 ceph-mon[32703]: pgmap v1193: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:13.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:09 smithi157 ceph-mon[32703]: pgmap v1194: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:13.083 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:11 smithi157 ceph-mon[32703]: pgmap v1195: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:13.126 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:05 smithi148 ceph-mon[32281]: pgmap v1192: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:13.133 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:07 smithi148 ceph-mon[32281]: pgmap v1193: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:13.140 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:09 smithi148 ceph-mon[32281]: pgmap v1194: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:13.213 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:11 smithi148 ceph-mon[32281]: pgmap v1195: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T20:36:13.883 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:13 smithi131 ceph-mon[27919]: pgmap v1196: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:13.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:13 smithi157 ceph-mon[32703]: pgmap v1196: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:14.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:13 smithi148 ceph-mon[32281]: pgmap v1196: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:15.867 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:15 smithi131 ceph-mon[27919]: pgmap v1197: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:15.963 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-26T20:36:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:36:16.024 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 0, 2024-04-26T20:36:16.034 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2, 2024-04-26T20:36:16.042 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 0, 2024-04-26T20:36:16.051 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 0, 2024-04-26T20:36:16.060 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-04-26T20:36:16.069 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 0, 2024-04-26T20:36:16.194 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.003829810, 2024-04-26T20:36:16.200 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T20:36:16.209 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.000751130, 2024-04-26T20:36:16.263 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:36:16.354 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:16.363 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:16.460 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:16.554 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:16.563 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:16.572 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:16.581 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:16.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:16.658 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:36:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:36:16.675 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:36:16.733 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 6, 2024-04-26T20:36:16.742 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 361, 2024-04-26T20:36:16.793 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 3264, 2024-04-26T20:36:16.856 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 220816, 2024-04-26T20:36:16.918 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-04-26T20:36:16.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-04-26T20:36:16.978 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.069003241, 2024-04-26T20:36:17.062 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.056135615, 2024-04-26T20:36:17.080 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.004315638, 2024-04-26T20:36:17.098 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:36:17.152 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:17.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 360, 2024-04-26T20:36:17.168 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.041488838, 2024-04-26T20:36:17.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000115246 2024-04-26T20:36:17.185 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:17.195 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:17.233 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:17.241 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:17.248 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:17.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:17.353 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T20:36:17.360 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 220816 2024-04-26T20:36:17.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:17.396 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:36:17.453 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 471, 2024-04-26T20:36:17.462 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 469, 2024-04-26T20:36:17.470 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 414176, 2024-04-26T20:36:17.530 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 290832, 2024-04-26T20:36:17.537 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 2, 2024-04-26T20:36:17.544 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 1, 2024-04-26T20:36:17.551 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.160129520, 2024-04-26T20:36:17.558 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.075284607, 2024-04-26T20:36:17.566 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.052194722, 2024-04-26T20:36:17.600 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.022149590, 2024-04-26T20:36:17.607 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:17.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 468, 2024-04-26T20:36:17.672 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.048668156, 2024-04-26T20:36:17.721 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000103991 2024-04-26T20:36:17.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:17.813 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:17.822 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:17.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:17.887 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 414176, 2024-04-26T20:36:17.896 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 290832 2024-04-26T20:36:17.905 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:17.914 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-04-26T20:36:17.922 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 0, 2024-04-26T20:36:17.931 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-04-26T20:36:17.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:17.982 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-04-26T20:36:17.990 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:36:17.999 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-04-26T20:36:18.055 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:36:18.064 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:36:18.142 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:36:18.152 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:36:18.197 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:36:18.206 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:36:18.215 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:36:18.223 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:36:18.273 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:36:18.283 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:36:18.293 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:36:18.302 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:36:18.322 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:36:18.379 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-04-26T20:36:18.388 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:36:18.397 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-04-26T20:36:18.453 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:36:18.502 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:36:18.514 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:36:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:36:18.585 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:36:18.594 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:36:18.602 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:36:18.639 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-04-26T20:36:18.693 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:36:18.702 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:36:18.711 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:36:18.720 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-04-26T20:36:18.728 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-04-26T20:36:18.751 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-04-26T20:36:18.791 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:36:18.800 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:36:18.809 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:36:18.864 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:36:18.919 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 149735, 2024-04-26T20:36:18.929 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 30, 2024-04-26T20:36:18.943 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:36:18.954 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-04-26T20:36:19.028 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-04-26T20:36:19.035 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-04-26T20:36:19.043 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:36:19.098 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-04-26T20:36:19.107 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:36:19.115 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-04-26T20:36:19.124 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 0, 2024-04-26T20:36:19.132 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 0, 2024-04-26T20:36:19.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:36:19.174 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:36:19.182 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-04-26T20:36:19.191 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-04-26T20:36:19.199 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 928, 2024-04-26T20:36:19.276 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2, 2024-04-26T20:36:19.289 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-04-26T20:36:19.322 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-04-26T20:36:19.385 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-04-26T20:36:19.436 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-04-26T20:36:19.445 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-26T20:36:19.475 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:36:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-04-26T20:36:19.548 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-04-26T20:36:19.566 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-04-26T20:36:19.575 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 473, 2024-04-26T20:36:19.583 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 347544, 2024-04-26T20:36:19.591 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:19.625 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 473, 2024-04-26T20:36:19.637 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-04-26T20:36:19.647 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-04-26T20:36:19.655 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 473, 2024-04-26T20:36:19.701 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 347544, 2024-04-26T20:36:19.713 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-04-26T20:36:19.755 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-04-26T20:36:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-04-26T20:36:19.813 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-04-26T20:36:19.822 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:19.878 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-26T20:36:19.903 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-26T20:36:19.915 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:15 smithi157 ceph-mon[32703]: pgmap v1197: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:19.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:17 smithi157 ceph-mon[32703]: pgmap v1198: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:19.932 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:18 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:36:19.941 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:15 smithi148 ceph-mon[32281]: pgmap v1197: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:19.950 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:17 smithi148 ceph-mon[32281]: pgmap v1198: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:19.983 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:18 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:36:19.991 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:19 smithi148 ceph-mon[32281]: pgmap v1199: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:20.001 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T20:36:20.029 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T20:36:20.042 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T20:36:20.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T20:36:20.102 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:17 smithi131 ceph-mon[27919]: pgmap v1198: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:20.111 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:18 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:36:20.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:19 smithi131 ceph-mon[27919]: pgmap v1199: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:20.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:19 smithi157 ceph-mon[32703]: pgmap v1199: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:36:21.855 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:21 smithi148 ceph-mon[32281]: pgmap v1200: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:21.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:21 smithi157 ceph-mon[32703]: pgmap v1200: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:22.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:21 smithi131 ceph-mon[27919]: pgmap v1200: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:23.564 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-26T20:36:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:36:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 4320, 2024-04-26T20:36:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3923, 2024-04-26T20:36:23.665 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1837569, 2024-04-26T20:36:23.673 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 5615587, 2024-04-26T20:36:23.682 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 11, 2024-04-26T20:36:23.741 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-26T20:36:23.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.398079527, 2024-04-26T20:36:23.759 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.120974993, 2024-04-26T20:36:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.124169027, 2024-04-26T20:36:23.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.121166922, 2024-04-26T20:36:23.785 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:23.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3925, 2024-04-26T20:36:23.802 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.248528452, 2024-04-26T20:36:23.812 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000063319 2024-04-26T20:36:23.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:23.828 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1627, 2024-04-26T20:36:23.936 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000450238, 2024-04-26T20:36:23.945 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000276 2024-04-26T20:36:24.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.022 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 468288, 2024-04-26T20:36:24.031 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 299120 2024-04-26T20:36:24.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:36:24.058 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3495, 2024-04-26T20:36:24.066 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3370, 2024-04-26T20:36:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1117006, 2024-04-26T20:36:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 7982638, 2024-04-26T20:36:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 166, 2024-04-26T20:36:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-26T20:36:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.349486588, 2024-04-26T20:36:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.099981310, 2024-04-26T20:36:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.133124891, 2024-04-26T20:36:24.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.091590803, 2024-04-26T20:36:24.233 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:24.292 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3372, 2024-04-26T20:36:24.301 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.180529886, 2024-04-26T20:36:24.310 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000053537 2024-04-26T20:36:24.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.343 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:24.352 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1635, 2024-04-26T20:36:24.360 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000412087, 2024-04-26T20:36:24.369 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000252 2024-04-26T20:36:24.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.385 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T20:36:24.394 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 803616 2024-04-26T20:36:24.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.412 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:36:24.424 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 11341, 2024-04-26T20:36:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 13097, 2024-04-26T20:36:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 4282602, 2024-04-26T20:36:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 10397756, 2024-04-26T20:36:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 17, 2024-04-26T20:36:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 8, 2024-04-26T20:36:24.550 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 1.053697705, 2024-04-26T20:36:24.558 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.341212731, 2024-04-26T20:36:24.576 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.500457399, 2024-04-26T20:36:24.586 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.275728078, 2024-04-26T20:36:24.594 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:24.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 13089, 2024-04-26T20:36:24.610 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.377640657, 2024-04-26T20:36:24.618 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000028851 2024-04-26T20:36:24.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.633 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:24.641 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 14171, 2024-04-26T20:36:24.671 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002552583, 2024-04-26T20:36:24.709 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000180 2024-04-26T20:36:24.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.766 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3420896, 2024-04-26T20:36:24.777 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 4412976 2024-04-26T20:36:24.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.795 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-26T20:36:24.826 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-26T20:36:24.835 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-26T20:36:24.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.853 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-26T20:36:24.863 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-26T20:36:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-26T20:36:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5738, 2024-04-26T20:36:24.890 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.450920681, 2024-04-26T20:36:24.899 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000427138 2024-04-26T20:36:24.909 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:24.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-26T20:36:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-26T20:36:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-26T20:36:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5639, 2024-04-26T20:36:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.188010113, 2024-04-26T20:36:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000033341 2024-04-26T20:36:25.070 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:25.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:25.086 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-26T20:36:25.097 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 2481, 2024-04-26T20:36:25.106 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 2481, 2024-04-26T20:36:25.115 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-26T20:36:25.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2481, 2024-04-26T20:36:25.132 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.729563196, 2024-04-26T20:36:25.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001100186 2024-04-26T20:36:25.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:25.188 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-26T20:36:25.237 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-26T20:36:25.246 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-26T20:36:25.297 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-26T20:36:25.306 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 133, 2024-04-26T20:36:25.315 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-26T20:36:25.324 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-26T20:36:25.343 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 24, 2024-04-26T20:36:25.353 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 4, 2024-04-26T20:36:25.362 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 20, 2024-04-26T20:36:25.370 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-26T20:36:25.379 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 18, 2024-04-26T20:36:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-26T20:36:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 4, 2024-04-26T20:36:25.405 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 4, 2024-04-26T20:36:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-26T20:36:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 2512, 2024-04-26T20:36:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 2503, 2024-04-26T20:36:25.528 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-26T20:36:25.538 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-26T20:36:25.553 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-26T20:36:25.616 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-26T20:36:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-26T20:36:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-26T20:36:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-26T20:36:25.669 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-26T20:36:25.678 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-26T20:36:25.687 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-26T20:36:25.694 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-26T20:36:25.703 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-26T20:36:25.711 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 37, 2024-04-26T20:36:25.720 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-26T20:36:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4, 2024-04-26T20:36:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 234, 2024-04-26T20:36:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-26T20:36:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 37, 2024-04-26T20:36:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-26T20:36:26.080 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T20:36:26.094 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-26T20:36:26.103 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 25, 2024-04-26T20:36:26.111 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 3, 2024-04-26T20:36:26.145 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 22, 2024-04-26T20:36:26.156 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 12, 2024-04-26T20:36:26.166 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 1, 2024-04-26T20:36:26.177 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 20, 2024-04-26T20:36:26.186 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 3641, 2024-04-26T20:36:26.195 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T20:36:26.205 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T20:36:26.217 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T20:36:26.227 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 2436, 2024-04-26T20:36:26.236 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 1212, 2024-04-26T20:36:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 19, 2024-04-26T20:36:26.386 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 1241 2024-04-26T20:36:26.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:26.431 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-26T20:36:26.443 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-26T20:36:26.451 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-26T20:36:26.461 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-26T20:36:26.494 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-26T20:36:26.502 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 4, 2024-04-26T20:36:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-26T20:36:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-26T20:36:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-26T20:36:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 1212, 2024-04-26T20:36:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 1212, 2024-04-26T20:36:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-26T20:36:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-26T20:36:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-26T20:36:26.661 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-26T20:36:26.701 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-26T20:36:26.711 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-26T20:36:26.756 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-26T20:36:26.764 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-04-26T20:36:26.773 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-04-26T20:36:26.782 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 3, 2024-04-26T20:36:26.795 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-26T20:36:26.803 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-26T20:36:26.812 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-26T20:36:26.820 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-26T20:36:26.831 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-26T20:36:26.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:26.849 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-26T20:36:26.858 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-26T20:36:26.867 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 6236, 2024-04-26T20:36:26.875 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 6233, 2024-04-26T20:36:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 6233, 2024-04-26T20:36:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 3, 2024-04-26T20:36:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-26T20:36:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 3, 2024-04-26T20:36:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 77, 2024-04-26T20:36:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 76, 2024-04-26T20:36:27.046 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 76, 2024-04-26T20:36:27.065 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 1, 2024-04-26T20:36:27.073 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 1, 2024-04-26T20:36:27.080 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-26T20:36:27.086 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 1, 2024-04-26T20:36:27.093 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 17194952, 2024-04-26T20:36:27.100 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 17203552, 2024-04-26T20:36:27.108 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-26T20:36:27.115 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-26T20:36:27.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2568, 2024-04-26T20:36:27.170 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4.510362906, 2024-04-26T20:36:27.179 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001756371 2024-04-26T20:36:27.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:27.255 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-26T20:36:27.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-26T20:36:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 23, 2024-04-26T20:36:27.332 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 1238, 2024-04-26T20:36:27.353 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 1215, 2024-04-26T20:36:27.362 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 18, 2024-04-26T20:36:27.370 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 24, 2024-04-26T20:36:27.379 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 6, 2024-04-26T20:36:27.388 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 24, 2024-04-26T20:36:27.396 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 1243, 2024-04-26T20:36:27.406 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 1219, 2024-04-26T20:36:27.415 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 4, 2024-04-26T20:36:27.424 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 1226, 2024-04-26T20:36:27.450 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 1222, 2024-04-26T20:36:27.459 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 59168, 2024-04-26T20:36:27.518 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 231800 2024-04-26T20:36:27.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:27.564 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-26T20:36:27.572 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 2510, 2024-04-26T20:36:27.581 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-26T20:36:27.589 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 2481, 2024-04-26T20:36:27.612 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 47, 2024-04-26T20:36:27.620 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-26T20:36:27.629 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-26T20:36:27.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1210, 2024-04-26T20:36:27.646 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.053034887, 2024-04-26T20:36:27.655 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000870276 2024-04-26T20:36:27.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-26T20:36:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 27, 2024-04-26T20:36:27.706 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.007606636, 2024-04-26T20:36:27.715 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000281727 2024-04-26T20:36:27.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:27.770 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-26T20:36:27.806 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:27.815 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:27.823 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:27.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:27.870 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-26T20:36:27.879 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:27.888 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:27.897 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:27.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:27.914 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-26T20:36:27.923 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:27.932 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:27.941 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:27.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.005 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-26T20:36:28.064 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:28.072 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:28.081 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:28.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.169 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-26T20:36:28.177 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-26T20:36:28.197 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000162695, 2024-04-26T20:36:28.205 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000162695 2024-04-26T20:36:28.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-26T20:36:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-26T20:36:28.310 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:28.319 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:28.370 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:28.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.430 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-26T20:36:28.439 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:28.448 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:28.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.488 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-26T20:36:28.495 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-26T20:36:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 12, 2024-04-26T20:36:28.547 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.020768069, 2024-04-26T20:36:28.589 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001730672 2024-04-26T20:36:28.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.650 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-26T20:36:28.658 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-26T20:36:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:28.780 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-26T20:36:28.801 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-26T20:36:28.808 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000382573, 2024-04-26T20:36:28.816 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000382573 2024-04-26T20:36:28.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:28.830 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-26T20:36:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2024-04-26T20:36:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003285540, 2024-04-26T20:36:28.954 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000547590 2024-04-26T20:36:29.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.014 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-26T20:36:29.021 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-26T20:36:29.029 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002313820, 2024-04-26T20:36:29.049 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001156910 2024-04-26T20:36:29.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.064 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-26T20:36:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-26T20:36:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.134 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-26T20:36:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2024-04-26T20:36:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.030284228, 2024-04-26T20:36:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005047371 2024-04-26T20:36:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-26T20:36:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.327 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.336 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.354 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-26T20:36:29.363 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.372 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.380 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.450 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-26T20:36:29.502 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-26T20:36:29.513 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001666109, 2024-04-26T20:36:29.572 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000833054 2024-04-26T20:36:29.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.588 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-26T20:36:29.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.628 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.654 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-26T20:36:29.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.672 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.681 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.698 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-26T20:36:29.707 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.756 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.858 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-26T20:36:29.867 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-26T20:36:29.875 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.023104610, 2024-04-26T20:36:29.883 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002888076 2024-04-26T20:36:29.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.908 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-26T20:36:29.916 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:29.926 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:29.934 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:29.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:29.949 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-26T20:36:29.958 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1206, 2024-04-26T20:36:29.966 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.586954029, 2024-04-26T20:36:29.975 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001315882 2024-04-26T20:36:30.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:30.071 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-26T20:36:30.080 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-26T20:36:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-26T20:36:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:30.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:30.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:30.192 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-26T20:36:30.200 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 2, 2024-04-26T20:36:30.209 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 3, 2024-04-26T20:36:30.217 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 1, 2024-04-26T20:36:30.225 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 2, 2024-04-26T20:36:30.234 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-26T20:36:30.242 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 420, 2024-04-26T20:36:30.272 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 210, 2024-04-26T20:36:30.281 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1528, 2024-04-26T20:36:30.369 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-26T20:36:30.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:30.423 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-26T20:36:30.432 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:36:30.441 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-26T20:36:30.450 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:36:30.484 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:36:30.493 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:36:30.501 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:36:30.510 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:36:30.518 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:36:30.527 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:36:30.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:36:30.543 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:36:30.552 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:36:30.560 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:36:30.614 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:36:30.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:36:30.681 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-26T20:36:30.713 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:36:30.720 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-26T20:36:30.727 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:36:30.734 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:36:30.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:36:30.775 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:36:30.782 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:36:30.788 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:36:30.795 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:36:30.803 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-26T20:36:30.810 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:36:30.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:36:30.823 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:36:30.848 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-26T20:36:30.858 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-26T20:36:30.945 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-26T20:36:30.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:36:31.006 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:36:31.015 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:36:31.026 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:36:31.044 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 294503, 2024-04-26T20:36:31.071 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 136, 2024-04-26T20:36:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 1232, 2024-04-26T20:36:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 14, 2024-04-26T20:36:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-26T20:36:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-26T20:36:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:36:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-26T20:36:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:36:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-26T20:36:31.154 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 9184, 2024-04-26T20:36:31.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 177, 2024-04-26T20:36:31.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:36:31.218 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:36:31.257 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-26T20:36:31.265 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-26T20:36:31.274 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 95634, 2024-04-26T20:36:31.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 650, 2024-04-26T20:36:31.321 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-26T20:36:31.333 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-26T20:36:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-26T20:36:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-26T20:36:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-26T20:36:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-26T20:36:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-26T20:36:31.391 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 8345, 2024-04-26T20:36:31.426 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 13596444, 2024-04-26T20:36:31.434 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 37, 2024-04-26T20:36:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 8345, 2024-04-26T20:36:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-26T20:36:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8345, 2024-04-26T20:36:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 43.941889309, 2024-04-26T20:36:31.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005265654 2024-04-26T20:36:31.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:31.621 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-26T20:36:31.630 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-26T20:36:31.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8345, 2024-04-26T20:36:31.646 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 9384 2024-04-26T20:36:31.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:31.662 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 8345, 2024-04-26T20:36:31.670 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 74, 2024-04-26T20:36:31.679 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 8271, 2024-04-26T20:36:31.687 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-26T20:36:31.695 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-26T20:36:31.718 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 111, 2024-04-26T20:36:31.772 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 457, 2024-04-26T20:36:31.781 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 34, 2024-04-26T20:36:31.879 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 2599, 2024-04-26T20:36:31.889 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 202, 2024-04-26T20:36:31.898 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-26T20:36:31.907 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-26T20:36:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-26T20:36:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-26T20:36:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 4716, 2024-04-26T20:36:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-26T20:36:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-26T20:36:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-26T20:36:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 40, 2024-04-26T20:36:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 703, 2024-04-26T20:36:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-26T20:36:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-26T20:36:32.061 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-26T20:36:32.121 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-26T20:36:32.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-26T20:36:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-26T20:36:32.179 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T20:36:32.188 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-26T20:36:32.196 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-26T20:36:32.216 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-26T20:36:32.225 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-26T20:36:32.233 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-26T20:36:32.242 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-26T20:36:32.248 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-26T20:36:32.257 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-26T20:36:32.266 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-26T20:36:32.275 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-26T20:36:32.284 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-26T20:36:32.316 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-26T20:36:32.325 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-26T20:36:32.380 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-26T20:36:32.389 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-26T20:36:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-26T20:36:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-26T20:36:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-26T20:36:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-26T20:36:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 97, 2024-04-26T20:36:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-26T20:36:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 11, 2024-04-26T20:36:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-04-26T20:36:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 13, 2024-04-26T20:36:32.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 1, 2024-04-26T20:36:32.543 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-26T20:36:32.551 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 484, 2024-04-26T20:36:32.558 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-26T20:36:32.597 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 38 2024-04-26T20:36:32.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:32.670 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-26T20:36:32.677 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-26T20:36:32.721 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-26T20:36:32.728 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-26T20:36:32.736 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-26T20:36:32.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:32.793 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-26T20:36:32.801 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-26T20:36:32.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:32.816 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-26T20:36:32.822 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-26T20:36:32.829 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 19, 2024-04-26T20:36:32.836 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-26T20:36:32.843 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 5, 2024-04-26T20:36:32.851 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 4698, 2024-04-26T20:36:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 1212, 2024-04-26T20:36:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-26T20:36:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:36:33.014 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:33.022 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-26T20:36:33.031 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:33.040 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 19152, 2024-04-26T20:36:33.056 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 5108462, 2024-04-26T20:36:33.065 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:33.074 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 19152, 2024-04-26T20:36:33.083 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:33.092 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:33.100 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 19152, 2024-04-26T20:36:33.109 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 5108462, 2024-04-26T20:36:33.117 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:33.126 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:33.150 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:33.160 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:33.209 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:33.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:33.279 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-26T20:36:33.288 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:33.299 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-26T20:36:33.308 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-26T20:36:33.389 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-26T20:36:33.398 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:33.406 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:33.415 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8345, 2024-04-26T20:36:33.424 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 13728504, 2024-04-26T20:36:33.432 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8345, 2024-04-26T20:36:33.440 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 13728504, 2024-04-26T20:36:33.449 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:33.504 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:33.514 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:33.561 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:33.570 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:33.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:33.639 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-26T20:36:33.649 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:33.658 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-26T20:36:33.698 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:33.707 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-26T20:36:33.717 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-26T20:36:33.726 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:33.735 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:33.743 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8345, 2024-04-26T20:36:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 8345, 2024-04-26T20:36:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8345, 2024-04-26T20:36:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 8345, 2024-04-26T20:36:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:33.832 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:33.891 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:33.900 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:33.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:33.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:33.972 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-26T20:36:33.982 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:34.016 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-26T20:36:34.026 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:34.035 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1212, 2024-04-26T20:36:34.044 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 132060, 2024-04-26T20:36:34.054 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:34.064 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1212, 2024-04-26T20:36:34.073 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 31, 2024-04-26T20:36:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 132060, 2024-04-26T20:36:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:34.186 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:34.196 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:34.229 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:34.237 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:34.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:34.255 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x556d99ff2500": { 2024-04-26T20:36:34.291 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:34.323 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-26T20:36:34.332 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:34.340 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 6236, 2024-04-26T20:36:34.349 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 13009248, 2024-04-26T20:36:34.358 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:34.367 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 6236, 2024-04-26T20:36:34.377 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:34.385 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:34.415 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2568, 2024-04-26T20:36:34.424 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 13009248, 2024-04-26T20:36:34.511 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:34.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:34.581 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:34.589 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:34.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:34.605 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:34.632 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-26T20:36:34.641 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:23 smithi148 ceph-mon[32281]: pgmap v1201: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:34.651 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.659 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:23 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:23 smithi131 ceph-mon[27919]: pgmap v1201: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:34.674 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:23 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.688 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: pgmap v1202: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:34.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.744 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.752 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:34.814 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:34.822 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:34.864 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:34.872 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:36:34.880 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:36:34.889 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.924 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:26 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:36:34.933 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:26 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:36:34.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:27 smithi131 ceph-mon[27919]: pgmap v1203: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:34.952 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:29 smithi131 ceph-mon[27919]: pgmap v1204: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:34.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:31 smithi131 ceph-mon[27919]: pgmap v1205: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:34.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:33 smithi131 ceph-mon[27919]: pgmap v1206: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:34.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:23 smithi157 ceph-mon[32703]: pgmap v1201: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:34.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:34.998 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:23 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:35.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: pgmap v1202: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:35.064 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:35.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:35.131 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.208 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.218 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:36:35.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:36:35.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:35.281 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:26 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:36:35.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:26 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:36:35.300 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:27 smithi157 ceph-mon[32703]: pgmap v1203: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:35.308 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:29 smithi157 ceph-mon[32703]: pgmap v1204: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:35.317 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:31 smithi157 ceph-mon[32703]: pgmap v1205: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:35.326 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:33 smithi157 ceph-mon[32703]: pgmap v1206: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:35.337 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-26T20:36:35.388 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-26T20:36:35.396 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T20:36:35.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T20:36:35.469 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: pgmap v1202: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:35.505 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:35.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:35.522 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.530 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.557 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:36:35.575 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:36:35.586 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:36:35.595 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:36:35.602 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:26 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi131 to 176.4M 2024-04-26T20:36:35.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:26 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi131 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T20:36:35.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:27 smithi148 ceph-mon[32281]: pgmap v1203: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:35.628 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:29 smithi148 ceph-mon[32281]: pgmap v1204: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:35.652 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:31 smithi148 ceph-mon[32281]: pgmap v1205: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:35.661 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:33 smithi148 ceph-mon[32281]: pgmap v1206: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:35.969 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:36:35.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:35 smithi157 ceph-mon[32703]: pgmap v1207: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:36.029 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:35 smithi131 ceph-mon[27919]: pgmap v1207: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:36.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:35 smithi148 ceph-mon[32281]: pgmap v1207: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:37.691 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-26T20:36:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:36:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 0, 2024-04-26T20:36:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2, 2024-04-26T20:36:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 0, 2024-04-26T20:36:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 0, 2024-04-26T20:36:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-04-26T20:36:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 0, 2024-04-26T20:36:37.791 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.001098832, 2024-04-26T20:36:37.800 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T20:36:37.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.000211773, 2024-04-26T20:36:37.818 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:36:37.879 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:37.964 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:37.973 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:38.030 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:38.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.048 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:38.057 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:38.067 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:38.075 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.093 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:36:38.102 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:36:38.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.120 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:36:38.130 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 479, 2024-04-26T20:36:38.139 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 475, 2024-04-26T20:36:38.148 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 418208, 2024-04-26T20:36:38.198 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 293840, 2024-04-26T20:36:38.260 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 2, 2024-04-26T20:36:38.269 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-04-26T20:36:38.296 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.076245717, 2024-04-26T20:36:38.307 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.032880308, 2024-04-26T20:36:38.316 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.023243887, 2024-04-26T20:36:38.325 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.012600317, 2024-04-26T20:36:38.334 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:38.343 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 475, 2024-04-26T20:36:38.352 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.028961520, 2024-04-26T20:36:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000060971 2024-04-26T20:36:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:38.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 418208, 2024-04-26T20:36:38.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 293840 2024-04-26T20:36:38.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.535 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:36:38.543 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2, 2024-04-26T20:36:38.551 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 362, 2024-04-26T20:36:38.560 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 320, 2024-04-26T20:36:38.586 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 220896, 2024-04-26T20:36:38.595 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-04-26T20:36:38.603 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 2, 2024-04-26T20:36:38.611 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.022000068, 2024-04-26T20:36:38.620 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.016827183, 2024-04-26T20:36:38.628 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.000831225, 2024-04-26T20:36:38.636 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:36:38.647 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:38.656 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 360, 2024-04-26T20:36:38.682 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.026722820, 2024-04-26T20:36:38.691 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000074230 2024-04-26T20:36:38.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.757 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:38.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T20:36:38.860 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 220896 2024-04-26T20:36:38.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.877 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-26T20:36:38.885 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 0, 2024-04-26T20:36:38.892 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-26T20:36:38.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:38.906 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-26T20:36:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:36:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-26T20:36:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:36:39.042 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:36:39.050 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:36:39.088 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:36:39.095 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:36:39.104 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:36:39.113 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:36:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:36:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:36:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:36:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:36:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:36:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:36:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-26T20:36:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:36:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-26T20:36:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:36:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:36:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:36:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:36:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:36:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:36:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:36:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-26T20:36:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:36:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:36:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:36:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-26T20:36:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-26T20:36:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-26T20:36:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:36:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:36:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:36:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:36:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 148722, 2024-04-26T20:36:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 30, 2024-04-26T20:36:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:36:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-26T20:36:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-26T20:36:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-26T20:36:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:36:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-26T20:36:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:36:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-26T20:36:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 0, 2024-04-26T20:36:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 0, 2024-04-26T20:36:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:36:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:36:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-26T20:36:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-26T20:36:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 928, 2024-04-26T20:36:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 2, 2024-04-26T20:36:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-26T20:36:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-26T20:36:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-26T20:36:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-26T20:36:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:36:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-26T20:36:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 477, 2024-04-26T20:36:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 348064, 2024-04-26T20:36:39.995 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 477, 2024-04-26T20:36:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 477, 2024-04-26T20:36:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 348064, 2024-04-26T20:36:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:40.137 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:40.146 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:40.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:40.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:40.226 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-26T20:36:40.236 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:37 smithi148 ceph-mon[32281]: pgmap v1208: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:40.245 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:37 smithi131 ceph-mon[27919]: pgmap v1208: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:40.254 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:39 smithi131 ceph-mon[27919]: pgmap v1209: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:40.263 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:37 smithi157 ceph-mon[32703]: pgmap v1208: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:40.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:39 smithi157 ceph-mon[32703]: pgmap v1209: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:40.283 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-26T20:36:40.291 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-26T20:36:40.300 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T20:36:40.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T20:36:40.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:39 smithi148 ceph-mon[32281]: pgmap v1209: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:40.701 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:36:41.980 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:41 smithi157 ceph-mon[32703]: pgmap v1210: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:42.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:41 smithi131 ceph-mon[27919]: pgmap v1210: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:42.042 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:41 smithi148 ceph-mon[32281]: pgmap v1210: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:42.488 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-26T20:36:42.495 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:36:42.605 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2150, 2024-04-26T20:36:42.614 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2146, 2024-04-26T20:36:42.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 998928, 2024-04-26T20:36:42.632 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 824042, 2024-04-26T20:36:42.646 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-26T20:36:42.654 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-04-26T20:36:42.663 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.268696551, 2024-04-26T20:36:42.672 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.112318698, 2024-04-26T20:36:42.680 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.062176558, 2024-04-26T20:36:42.687 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.065945698, 2024-04-26T20:36:42.742 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:42.822 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2145, 2024-04-26T20:36:42.832 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.127091019, 2024-04-26T20:36:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000059249 2024-04-26T20:36:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:43.022 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:43.031 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:43.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.049 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 438816, 2024-04-26T20:36:43.058 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 293936 2024-04-26T20:36:43.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.076 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:36:43.085 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 4, 2024-04-26T20:36:43.094 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 364, 2024-04-26T20:36:43.144 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 876, 2024-04-26T20:36:43.154 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 858666, 2024-04-26T20:36:43.211 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2024-04-26T20:36:43.220 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-26T20:36:43.245 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.020783464, 2024-04-26T20:36:43.254 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.014815744, 2024-04-26T20:36:43.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.002014070, 2024-04-26T20:36:43.271 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000082054, 2024-04-26T20:36:43.320 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:43.329 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 362, 2024-04-26T20:36:43.337 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.027086151, 2024-04-26T20:36:43.348 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000074823 2024-04-26T20:36:43.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.365 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:43.373 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T20:36:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 857968 2024-04-26T20:36:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.537 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:36:43.546 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3112, 2024-04-26T20:36:43.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3112, 2024-04-26T20:36:43.563 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 796118, 2024-04-26T20:36:43.571 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 988799, 2024-04-26T20:36:43.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-26T20:36:43.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-26T20:36:43.645 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.203762929, 2024-04-26T20:36:43.653 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.046588644, 2024-04-26T20:36:43.662 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.058025433, 2024-04-26T20:36:43.670 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.086512248, 2024-04-26T20:36:43.679 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:43.687 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3111, 2024-04-26T20:36:43.712 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.080663997, 2024-04-26T20:36:43.720 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000025928 2024-04-26T20:36:43.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.779 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:43.814 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:43.821 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:43.828 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:43.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.909 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T20:36:43.916 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T20:36:43.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.931 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-26T20:36:43.938 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-26T20:36:43.946 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-26T20:36:43.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:43.963 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-26T20:36:43.972 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-26T20:36:44.001 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-26T20:36:44.009 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8093, 2024-04-26T20:36:44.075 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.203236111, 2024-04-26T20:36:44.084 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000025112 2024-04-26T20:36:44.093 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:44.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:44.111 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-26T20:36:44.128 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-26T20:36:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-26T20:36:44.212 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:44.229 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:44.237 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:44.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:44.252 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-26T20:36:44.261 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-04-26T20:36:44.268 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-04-26T20:36:44.323 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-26T20:36:44.330 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:44.414 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:44.422 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:44.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:44.487 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-26T20:36:44.493 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-26T20:36:44.501 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-26T20:36:44.556 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-26T20:36:44.566 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-04-26T20:36:44.574 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-26T20:36:44.582 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-26T20:36:44.590 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 0, 2024-04-26T20:36:44.598 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-26T20:36:44.606 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 0, 2024-04-26T20:36:44.614 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-26T20:36:44.621 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 0, 2024-04-26T20:36:44.629 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-26T20:36:44.679 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-04-26T20:36:44.732 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-04-26T20:36:44.771 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 1, 2024-04-26T20:36:44.828 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-04-26T20:36:44.836 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-04-26T20:36:44.844 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-26T20:36:44.853 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-26T20:36:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-26T20:36:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-26T20:36:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-26T20:36:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-26T20:36:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-26T20:36:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-26T20:36:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-26T20:36:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-26T20:36:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-26T20:36:45.004 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-26T20:36:45.060 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-26T20:36:45.114 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-26T20:36:45.132 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-04-26T20:36:45.141 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-04-26T20:36:45.150 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-26T20:36:45.158 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T20:36:45.178 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-26T20:36:45.227 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T20:36:45.235 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-26T20:36:45.244 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-26T20:36:45.252 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T20:36:45.259 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T20:36:45.267 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-26T20:36:45.276 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-26T20:36:45.284 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T20:36:45.291 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-04-26T20:36:45.300 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T20:36:45.327 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T20:36:45.398 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T20:36:45.406 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-04-26T20:36:45.428 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-04-26T20:36:45.436 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-04-26T20:36:45.448 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-04-26T20:36:45.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:45.511 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-26T20:36:45.518 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-26T20:36:45.526 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-26T20:36:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-26T20:36:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-26T20:36:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-26T20:36:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-26T20:36:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-26T20:36:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-26T20:36:45.604 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-04-26T20:36:45.613 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-04-26T20:36:45.667 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-26T20:36:45.675 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-26T20:36:45.695 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-26T20:36:45.703 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-26T20:36:45.712 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-26T20:36:45.720 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-26T20:36:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-26T20:36:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-04-26T20:36:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-04-26T20:36:45.796 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T20:36:45.806 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-26T20:36:45.814 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-26T20:36:45.822 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-26T20:36:45.830 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-26T20:36:45.836 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-26T20:36:45.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:45.881 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-26T20:36:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-26T20:36:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-04-26T20:36:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-26T20:36:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-26T20:36:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 4, 2024-04-26T20:36:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-26T20:36:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-26T20:36:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-04-26T20:36:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-26T20:36:46.103 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-26T20:36:46.111 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-04-26T20:36:46.120 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-04-26T20:36:46.128 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-26T20:36:46.137 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-26T20:36:46.145 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-26T20:36:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4195593, 2024-04-26T20:36:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 4195283, 2024-04-26T20:36:46.267 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-26T20:36:46.274 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 4 2024-04-26T20:36:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-26T20:36:46.425 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 3, 2024-04-26T20:36:46.433 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 3, 2024-04-26T20:36:46.442 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 0, 2024-04-26T20:36:46.450 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 1, 2024-04-26T20:36:46.459 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 1, 2024-04-26T20:36:46.467 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 0, 2024-04-26T20:36:46.476 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 0, 2024-04-26T20:36:46.526 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 0, 2024-04-26T20:36:46.534 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 0, 2024-04-26T20:36:46.617 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-04-26T20:36:46.625 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-04-26T20:36:46.645 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-04-26T20:36:46.653 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 49548, 2024-04-26T20:36:46.661 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215416 2024-04-26T20:36:46.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:46.725 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-26T20:36:46.733 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-04-26T20:36:46.741 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-26T20:36:46.751 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-04-26T20:36:46.759 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-04-26T20:36:46.768 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-26T20:36:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-26T20:36:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:46.911 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-26T20:36:46.919 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:46.994 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.003 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.020 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-26T20:36:47.077 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.085 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.094 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-26T20:36:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.154 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-26T20:36:47.207 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.252 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.271 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.320 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-26T20:36:47.329 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.338 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.469 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.487 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-26T20:36:47.496 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.505 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.514 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.531 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-26T20:36:47.538 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.546 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.617 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.725 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-26T20:36:47.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.769 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.776 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.872 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-26T20:36:47.880 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.888 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.897 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:47.914 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-26T20:36:47.922 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:47.930 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:47.939 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:47.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.002 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-26T20:36:48.057 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.068 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:43 smithi131 ceph-mon[27919]: pgmap v1211: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:48.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:45 smithi131 ceph-mon[27919]: pgmap v1212: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:48.102 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:47 smithi131 ceph-mon[27919]: pgmap v1213: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:48.111 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:43 smithi157 ceph-mon[32703]: pgmap v1211: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:48.120 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:45 smithi157 ceph-mon[32703]: pgmap v1212: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:48.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:47 smithi157 ceph-mon[32703]: pgmap v1213: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:48.187 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.195 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.211 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-26T20:36:48.219 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.228 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.237 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.306 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-26T20:36:48.315 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.369 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.377 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.410 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-26T20:36:48.419 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.427 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.540 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-26T20:36:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.556 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.564 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.580 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-26T20:36:48.588 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.611 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.619 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.719 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-26T20:36:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-26T20:36:48.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.799 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.844 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.865 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-26T20:36:48.873 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:48.880 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:48.888 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:48.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:48.958 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-26T20:36:49.018 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.025 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.068 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.082 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-26T20:36:49.090 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.162 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.169 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.182 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-26T20:36:49.189 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.196 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.203 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.217 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-26T20:36:49.271 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.278 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.349 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.393 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-26T20:36:49.403 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.412 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.420 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.494 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-26T20:36:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-26T20:36:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.609 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-26T20:36:49.694 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.718 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.727 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.747 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-26T20:36:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-26T20:36:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-26T20:36:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-26T20:36:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:49.895 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:49.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:49.970 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-26T20:36:49.993 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 2, 2024-04-26T20:36:50.005 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 2, 2024-04-26T20:36:50.013 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-04-26T20:36:50.021 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 2, 2024-04-26T20:36:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-26T20:36:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-04-26T20:36:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-04-26T20:36:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 3302, 2024-04-26T20:36:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-26T20:36:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-26T20:36:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:36:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-26T20:36:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:36:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:36:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:36:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:36:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:36:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:36:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:36:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:36:50.404 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:36:50.413 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:36:50.422 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:36:50.430 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:36:50.438 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:36:50.447 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-26T20:36:50.456 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:36:50.465 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-26T20:36:50.473 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:36:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:36:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:36:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:36:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:36:50.708 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:36:50.718 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:36:50.727 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-26T20:36:50.736 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:36:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:36:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:36:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-26T20:36:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-26T20:36:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-26T20:36:50.828 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:36:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:36:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:36:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:36:50.903 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 182065, 2024-04-26T20:36:50.910 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 48, 2024-04-26T20:36:50.969 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:36:50.976 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-26T20:36:51.045 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-26T20:36:51.053 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-26T20:36:51.059 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:36:51.066 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-26T20:36:51.119 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:36:51.126 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-26T20:36:51.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 9184, 2024-04-26T20:36:51.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 177, 2024-04-26T20:36:51.147 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:36:51.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:36:51.162 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-26T20:36:51.169 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-26T20:36:51.176 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 8512, 2024-04-26T20:36:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 13, 2024-04-26T20:36:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-26T20:36:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-26T20:36:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-26T20:36:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-26T20:36:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-26T20:36:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-26T20:36:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-26T20:36:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 4777, 2024-04-26T20:36:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 21, 2024-04-26T20:36:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 3, 2024-04-26T20:36:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 4777, 2024-04-26T20:36:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-26T20:36:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4777, 2024-04-26T20:36:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.479937404, 2024-04-26T20:36:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000519141 2024-04-26T20:36:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-26T20:36:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-26T20:36:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4777, 2024-04-26T20:36:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4778 2024-04-26T20:36:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 4777, 2024-04-26T20:36:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 4777, 2024-04-26T20:36:51.801 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-04-26T20:36:51.811 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-26T20:36:51.819 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-26T20:36:51.828 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 3102, 2024-04-26T20:36:51.837 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-04-26T20:36:51.846 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 1673, 2024-04-26T20:36:51.854 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-04-26T20:36:51.863 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-04-26T20:36:51.871 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-26T20:36:51.934 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-26T20:36:51.943 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-26T20:36:52.007 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-26T20:36:52.017 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-04-26T20:36:52.026 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-26T20:36:52.034 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-26T20:36:52.042 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-26T20:36:52.069 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 1, 2024-04-26T20:36:52.118 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-04-26T20:36:52.126 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-26T20:36:52.134 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-26T20:36:52.143 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-26T20:36:52.152 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-26T20:36:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-26T20:36:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-26T20:36:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T20:36:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-26T20:36:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-26T20:36:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-26T20:36:52.275 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-26T20:36:52.283 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-26T20:36:52.311 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-26T20:36:52.320 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-26T20:36:52.329 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-26T20:36:52.337 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-26T20:36:52.384 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-26T20:36:52.393 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-26T20:36:52.402 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-26T20:36:52.410 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-26T20:36:52.419 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-26T20:36:52.428 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-26T20:36:52.436 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-26T20:36:52.446 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-26T20:36:52.455 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-26T20:36:52.514 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-26T20:36:52.523 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 97, 2024-04-26T20:36:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-26T20:36:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-04-26T20:36:52.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 4, 2024-04-26T20:36:52.616 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 6, 2024-04-26T20:36:52.624 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 2, 2024-04-26T20:36:52.633 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-26T20:36:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-04-26T20:36:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-04-26T20:36:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-04-26T20:36:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-26T20:36:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-04-26T20:36:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-26T20:36:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-26T20:36:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-26T20:36:52.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:52.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-26T20:36:52.826 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-26T20:36:52.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-26T20:36:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-26T20:36:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T20:36:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-26T20:36:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-04-26T20:36:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-04-26T20:36:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-04-26T20:36:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-26T20:36:52.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:52.978 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:36:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:52.993 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-26T20:36:53.001 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:53.009 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 5262, 2024-04-26T20:36:53.065 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1294443, 2024-04-26T20:36:53.124 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:53.133 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 5262, 2024-04-26T20:36:53.159 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:53.168 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:53.178 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 5262, 2024-04-26T20:36:53.186 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1294443, 2024-04-26T20:36:53.242 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:53.250 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:53.257 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:53.264 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:53.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:53.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:53.285 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-26T20:36:53.291 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:53.298 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-26T20:36:53.321 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:53.328 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-26T20:36:53.388 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-26T20:36:53.396 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4777, 2024-04-26T20:36:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 1294, 2024-04-26T20:36:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4777, 2024-04-26T20:36:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1294, 2024-04-26T20:36:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:53.531 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:53.538 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:53.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:53.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:53.564 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-26T20:36:53.573 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:53.600 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-26T20:36:53.609 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:53.677 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-26T20:36:53.686 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-26T20:36:53.725 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:53.734 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:53.743 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4777, 2024-04-26T20:36:53.752 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 4777, 2024-04-26T20:36:53.817 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4777, 2024-04-26T20:36:53.827 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4777, 2024-04-26T20:36:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-26T20:36:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-26T20:36:54.018 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:54.027 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-26T20:36:54.069 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-26T20:36:54.089 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:54.098 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:54.106 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:54.150 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:54.159 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-26T20:36:54.167 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-26T20:36:54.176 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:54.184 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:54.193 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:54.202 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:54.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:54.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:54.277 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x563765a4a6e0": { 2024-04-26T20:36:54.284 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:54.345 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-26T20:36:54.354 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-26T20:36:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-26T20:36:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-26T20:36:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-26T20:36:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-26T20:36:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:54.575 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:54.582 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-26T20:36:54.626 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T20:36:54.634 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-26T20:36:54.669 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T20:36:54.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T20:36:54.687 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:43 smithi148 ceph-mon[32281]: pgmap v1211: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:54.694 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:45 smithi148 ceph-mon[32281]: pgmap v1212: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:54.751 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:47 smithi148 ceph-mon[32281]: pgmap v1213: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:54.759 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:49 smithi148 ceph-mon[32281]: pgmap v1214: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:54.769 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:51 smithi148 ceph-mon[32281]: pgmap v1215: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:54.779 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:53 smithi148 ceph-mon[32281]: pgmap v1216: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:54.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:49 smithi131 ceph-mon[27919]: pgmap v1214: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:54.798 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:51 smithi131 ceph-mon[27919]: pgmap v1215: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:54.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:53 smithi131 ceph-mon[27919]: pgmap v1216: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:54.817 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:49 smithi157 ceph-mon[32703]: pgmap v1214: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:36:54.826 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:51 smithi157 ceph-mon[32703]: pgmap v1215: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:36:54.890 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:53 smithi157 ceph-mon[32703]: pgmap v1216: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:55.043 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T20:36:56.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:55 smithi131 ceph-mon[27919]: pgmap v1217: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:56.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:55 smithi148 ceph-mon[32281]: pgmap v1217: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:56.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:55 smithi157 ceph-mon[32703]: pgmap v1217: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:36:56.721 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-26T20:36:56.731 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:36:56.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 483, 2024-04-26T20:36:56.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 481, 2024-04-26T20:36:56.816 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 418064, 2024-04-26T20:36:56.826 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 298208, 2024-04-26T20:36:56.836 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-04-26T20:36:56.845 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 0, 2024-04-26T20:36:56.854 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.077108427, 2024-04-26T20:36:56.865 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.033963111, 2024-04-26T20:36:56.874 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.022030552, 2024-04-26T20:36:56.883 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.012474007, 2024-04-26T20:36:56.943 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:56.952 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 480, 2024-04-26T20:36:57.034 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.025018061, 2024-04-26T20:36:57.042 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000052120 2024-04-26T20:36:57.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:57.092 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:57.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:57.111 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:57.158 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:57.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:57.173 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 418064, 2024-04-26T20:36:57.180 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 298208 2024-04-26T20:36:57.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:57.195 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:36:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 6, 2024-04-26T20:36:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 371, 2024-04-26T20:36:57.216 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 3264, 2024-04-26T20:36:57.274 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 301344, 2024-04-26T20:36:57.281 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-26T20:36:57.341 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 2, 2024-04-26T20:36:57.348 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.023358738, 2024-04-26T20:36:57.376 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.017572515, 2024-04-26T20:36:57.384 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.001570984, 2024-04-26T20:36:57.391 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:36:57.397 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:57.458 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 370, 2024-04-26T20:36:57.468 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.027527763, 2024-04-26T20:36:57.476 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000074399 2024-04-26T20:36:57.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:57.494 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:57.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:57.512 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:57.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:57.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:57.563 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T20:36:57.573 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 301344 2024-04-26T20:36:57.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:36:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 0, 2024-04-26T20:36:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2, 2024-04-26T20:36:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 0, 2024-04-26T20:36:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 0, 2024-04-26T20:36:57.791 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 2, 2024-04-26T20:36:57.800 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-04-26T20:36:57.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.000234587, 2024-04-26T20:36:57.818 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T20:36:57.827 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.000162532, 2024-04-26T20:36:57.836 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:36:57.844 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:36:57.853 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:57.862 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:57.918 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:57.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:57.991 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:36:58.001 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:58.039 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:58.048 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:58.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:58.062 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:36:58.108 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:36:58.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:58.123 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-26T20:36:58.133 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 0, 2024-04-26T20:36:58.144 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-26T20:36:58.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:58.162 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-26T20:36:58.173 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:36:58.183 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-26T20:36:58.193 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:36:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:36:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:36:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:36:58.358 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:36:58.368 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:36:58.378 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:36:58.389 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:36:58.433 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:36:58.440 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:36:58.450 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:36:58.458 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:36:58.468 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:36:58.477 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-26T20:36:58.484 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:36:58.494 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-26T20:36:58.504 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:36:58.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:36:58.578 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:36:58.641 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:36:58.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:36:58.674 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:36:58.684 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:36:58.696 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-26T20:36:58.706 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:36:58.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:36:58.784 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:36:58.793 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-26T20:36:58.803 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-26T20:36:58.813 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-26T20:36:58.822 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:36:58.831 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:36:58.840 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:36:58.849 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:36:58.856 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 148682, 2024-04-26T20:36:58.920 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 29, 2024-04-26T20:36:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:36:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-26T20:36:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-26T20:36:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-26T20:36:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:36:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-26T20:36:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:36:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-26T20:36:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 0, 2024-04-26T20:36:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 0, 2024-04-26T20:36:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:36:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:36:59.156 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-26T20:36:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-26T20:36:59.173 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 928, 2024-04-26T20:36:59.214 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 2, 2024-04-26T20:36:59.227 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-26T20:36:59.288 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-26T20:36:59.298 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-26T20:36:59.330 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-26T20:36:59.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-26T20:36:59.347 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:36:59.358 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-26T20:36:59.416 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-26T20:36:59.425 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-26T20:36:59.433 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 485, 2024-04-26T20:36:59.442 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 349741, 2024-04-26T20:36:59.450 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-26T20:36:59.458 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 485, 2024-04-26T20:36:59.467 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-26T20:36:59.477 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-26T20:36:59.486 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 485, 2024-04-26T20:36:59.545 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 349741, 2024-04-26T20:36:59.553 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-26T20:36:59.607 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-26T20:36:59.625 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-26T20:36:59.650 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-26T20:36:59.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:59.672 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-26T20:36:59.682 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-26T20:36:59.733 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:57 smithi157 ceph-mon[32703]: pgmap v1218: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:59.742 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:57 smithi131 ceph-mon[27919]: pgmap v1218: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:36:59.753 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-26T20:36:59.762 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-26T20:36:59.771 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T20:36:59.781 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T20:36:59.791 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:57 smithi148 ceph-mon[32281]: pgmap v1218: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:00.048 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:37:00.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:36:59 smithi148 ceph-mon[32281]: pgmap v1219: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:00.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:36:59 smithi157 ceph-mon[32703]: pgmap v1219: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:00.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:36:59 smithi131 ceph-mon[27919]: pgmap v1219: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:02.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:01 smithi157 ceph-mon[32703]: pgmap v1220: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:02.020 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-26T20:37:02.050 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:37:02.058 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1690, 2024-04-26T20:37:02.065 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1692, 2024-04-26T20:37:02.072 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 444285, 2024-04-26T20:37:02.158 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 537484, 2024-04-26T20:37:02.165 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-04-26T20:37:02.173 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-04-26T20:37:02.181 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.181871011, 2024-04-26T20:37:02.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.052283946, 2024-04-26T20:37:02.197 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.050296417, 2024-04-26T20:37:02.204 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.061946022, 2024-04-26T20:37:02.211 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:02.219 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1690, 2024-04-26T20:37:02.225 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.073522124, 2024-04-26T20:37:02.281 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000043504 2024-04-26T20:37:02.383 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:02.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:02.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:02.436 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:02.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:02.450 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:02.500 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:37:02.508 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:37:02.515 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:02.523 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:37:02.530 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1335, 2024-04-26T20:37:02.537 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1698, 2024-04-26T20:37:02.545 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 330220, 2024-04-26T20:37:02.552 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 1288387, 2024-04-26T20:37:02.559 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-04-26T20:37:02.611 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-04-26T20:37:02.618 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.169481442, 2024-04-26T20:37:02.665 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.053457091, 2024-04-26T20:37:02.672 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.039354289, 2024-04-26T20:37:02.703 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.061188464, 2024-04-26T20:37:02.711 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:02.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1697, 2024-04-26T20:37:02.725 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.071993789, 2024-04-26T20:37:02.765 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000042424 2024-04-26T20:37:02.772 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:02.779 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:02.786 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:02.794 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:02.801 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:02.808 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:02.816 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T20:37:02.824 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 865760 2024-04-26T20:37:02.883 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:02.891 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:37:02.962 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2174, 2024-04-26T20:37:02.971 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2170, 2024-04-26T20:37:03.003 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1007232, 2024-04-26T20:37:03.011 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 833302, 2024-04-26T20:37:03.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 3, 2024-04-26T20:37:03.030 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-04-26T20:37:03.082 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.288355362, 2024-04-26T20:37:03.091 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.099586133, 2024-04-26T20:37:03.104 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.077609244, 2024-04-26T20:37:03.114 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.082949904, 2024-04-26T20:37:03.123 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:03.133 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2169, 2024-04-26T20:37:03.142 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.130406278, 2024-04-26T20:37:03.153 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000060122 2024-04-26T20:37:03.162 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:03.224 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:03.233 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:03.281 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:03.291 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:03.327 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 441072, 2024-04-26T20:37:03.347 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 297472 2024-04-26T20:37:03.356 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:03.407 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-26T20:37:03.418 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-04-26T20:37:03.428 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-26T20:37:03.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:03.447 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-04-26T20:37:03.457 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-26T20:37:03.466 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-26T20:37:03.476 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8015, 2024-04-26T20:37:03.485 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.263001217, 2024-04-26T20:37:03.495 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000032813 2024-04-26T20:37:03.554 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:03.621 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:03.630 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-04-26T20:37:03.680 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-26T20:37:03.691 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-26T20:37:03.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:03.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:03.756 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:03.767 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:03.776 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:03.785 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-04-26T20:37:03.794 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-04-26T20:37:03.803 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-04-26T20:37:03.811 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-04-26T20:37:03.820 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:03.830 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:03.891 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:03.899 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:03.956 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-04-26T20:37:03.965 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-04-26T20:37:03.998 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-04-26T20:37:04.006 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-04-26T20:37:04.015 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-04-26T20:37:04.025 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-04-26T20:37:04.082 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-04-26T20:37:04.091 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 0, 2024-04-26T20:37:04.098 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-04-26T20:37:04.106 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 0, 2024-04-26T20:37:04.113 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-04-26T20:37:04.120 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 0, 2024-04-26T20:37:04.126 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-04-26T20:37:04.134 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-04-26T20:37:04.142 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-04-26T20:37:04.191 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-04-26T20:37:04.198 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-04-26T20:37:04.247 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-04-26T20:37:04.255 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-04-26T20:37:04.291 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-04-26T20:37:04.298 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-04-26T20:37:04.305 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-04-26T20:37:04.311 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-04-26T20:37:04.365 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-04-26T20:37:04.373 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-04-26T20:37:04.381 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-04-26T20:37:04.388 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-04-26T20:37:04.395 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-04-26T20:37:04.403 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-04-26T20:37:04.410 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-04-26T20:37:04.417 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-04-26T20:37:04.425 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-04-26T20:37:04.448 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-04-26T20:37:04.456 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-04-26T20:37:04.510 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-04-26T20:37:04.517 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T20:37:04.547 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-04-26T20:37:04.555 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T20:37:04.562 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-04-26T20:37:04.570 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-04-26T20:37:04.625 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T20:37:04.633 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T20:37:04.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-04-26T20:37:04.648 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-04-26T20:37:04.656 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T20:37:04.662 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-04-26T20:37:04.668 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T20:37:04.675 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T20:37:04.682 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T20:37:04.689 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-04-26T20:37:04.746 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-04-26T20:37:04.804 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-04-26T20:37:04.811 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-04-26T20:37:04.865 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:04.872 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-04-26T20:37:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-04-26T20:37:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-04-26T20:37:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-04-26T20:37:04.958 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-04-26T20:37:04.966 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-04-26T20:37:04.973 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-04-26T20:37:04.981 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-04-26T20:37:04.989 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-04-26T20:37:04.999 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-04-26T20:37:05.009 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-04-26T20:37:05.019 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-04-26T20:37:05.060 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-04-26T20:37:05.071 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-04-26T20:37:05.107 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-04-26T20:37:05.145 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-04-26T20:37:05.188 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-04-26T20:37:05.200 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-04-26T20:37:05.210 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-04-26T20:37:05.220 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-04-26T20:37:05.305 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T20:37:05.314 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-04-26T20:37:05.324 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-04-26T20:37:05.334 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-04-26T20:37:05.344 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-04-26T20:37:05.353 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-04-26T20:37:05.363 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:05.373 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-04-26T20:37:05.382 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-04-26T20:37:05.391 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-04-26T20:37:05.452 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-04-26T20:37:05.496 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-04-26T20:37:05.522 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 3, 2024-04-26T20:37:05.555 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-04-26T20:37:05.567 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-04-26T20:37:05.576 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-04-26T20:37:05.588 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-04-26T20:37:05.631 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-04-26T20:37:05.640 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-04-26T20:37:05.650 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-04-26T20:37:05.659 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-04-26T20:37:05.669 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-04-26T20:37:05.678 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-04-26T20:37:05.687 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4195202, 2024-04-26T20:37:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 4194892, 2024-04-26T20:37:05.706 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-04-26T20:37:05.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:05.774 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:05.833 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:05.842 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 3 2024-04-26T20:37:05.884 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:05.893 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-04-26T20:37:05.902 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 3, 2024-04-26T20:37:05.964 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 3, 2024-04-26T20:37:05.973 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-04-26T20:37:05.982 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1, 2024-04-26T20:37:05.991 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1, 2024-04-26T20:37:05.999 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2024-04-26T20:37:06.008 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 0, 2024-04-26T20:37:06.016 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 0, 2024-04-26T20:37:06.024 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-04-26T20:37:06.033 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-04-26T20:37:06.062 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-04-26T20:37:06.071 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-04-26T20:37:06.130 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 52552, 2024-04-26T20:37:06.138 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215416 2024-04-26T20:37:06.163 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.170 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-04-26T20:37:06.177 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-04-26T20:37:06.185 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-04-26T20:37:06.223 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-04-26T20:37:06.230 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-04-26T20:37:06.237 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-04-26T20:37:06.244 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-04-26T20:37:06.251 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.261 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.269 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.276 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.284 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-04-26T20:37:06.325 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.332 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.397 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.430 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-04-26T20:37:06.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.452 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.514 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-04-26T20:37:06.521 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.528 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.536 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.542 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.549 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-04-26T20:37:06.556 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.563 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.590 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.597 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.662 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-04-26T20:37:06.669 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.697 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.719 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-04-26T20:37:06.799 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.819 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.831 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.840 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.848 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-04-26T20:37:06.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.867 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:06.890 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:06.899 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:06.931 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-04-26T20:37:06.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:06.998 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.007 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.074 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.084 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-04-26T20:37:07.093 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.101 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.189 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.199 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.208 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-04-26T20:37:07.216 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.226 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.236 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.245 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.254 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-04-26T20:37:07.263 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.326 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.335 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.399 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.411 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-04-26T20:37:07.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.456 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.465 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.475 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.544 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-04-26T20:37:07.552 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.564 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.573 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.581 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.591 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-04-26T20:37:07.598 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.607 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.615 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-04-26T20:37:07.738 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.747 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.783 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.792 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.800 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-04-26T20:37:07.808 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.847 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.864 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.875 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-04-26T20:37:07.883 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.892 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.900 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:07.908 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:07.916 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-04-26T20:37:07.947 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:07.956 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:07.998 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.006 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.043 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-04-26T20:37:08.053 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.062 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.072 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.114 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.122 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-04-26T20:37:08.130 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.136 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.144 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.151 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.158 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-04-26T20:37:08.166 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.173 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.202 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.209 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.270 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-04-26T20:37:08.278 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.309 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.317 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.323 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.330 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-04-26T20:37:08.372 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.378 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.392 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.399 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-04-26T20:37:08.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.414 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.420 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.492 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-04-26T20:37:08.499 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.598 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.606 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.643 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.649 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-04-26T20:37:08.659 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.669 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.723 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.733 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-04-26T20:37:08.743 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.753 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.763 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.773 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.783 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-04-26T20:37:08.790 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:08.800 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:08.858 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:08.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:08.961 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-04-26T20:37:09.020 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-04-26T20:37:09.030 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-04-26T20:37:09.040 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:09.049 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:09.097 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:09.105 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:09.114 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:09.122 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-04-26T20:37:09.133 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-04-26T20:37:09.142 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 1, 2024-04-26T20:37:09.152 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2024-04-26T20:37:09.161 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-04-26T20:37:09.170 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-04-26T20:37:09.227 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-04-26T20:37:09.236 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-04-26T20:37:09.308 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 3313, 2024-04-26T20:37:09.318 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-04-26T20:37:09.348 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:09.357 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-26T20:37:09.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:37:09.375 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-26T20:37:09.423 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:37:09.434 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:37:09.446 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:37:09.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:37:09.464 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:37:09.473 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:37:09.483 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:37:09.494 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:37:09.504 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:37:09.514 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:37:09.569 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:37:09.621 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:37:09.630 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:37:09.664 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-26T20:37:09.674 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:37:09.684 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-26T20:37:09.692 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:37:09.751 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:37:09.763 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:37:09.772 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:37:09.781 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:37:09.789 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:37:09.798 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:37:09.806 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-26T20:37:09.815 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:37:09.824 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:37:09.880 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:37:09.889 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-26T20:37:09.952 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-26T20:37:09.961 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-26T20:37:09.992 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:37:10.001 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:37:10.009 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:37:10.018 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:37:10.055 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 190929, 2024-04-26T20:37:10.066 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 54, 2024-04-26T20:37:10.074 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:37:10.083 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-04-26T20:37:10.091 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-26T20:37:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-26T20:37:10.108 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:37:10.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-26T20:37:10.124 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:37:10.154 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-26T20:37:10.162 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 9184, 2024-04-26T20:37:10.273 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 177, 2024-04-26T20:37:10.282 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:37:10.327 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:37:10.336 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-26T20:37:10.344 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-26T20:37:10.353 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 8512, 2024-04-26T20:37:10.396 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 13, 2024-04-26T20:37:10.405 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-26T20:37:10.412 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-26T20:37:10.420 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-26T20:37:10.428 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-26T20:37:10.436 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:10.443 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-04-26T20:37:10.451 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-04-26T20:37:10.458 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-04-26T20:37:10.509 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 4704, 2024-04-26T20:37:10.518 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-04-26T20:37:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-04-26T20:37:10.585 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 4704, 2024-04-26T20:37:10.647 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-04-26T20:37:10.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4704, 2024-04-26T20:37:10.670 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 4.176920982, 2024-04-26T20:37:10.680 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000887950 2024-04-26T20:37:10.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:10.740 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-04-26T20:37:10.750 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-04-26T20:37:10.759 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4704, 2024-04-26T20:37:10.769 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 4705 2024-04-26T20:37:10.779 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:10.789 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 4704, 2024-04-26T20:37:10.799 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 4704, 2024-04-26T20:37:10.808 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-04-26T20:37:10.865 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-04-26T20:37:10.879 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-04-26T20:37:10.944 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 3012, 2024-04-26T20:37:10.951 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-04-26T20:37:10.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 1690, 2024-04-26T20:37:10.996 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-04-26T20:37:11.004 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-04-26T20:37:11.015 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-04-26T20:37:11.055 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-04-26T20:37:11.064 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-04-26T20:37:11.072 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-04-26T20:37:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-04-26T20:37:11.091 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-04-26T20:37:11.099 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-04-26T20:37:11.109 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-04-26T20:37:11.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-04-26T20:37:11.126 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-04-26T20:37:11.178 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-04-26T20:37:11.187 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-04-26T20:37:11.254 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-04-26T20:37:11.261 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-04-26T20:37:11.305 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-04-26T20:37:11.314 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-04-26T20:37:11.323 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T20:37:11.333 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-04-26T20:37:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-04-26T20:37:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-04-26T20:37:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-04-26T20:37:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-04-26T20:37:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-04-26T20:37:11.412 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-04-26T20:37:11.422 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-04-26T20:37:11.431 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-04-26T20:37:11.440 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-04-26T20:37:11.492 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-04-26T20:37:11.502 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-04-26T20:37:11.556 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-04-26T20:37:11.565 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-04-26T20:37:11.619 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-04-26T20:37:11.628 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-04-26T20:37:11.637 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-04-26T20:37:11.646 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-04-26T20:37:11.696 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-04-26T20:37:11.705 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 97, 2024-04-26T20:37:11.713 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-04-26T20:37:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 2, 2024-04-26T20:37:11.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 5, 2024-04-26T20:37:11.738 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 6, 2024-04-26T20:37:11.746 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 1, 2024-04-26T20:37:11.754 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-04-26T20:37:11.763 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-04-26T20:37:11.816 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-04-26T20:37:11.826 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-04-26T20:37:11.884 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:11.894 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-04-26T20:37:11.947 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-04-26T20:37:11.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-04-26T20:37:11.966 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-04-26T20:37:11.977 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-04-26T20:37:12.013 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:12.022 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-04-26T20:37:12.031 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-04-26T20:37:12.039 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:12.048 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-04-26T20:37:12.057 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-04-26T20:37:12.067 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T20:37:12.075 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-04-26T20:37:12.084 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-04-26T20:37:12.137 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-04-26T20:37:12.145 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-04-26T20:37:12.202 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-04-26T20:37:12.211 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:12.238 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:37:12.247 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:12.258 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-26T20:37:12.266 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:12.304 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 5195, 2024-04-26T20:37:12.314 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1285989, 2024-04-26T20:37:12.322 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:12.330 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 5195, 2024-04-26T20:37:12.339 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:12.348 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:12.357 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5195, 2024-04-26T20:37:12.365 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1285989, 2024-04-26T20:37:12.373 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:12.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:12.435 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:12.504 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:12.512 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:12.555 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:12.566 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-04-26T20:37:12.576 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:12.586 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-26T20:37:12.637 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:12.648 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:12.658 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:12.670 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:12.680 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:12.687 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 4704, 2024-04-26T20:37:12.696 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 903, 2024-04-26T20:37:12.705 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 4704, 2024-04-26T20:37:12.714 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 903, 2024-04-26T20:37:12.757 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:12.767 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:12.868 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:12.876 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:12.954 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:12.963 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:12.972 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-04-26T20:37:12.982 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:13.136 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-04-26T20:37:13.143 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:13.153 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:13.164 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:13.172 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:13.180 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:13.188 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 4704, 2024-04-26T20:37:13.196 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 4704, 2024-04-26T20:37:13.206 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 4704, 2024-04-26T20:37:13.257 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 4704, 2024-04-26T20:37:13.266 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:13.330 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:13.360 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:13.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:13.417 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:13.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:13.435 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-04-26T20:37:13.487 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:13.494 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-26T20:37:13.503 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:13.512 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:13.523 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:13.532 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:13.541 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:13.548 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:13.555 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:13.585 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-26T20:37:13.592 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-26T20:37:13.675 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:13.682 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:13.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:13.722 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:13.730 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:13.737 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:13.777 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55eca8f506e0": { 2024-04-26T20:37:13.784 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:13.791 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-26T20:37:13.799 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:13.806 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:13.814 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:13.821 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:13.828 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:13.836 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:13.891 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:13.898 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-26T20:37:13.972 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-26T20:37:13.983 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:14.017 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:14.027 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:14.036 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:14.046 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:14.117 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:14.126 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-26T20:37:14.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:01 smithi131 ceph-mon[27919]: pgmap v1220: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:14.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:03 smithi131 ceph-mon[27919]: pgmap v1221: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.156 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:05 smithi131 ceph-mon[27919]: pgmap v1222: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:14.166 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:07 smithi131 ceph-mon[27919]: pgmap v1223: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.176 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:09 smithi131 ceph-mon[27919]: pgmap v1224: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:14.184 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:11 smithi131 ceph-mon[27919]: pgmap v1225: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:14.195 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-26T20:37:14.246 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-26T20:37:14.256 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T20:37:14.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T20:37:14.338 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:01 smithi148 ceph-mon[32281]: pgmap v1220: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:14.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:03 smithi148 ceph-mon[32281]: pgmap v1221: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:05 smithi148 ceph-mon[32281]: pgmap v1222: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:14.395 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:07 smithi148 ceph-mon[32281]: pgmap v1223: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.405 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:09 smithi148 ceph-mon[32281]: pgmap v1224: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:14.456 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:11 smithi148 ceph-mon[32281]: pgmap v1225: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:14.465 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:13 smithi148 ceph-mon[32281]: pgmap v1226: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.475 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:03 smithi157 ceph-mon[32703]: pgmap v1221: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.484 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:05 smithi157 ceph-mon[32703]: pgmap v1222: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:14.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:07 smithi157 ceph-mon[32703]: pgmap v1223: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:09 smithi157 ceph-mon[32703]: pgmap v1224: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:14.513 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:11 smithi157 ceph-mon[32703]: pgmap v1225: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:14.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:13 smithi157 ceph-mon[32703]: pgmap v1226: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:13 smithi131 ceph-mon[27919]: pgmap v1226: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:14.783 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:37:16.059 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:15 smithi131 ceph-mon[27919]: pgmap v1227: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:16.086 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:15 smithi148 ceph-mon[32281]: pgmap v1227: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:16.099 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:15 smithi157 ceph-mon[32703]: pgmap v1227: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:16.525 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-26T20:37:16.534 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:37:16.543 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 0, 2024-04-26T20:37:16.552 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2, 2024-04-26T20:37:16.611 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 0, 2024-04-26T20:37:16.621 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 0, 2024-04-26T20:37:16.629 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-04-26T20:37:16.637 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 0, 2024-04-26T20:37:16.645 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.001124447, 2024-04-26T20:37:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T20:37:16.662 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.000149794, 2024-04-26T20:37:16.670 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:37:16.679 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:16.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:16.741 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:16.818 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:16.827 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:16.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:16.858 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:16.867 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:16.876 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:16.925 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:16.934 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:37:16.943 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:37:16.949 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:16.957 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:37:16.964 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 485, 2024-04-26T20:37:16.972 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 483, 2024-04-26T20:37:16.980 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 418608, 2024-04-26T20:37:16.989 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 299488, 2024-04-26T20:37:17.047 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 2, 2024-04-26T20:37:17.062 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-04-26T20:37:17.121 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.075428330, 2024-04-26T20:37:17.130 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.031661305, 2024-04-26T20:37:17.159 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.022688762, 2024-04-26T20:37:17.168 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.013230229, 2024-04-26T20:37:17.178 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:17.186 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 482, 2024-04-26T20:37:17.242 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.028479168, 2024-04-26T20:37:17.252 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000059085 2024-04-26T20:37:17.260 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:17.272 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:17.279 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:17.289 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:17.299 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:17.323 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:17.330 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 418608, 2024-04-26T20:37:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 299488 2024-04-26T20:37:17.397 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:17.451 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:37:17.458 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 8, 2024-04-26T20:37:17.479 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 374, 2024-04-26T20:37:17.489 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 3584, 2024-04-26T20:37:17.499 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 380640, 2024-04-26T20:37:17.508 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-04-26T20:37:17.553 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-04-26T20:37:17.563 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.025932786, 2024-04-26T20:37:17.572 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.019201442, 2024-04-26T20:37:17.579 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.002129604, 2024-04-26T20:37:17.589 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:37:17.597 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:17.605 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 373, 2024-04-26T20:37:17.613 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.028746867, 2024-04-26T20:37:17.622 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000077069 2024-04-26T20:37:17.650 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:17.659 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:17.766 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:17.774 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:17.783 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3584, 2024-04-26T20:37:17.791 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 380640 2024-04-26T20:37:17.836 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:17.846 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-26T20:37:17.855 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 0, 2024-04-26T20:37:17.863 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-26T20:37:17.872 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:17.881 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-26T20:37:17.890 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:37:17.899 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-26T20:37:17.908 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:37:17.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:37:17.938 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:37:17.995 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:37:18.003 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:37:18.028 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:37:18.037 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:37:18.046 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:37:18.055 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:37:18.105 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:37:18.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:37:18.123 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:37:18.132 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:37:18.140 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-26T20:37:18.150 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:37:18.158 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-26T20:37:18.166 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:37:18.175 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:37:18.184 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:37:18.238 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:37:18.291 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:37:18.301 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:37:18.320 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:37:18.328 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-26T20:37:18.337 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:37:18.345 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:37:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:37:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-26T20:37:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-26T20:37:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-26T20:37:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:37:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:37:18.470 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:37:18.478 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:37:18.487 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 147898, 2024-04-26T20:37:18.517 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 27, 2024-04-26T20:37:18.525 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:37:18.602 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-04-26T20:37:18.611 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-26T20:37:18.637 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-26T20:37:18.646 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:37:18.658 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-26T20:37:18.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:37:18.675 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-26T20:37:18.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 0, 2024-04-26T20:37:18.719 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 0, 2024-04-26T20:37:18.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:37:18.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:37:18.745 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-26T20:37:18.754 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-26T20:37:18.763 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 928, 2024-04-26T20:37:18.771 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2, 2024-04-26T20:37:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-26T20:37:18.834 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-26T20:37:18.900 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-26T20:37:18.910 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-26T20:37:18.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:18.959 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:37:18.968 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:18.976 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-26T20:37:19.035 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:19.044 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 489, 2024-04-26T20:37:19.052 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 350055, 2024-04-26T20:37:19.062 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:19.071 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 489, 2024-04-26T20:37:19.080 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:19.089 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:19.097 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 489, 2024-04-26T20:37:19.107 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 350055, 2024-04-26T20:37:19.165 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:19.174 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:19.248 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:19.257 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:19.295 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:19.304 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:19.313 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-26T20:37:19.322 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:17 smithi131 ceph-mon[27919]: pgmap v1228: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:19.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:17 smithi148 ceph-mon[32281]: pgmap v1228: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:19.387 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-04-26T20:37:19.397 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-04-26T20:37:19.406 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T20:37:19.416 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T20:37:19.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:17 smithi157 ceph-mon[32703]: pgmap v1228: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:19.688 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:37:20.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:19 smithi148 ceph-mon[32281]: pgmap v1229: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:20.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:19 smithi157 ceph-mon[32703]: pgmap v1229: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:20.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:19 smithi131 ceph-mon[27919]: pgmap v1229: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:21.343 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-26T20:37:21.372 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:37:21.381 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 3474, 2024-04-26T20:37:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 3476, 2024-04-26T20:37:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 5220211, 2024-04-26T20:37:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 1104742, 2024-04-26T20:37:21.454 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-04-26T20:37:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-04-26T20:37:21.471 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.361246395, 2024-04-26T20:37:21.480 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.114287971, 2024-04-26T20:37:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.091591798, 2024-04-26T20:37:21.589 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.122112478, 2024-04-26T20:37:21.598 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:21.607 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3474, 2024-04-26T20:37:21.617 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.144712426, 2024-04-26T20:37:21.626 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000041655 2024-04-26T20:37:21.635 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:21.679 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:21.765 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:21.775 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:21.828 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:21.837 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:21.846 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T20:37:21.856 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T20:37:21.864 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:21.872 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:37:21.910 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1186, 2024-04-26T20:37:21.918 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1182, 2024-04-26T20:37:21.926 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 5243912, 2024-04-26T20:37:21.935 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 520314, 2024-04-26T20:37:21.944 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-04-26T20:37:21.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-04-26T20:37:21.961 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.151848226, 2024-04-26T20:37:21.970 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.054520806, 2024-04-26T20:37:22.026 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.045728196, 2024-04-26T20:37:22.081 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.037621806, 2024-04-26T20:37:22.089 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:22.111 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1181, 2024-04-26T20:37:22.120 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.061087968, 2024-04-26T20:37:22.129 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000051725 2024-04-26T20:37:22.137 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.199 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:22.209 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:22.218 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:22.226 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:22.235 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 442320, 2024-04-26T20:37:22.252 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 300192 2024-04-26T20:37:22.261 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.269 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:37:22.304 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1114, 2024-04-26T20:37:22.329 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1479, 2024-04-26T20:37:22.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 4470377, 2024-04-26T20:37:22.398 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 1222987, 2024-04-26T20:37:22.436 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-04-26T20:37:22.445 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-04-26T20:37:22.456 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.128987541, 2024-04-26T20:37:22.464 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.056126838, 2024-04-26T20:37:22.551 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.024544486, 2024-04-26T20:37:22.560 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.034934721, 2024-04-26T20:37:22.569 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:22.578 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1477, 2024-04-26T20:37:22.587 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.063646575, 2024-04-26T20:37:22.596 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000043091 2024-04-26T20:37:22.604 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.613 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:22.622 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:22.630 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:22.684 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:22.743 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.750 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-04-26T20:37:22.769 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 870960 2024-04-26T20:37:22.777 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.784 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-26T20:37:22.792 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-04-26T20:37:22.822 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-26T20:37:22.829 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.837 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-04-26T20:37:22.844 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-26T20:37:22.852 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-26T20:37:22.859 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8785, 2024-04-26T20:37:22.866 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.272349563, 2024-04-26T20:37:22.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000031001 2024-04-26T20:37:22.882 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:22.939 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:22.946 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-04-26T20:37:23.006 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-26T20:37:23.014 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-26T20:37:23.036 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:23.044 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:23.052 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:23.059 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:23.110 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:23.117 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-04-26T20:37:23.125 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-04-26T20:37:23.133 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-04-26T20:37:23.142 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-04-26T20:37:23.149 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:23.158 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:23.167 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:23.176 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:23.202 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-04-26T20:37:23.210 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-04-26T20:37:23.258 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-04-26T20:37:23.267 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-04-26T20:37:23.299 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-04-26T20:37:23.307 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-04-26T20:37:23.316 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-04-26T20:37:23.324 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 1235, 2024-04-26T20:37:23.332 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 852, 2024-04-26T20:37:23.341 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 383, 2024-04-26T20:37:23.350 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-04-26T20:37:23.358 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 17, 2024-04-26T20:37:23.385 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-04-26T20:37:23.394 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-04-26T20:37:23.404 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-04-26T20:37:23.412 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2024-04-26T20:37:23.420 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-04-26T20:37:23.449 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-04-26T20:37:23.458 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-04-26T20:37:23.513 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-04-26T20:37:23.521 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-04-26T20:37:23.568 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-04-26T20:37:23.577 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-04-26T20:37:23.586 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-04-26T20:37:23.596 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-04-26T20:37:23.662 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-04-26T20:37:23.672 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-04-26T20:37:23.680 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-04-26T20:37:23.689 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-04-26T20:37:23.698 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-04-26T20:37:23.707 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-04-26T20:37:23.715 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-04-26T20:37:23.724 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-04-26T20:37:23.732 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-04-26T20:37:23.788 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-04-26T20:37:23.797 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T20:37:23.864 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-04-26T20:37:23.873 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T20:37:23.919 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-04-26T20:37:23.929 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-04-26T20:37:23.937 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T20:37:23.945 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T20:37:24.010 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-04-26T20:37:24.018 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-04-26T20:37:24.027 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T20:37:24.036 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-04-26T20:37:24.045 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T20:37:24.054 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T20:37:24.062 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T20:37:24.071 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-04-26T20:37:24.080 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-04-26T20:37:24.136 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-04-26T20:37:24.144 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-04-26T20:37:24.224 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:24.233 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-04-26T20:37:24.294 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-04-26T20:37:24.302 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-04-26T20:37:24.311 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-04-26T20:37:24.320 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-04-26T20:37:24.369 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-04-26T20:37:24.376 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-04-26T20:37:24.383 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-04-26T20:37:24.392 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-04-26T20:37:24.401 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-04-26T20:37:24.410 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-04-26T20:37:24.418 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-04-26T20:37:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-04-26T20:37:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-04-26T20:37:24.445 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-04-26T20:37:24.494 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-04-26T20:37:24.561 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-04-26T20:37:24.567 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-04-26T20:37:24.613 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-04-26T20:37:24.620 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-04-26T20:37:24.627 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T20:37:24.634 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-04-26T20:37:24.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-04-26T20:37:24.710 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-04-26T20:37:24.718 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-04-26T20:37:24.725 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-04-26T20:37:24.732 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:24.738 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-04-26T20:37:24.744 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-04-26T20:37:24.751 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-04-26T20:37:24.758 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-04-26T20:37:24.765 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-04-26T20:37:24.815 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 6236, 2024-04-26T20:37:24.867 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-04-26T20:37:24.874 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-04-26T20:37:24.935 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-04-26T20:37:24.942 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-04-26T20:37:24.950 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-04-26T20:37:24.957 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-04-26T20:37:25.069 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 72, 2024-04-26T20:37:25.076 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-04-26T20:37:25.083 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-04-26T20:37:25.093 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 17194952, 2024-04-26T20:37:25.100 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 17203552, 2024-04-26T20:37:25.106 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 17202953, 2024-04-26T20:37:25.114 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-04-26T20:37:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:25.128 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:25.187 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:25.194 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:25.254 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 6236 2024-04-26T20:37:25.263 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:25.327 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-04-26T20:37:25.334 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 23, 2024-04-26T20:37:25.342 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 1235, 2024-04-26T20:37:25.349 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 1212, 2024-04-26T20:37:25.393 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 18, 2024-04-26T20:37:25.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 24, 2024-04-26T20:37:25.408 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 6, 2024-04-26T20:37:25.414 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 1235, 2024-04-26T20:37:25.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 1242, 2024-04-26T20:37:25.429 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 7, 2024-04-26T20:37:25.437 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-04-26T20:37:25.444 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-04-26T20:37:25.451 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-04-26T20:37:25.475 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 52220, 2024-04-26T20:37:25.483 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215416 2024-04-26T20:37:25.536 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:25.544 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-04-26T20:37:25.553 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-04-26T20:37:25.607 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-04-26T20:37:25.615 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-04-26T20:37:25.624 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-04-26T20:37:25.684 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-04-26T20:37:25.692 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-04-26T20:37:25.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:25.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:25.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:25.728 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:25.737 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-04-26T20:37:25.746 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:25.754 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:25.807 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:25.816 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:25.891 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-04-26T20:37:25.900 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:25.935 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:25.948 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:25.956 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:25.965 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-04-26T20:37:26.023 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.049 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.057 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-04-26T20:37:26.066 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.075 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.084 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.092 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.148 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-04-26T20:37:26.157 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.215 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.225 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.257 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.265 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-04-26T20:37:26.273 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.282 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.334 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.342 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.351 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-04-26T20:37:26.360 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.368 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.385 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.394 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-04-26T20:37:26.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.497 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.506 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-04-26T20:37:26.546 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.554 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.563 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.572 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.627 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-04-26T20:37:26.635 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.644 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.652 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.661 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.668 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-04-26T20:37:26.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.683 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.725 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.733 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-04-26T20:37:26.788 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.795 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.826 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.834 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.842 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-04-26T20:37:26.849 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.918 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.932 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.939 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-04-26T20:37:26.946 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:26.954 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:26.961 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:26.968 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:26.976 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-04-26T20:37:26.998 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:27.005 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:27.095 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:27.102 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:27.126 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-04-26T20:37:27.134 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:27.141 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:27.149 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:27.193 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:27.200 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-04-26T20:37:27.253 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:27.451 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:27.481 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:27.496 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:27.510 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-04-26T20:37:27.530 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:27.546 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:27.629 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:27.673 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:27.722 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-04-26T20:37:27.766 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:27.774 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:27.814 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:27.822 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:27.831 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-04-26T20:37:27.840 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:27.885 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:27.892 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:27.908 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-04-26T20:37:27.915 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:27.922 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:27.930 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:27.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:27.991 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-04-26T20:37:27.998 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.083 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.089 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.136 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.144 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-04-26T20:37:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.161 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.230 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.238 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.245 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-04-26T20:37:28.253 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.260 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.266 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.273 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.281 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-04-26T20:37:28.288 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.321 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.328 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.410 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-04-26T20:37:28.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.457 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.464 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.471 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.529 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-04-26T20:37:28.535 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.543 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.549 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.557 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.565 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-04-26T20:37:28.572 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.578 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.585 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.635 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.643 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-04-26T20:37:28.700 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-04-26T20:37:28.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-04-26T20:37:28.749 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:28.757 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:28.764 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:28.773 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:28.842 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:28.851 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-04-26T20:37:28.868 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-04-26T20:37:28.878 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 3, 2024-04-26T20:37:28.886 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-04-26T20:37:28.895 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-04-26T20:37:28.903 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-04-26T20:37:28.913 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-04-26T20:37:28.922 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-04-26T20:37:28.950 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 3334, 2024-04-26T20:37:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-04-26T20:37:29.019 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:29.028 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-26T20:37:29.064 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:37:29.076 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-26T20:37:29.085 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:37:29.094 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:37:29.137 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:37:29.146 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:37:29.156 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:37:29.164 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:37:29.173 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:37:29.182 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:37:29.191 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:37:29.200 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:37:29.208 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:37:29.238 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:37:29.247 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:37:29.303 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-26T20:37:29.312 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:37:29.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-26T20:37:29.375 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:37:29.385 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:37:29.393 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:37:29.458 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:37:29.466 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:37:29.475 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:37:29.483 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:37:29.492 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-26T20:37:29.500 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:37:29.508 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:37:29.517 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:37:29.525 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-26T20:37:29.533 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-26T20:37:29.588 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-26T20:37:29.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:37:29.668 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:37:29.714 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:37:29.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:37:29.730 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 190129, 2024-04-26T20:37:29.737 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 51, 2024-04-26T20:37:29.816 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:37:29.823 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-04-26T20:37:29.831 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-26T20:37:29.837 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-26T20:37:29.844 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:37:29.852 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-26T20:37:29.859 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:37:29.867 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-26T20:37:29.874 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 9184, 2024-04-26T20:37:29.900 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 177, 2024-04-26T20:37:29.907 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:37:29.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:37:29.993 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-26T20:37:30.042 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-26T20:37:30.049 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 944122, 2024-04-26T20:37:30.056 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 5235, 2024-04-26T20:37:30.063 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-26T20:37:30.123 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-26T20:37:30.131 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-26T20:37:30.138 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-26T20:37:30.145 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:30.153 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-04-26T20:37:30.161 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-04-26T20:37:30.168 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-04-26T20:37:30.175 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 5273, 2024-04-26T20:37:30.183 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 26, 2024-04-26T20:37:30.238 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 4, 2024-04-26T20:37:30.246 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 5273, 2024-04-26T20:37:30.314 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-04-26T20:37:30.321 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5273, 2024-04-26T20:37:30.354 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.842946180, 2024-04-26T20:37:30.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000539151 2024-04-26T20:37:30.368 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:30.376 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-04-26T20:37:30.455 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-04-26T20:37:30.462 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5273, 2024-04-26T20:37:30.470 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5274 2024-04-26T20:37:30.477 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:30.486 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 5273, 2024-04-26T20:37:30.495 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 5273, 2024-04-26T20:37:30.503 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-04-26T20:37:30.512 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-04-26T20:37:30.521 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-04-26T20:37:30.578 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 3410, 2024-04-26T20:37:30.587 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-04-26T20:37:30.657 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 1860, 2024-04-26T20:37:30.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-04-26T20:37:30.709 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-04-26T20:37:30.718 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-04-26T20:37:30.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-04-26T20:37:30.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-04-26T20:37:30.805 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-04-26T20:37:30.815 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-04-26T20:37:30.825 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-04-26T20:37:30.833 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-04-26T20:37:30.842 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-04-26T20:37:30.852 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2, 2024-04-26T20:37:30.861 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-04-26T20:37:30.869 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-04-26T20:37:30.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-04-26T20:37:30.887 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-04-26T20:37:30.911 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-04-26T20:37:30.978 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-04-26T20:37:30.987 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-04-26T20:37:30.996 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T20:37:31.006 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-04-26T20:37:31.036 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-04-26T20:37:31.045 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-04-26T20:37:31.125 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-04-26T20:37:31.133 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-04-26T20:37:31.142 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-04-26T20:37:31.151 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-04-26T20:37:31.160 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-04-26T20:37:31.169 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-04-26T20:37:31.178 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-04-26T20:37:31.186 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-04-26T20:37:31.195 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-04-26T20:37:31.247 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-04-26T20:37:31.256 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-04-26T20:37:31.341 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-04-26T20:37:31.350 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-04-26T20:37:31.407 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-04-26T20:37:31.416 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-04-26T20:37:31.425 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-04-26T20:37:31.433 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 97, 2024-04-26T20:37:31.497 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-04-26T20:37:31.507 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 2, 2024-04-26T20:37:31.515 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 5, 2024-04-26T20:37:31.523 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 7, 2024-04-26T20:37:31.537 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 2, 2024-04-26T20:37:31.546 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-04-26T20:37:31.554 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-04-26T20:37:31.563 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-04-26T20:37:31.572 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-04-26T20:37:31.581 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:31.639 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-04-26T20:37:31.714 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-04-26T20:37:31.723 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-04-26T20:37:31.770 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-04-26T20:37:31.778 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-04-26T20:37:31.785 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:31.792 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-04-26T20:37:31.853 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-04-26T20:37:31.861 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:31.868 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-04-26T20:37:31.875 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-04-26T20:37:31.882 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T20:37:31.889 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-04-26T20:37:31.896 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-04-26T20:37:31.903 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-04-26T20:37:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-04-26T20:37:31.967 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-04-26T20:37:31.974 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:32.047 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:37:32.055 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-26T20:37:32.111 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:32.118 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 5770, 2024-04-26T20:37:32.125 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 14386705, 2024-04-26T20:37:32.190 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:32.197 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 5770, 2024-04-26T20:37:32.204 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:32.211 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:32.219 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5770, 2024-04-26T20:37:32.226 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 14386705, 2024-04-26T20:37:32.233 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:32.240 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:32.248 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:32.274 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:32.281 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:32.353 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:32.416 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-04-26T20:37:32.435 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:32.481 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-26T20:37:32.489 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:32.496 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:32.552 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:32.560 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:32.566 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:32.574 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 5273, 2024-04-26T20:37:32.580 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 13009258, 2024-04-26T20:37:32.587 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5273, 2024-04-26T20:37:32.594 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 13009258, 2024-04-26T20:37:32.601 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:32.608 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:32.662 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:32.669 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:32.732 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:32.741 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:32.785 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-04-26T20:37:32.794 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:32.802 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-04-26T20:37:32.810 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:32.866 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:32.875 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:32.884 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:32.892 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:32.900 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 5273, 2024-04-26T20:37:32.909 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 5273, 2024-04-26T20:37:32.918 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5273, 2024-04-26T20:37:32.926 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 5273, 2024-04-26T20:37:32.935 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:32.978 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:32.987 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:33.060 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:33.069 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:33.135 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:33.144 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-04-26T20:37:33.153 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:33.161 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-26T20:37:33.225 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:33.234 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:33.243 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:33.251 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:33.260 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:33.268 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:33.276 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:33.285 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-26T20:37:33.294 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-26T20:37:33.302 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:33.345 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:33.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:33.418 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:33.472 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:33.482 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:33.491 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x561557e186e0": { 2024-04-26T20:37:33.500 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:33.549 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-26T20:37:33.557 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:33.566 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-26T20:37:33.575 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-26T20:37:33.583 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:33.592 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-26T20:37:33.600 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:33.608 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:33.617 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-26T20:37:33.687 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-26T20:37:33.695 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:33.798 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:33.808 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:33.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:33.876 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:33.885 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:33.893 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-26T20:37:33.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:21 smithi131 ceph-mon[27919]: pgmap v1230: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:33.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:23 smithi131 ceph-mon[27919]: pgmap v1231: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:33.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:25 smithi131 ceph-mon[27919]: pgmap v1232: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:33.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:25 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:37:33.986 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:21 smithi148 ceph-mon[32281]: pgmap v1230: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:33.995 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:23 smithi148 ceph-mon[32281]: pgmap v1231: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:34.003 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:25 smithi148 ceph-mon[32281]: pgmap v1232: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:34.011 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:25 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:37:34.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:21 smithi157 ceph-mon[32703]: pgmap v1230: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:34.077 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-26T20:37:34.086 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-26T20:37:34.143 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T20:37:34.152 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T20:37:34.212 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:23 smithi157 ceph-mon[32703]: pgmap v1231: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:34.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:25 smithi157 ceph-mon[32703]: pgmap v1232: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:34.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:25 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:37:34.234 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:26 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.308 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:26 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.315 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:26 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.322 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:26 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.329 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:27 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:37:34.336 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:27 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:37:34.342 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:27 smithi157 ceph-mon[32703]: pgmap v1233: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:34.350 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:29 smithi157 ceph-mon[32703]: pgmap v1234: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:34.356 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:34.363 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:34.369 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.560 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:37:34.567 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:37:34.574 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:37:34.582 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:37:34.588 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:30 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:34.595 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:31 smithi157 ceph-mon[32703]: pgmap v1235: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:34.603 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:33 smithi157 ceph-mon[32703]: pgmap v1236: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:34.665 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:26 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:26 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:26 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.690 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:26 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:27 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:37:34.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:27 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:37:34.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:27 smithi131 ceph-mon[27919]: pgmap v1233: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:34.737 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:29 smithi131 ceph-mon[27919]: pgmap v1234: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:34.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:34.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:34.851 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.869 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.878 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:34.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:37:34.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:37:34.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:37:34.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:37:34.989 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:30 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:34.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:31 smithi131 ceph-mon[27919]: pgmap v1235: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:35.007 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:33 smithi131 ceph-mon[27919]: pgmap v1236: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:35.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:26 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.026 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:26 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.034 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:26 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.087 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:26 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.150 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:27 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-26T20:37:35.157 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:27 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi148 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:37:35.224 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:27 smithi148 ceph-mon[32281]: pgmap v1233: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:35.234 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:29 smithi148 ceph-mon[32281]: pgmap v1234: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:35.243 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:35.252 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:35.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:37:35.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-26T20:37:35.384 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T20:37:35.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:37:35.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:37:35.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:30 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' 2024-04-26T20:37:35.418 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:31 smithi148 ceph-mon[32281]: pgmap v1235: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:35.476 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:33 smithi148 ceph-mon[32281]: pgmap v1236: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:35.531 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T20:37:36.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:35 smithi157 ceph-mon[32703]: pgmap v1237: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:36.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:35 smithi131 ceph-mon[27919]: pgmap v1237: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:36.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:35 smithi148 ceph-mon[32281]: pgmap v1237: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:36.609 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-26T20:37:36.616 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T20:37:36.715 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4, 2024-04-26T20:37:36.725 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 4, 2024-04-26T20:37:36.734 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2944, 2024-04-26T20:37:36.742 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 448, 2024-04-26T20:37:36.751 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-04-26T20:37:36.761 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 0, 2024-04-26T20:37:36.770 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.001487519, 2024-04-26T20:37:36.779 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.000033326, 2024-04-26T20:37:36.788 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.000517536, 2024-04-26T20:37:36.797 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:37:36.847 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:36.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2024-04-26T20:37:36.933 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000664713, 2024-04-26T20:37:37.013 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000221571 2024-04-26T20:37:37.020 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:37.028 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:37.036 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:37.110 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:37.118 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:37.127 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:37.135 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T20:37:37.143 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T20:37:37.151 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:37.159 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T20:37:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 494, 2024-04-26T20:37:37.179 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 492, 2024-04-26T20:37:37.187 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 421056, 2024-04-26T20:37:37.237 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 305248, 2024-04-26T20:37:37.301 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 2, 2024-04-26T20:37:37.310 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-04-26T20:37:37.352 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.078048868, 2024-04-26T20:37:37.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.033140146, 2024-04-26T20:37:37.368 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.022828049, 2024-04-26T20:37:37.379 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.013570874, 2024-04-26T20:37:37.457 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:37.466 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 491, 2024-04-26T20:37:37.474 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.025689179, 2024-04-26T20:37:37.482 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000052320 2024-04-26T20:37:37.491 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:37.499 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:37.508 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:37.518 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:37.527 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:37.583 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:37.591 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 421056, 2024-04-26T20:37:37.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 305248 2024-04-26T20:37:37.660 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:37.692 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T20:37:37.701 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4, 2024-04-26T20:37:37.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 380, 2024-04-26T20:37:37.715 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 640, 2024-04-26T20:37:37.795 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 381600, 2024-04-26T20:37:37.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-04-26T20:37:37.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-04-26T20:37:37.818 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.025085941, 2024-04-26T20:37:37.825 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.018643834, 2024-04-26T20:37:37.832 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.001786977, 2024-04-26T20:37:37.868 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T20:37:37.876 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T20:37:37.883 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 378, 2024-04-26T20:37:37.914 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.028618754, 2024-04-26T20:37:37.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000075710 2024-04-26T20:37:37.985 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:37.993 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-26T20:37:38.043 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:38.052 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:38.061 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:38.070 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:38.140 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 640, 2024-04-26T20:37:38.148 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 381600 2024-04-26T20:37:38.156 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:38.165 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-26T20:37:38.175 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 0, 2024-04-26T20:37:38.185 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-26T20:37:38.194 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:38.203 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-26T20:37:38.212 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-26T20:37:38.268 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-26T20:37:38.277 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T20:37:38.338 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-26T20:37:38.347 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T20:37:38.381 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-26T20:37:38.391 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T20:37:38.400 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T20:37:38.412 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T20:37:38.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T20:37:38.506 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T20:37:38.515 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-26T20:37:38.523 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T20:37:38.533 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T20:37:38.541 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-26T20:37:38.550 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-26T20:37:38.558 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-26T20:37:38.566 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-26T20:37:38.574 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T20:37:38.635 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T20:37:38.714 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T20:37:38.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T20:37:38.766 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T20:37:38.775 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-26T20:37:38.783 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-26T20:37:38.792 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-26T20:37:38.865 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T20:37:38.873 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T20:37:38.881 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-26T20:37:38.891 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-26T20:37:38.901 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-26T20:37:38.909 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-26T20:37:38.917 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T20:37:38.925 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-26T20:37:38.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T20:37:38.941 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-26T20:37:38.999 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 147898, 2024-04-26T20:37:39.050 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 27, 2024-04-26T20:37:39.058 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-04-26T20:37:39.066 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-04-26T20:37:39.124 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-26T20:37:39.133 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-26T20:37:39.141 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-26T20:37:39.215 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-26T20:37:39.223 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-26T20:37:39.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-26T20:37:39.241 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 0, 2024-04-26T20:37:39.249 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 0, 2024-04-26T20:37:39.258 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T20:37:39.267 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-26T20:37:39.276 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-26T20:37:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-26T20:37:39.344 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 928, 2024-04-26T20:37:39.353 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2, 2024-04-26T20:37:39.411 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-26T20:37:39.421 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-26T20:37:39.515 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-26T20:37:39.525 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-26T20:37:39.534 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-26T20:37:39.543 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T20:37:39.623 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-26T20:37:39.632 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-26T20:37:39.641 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-26T20:37:39.651 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 498, 2024-04-26T20:37:39.659 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 351225, 2024-04-26T20:37:39.666 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-26T20:37:39.674 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 498, 2024-04-26T20:37:39.682 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-26T20:37:39.690 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-26T20:37:39.742 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 498, 2024-04-26T20:37:39.749 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 351225, 2024-04-26T20:37:39.809 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-26T20:37:39.818 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-26T20:37:39.849 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-26T20:37:39.858 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-26T20:37:39.866 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:39.874 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-26T20:37:39.948 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-26T20:37:39.960 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:37 smithi148 ceph-mon[32281]: pgmap v1238: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:39.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:37 smithi131 ceph-mon[27919]: pgmap v1238: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:39.982 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-26T20:37:39.991 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-26T20:37:40.001 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T20:37:40.018 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T20:37:40.027 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:37:40.037 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:37:40.106 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:37 smithi157 ceph-mon[32703]: pgmap v1238: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:40.173 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-26T20:37:40.215 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:37:40.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:39 smithi157 ceph-mon[32703]: pgmap v1239: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:40.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:39 smithi131 ceph-mon[27919]: pgmap v1239: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:40.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:37:40.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:39 smithi148 ceph-mon[32281]: pgmap v1239: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:40.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 -- 172.21.15.131:0/3788832541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc10a940 msgr2=0x7f34bc1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:40.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 --2- 172.21.15.131:0/3788832541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc10a940 0x7f34bc1111d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009be0 tx=0x7f34b402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:37:40.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 -- 172.21.15.131:0/3788832541 shutdown_connections 2024-04-26T20:37:40.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 --2- 172.21.15.131:0/3788832541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc10a940 0x7f34bc1111d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:40.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 --2- 172.21.15.131:0/3788832541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc109f80 0x7f34bc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:40.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 --2- 172.21.15.131:0/3788832541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34bc108d90 0x7f34bc1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:40.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 -- 172.21.15.131:0/3788832541 >> 172.21.15.131:0/3788832541 conn(0x7f34bc077d30 msgr2=0x7f34bc07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:40.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 -- 172.21.15.131:0/3788832541 shutdown_connections 2024-04-26T20:37:40.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.540+0000 7f34bbe00640 1 -- 172.21.15.131:0/3788832541 wait complete. 2024-04-26T20:37:40.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 Processor -- start 2024-04-26T20:37:40.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 -- start start 2024-04-26T20:37:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc108d90 0x7f34bc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:40.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc109f80 0x7f34bc19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34bc10a940 0x7f34bc1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:40.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1148a0 con 0x7f34bc108d90 2024-04-26T20:37:40.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc114720 con 0x7f34bc109f80 2024-04-26T20:37:40.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.541+0000 7f34bbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc114a20 con 0x7f34bc10a940 2024-04-26T20:37:40.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc108d90 0x7f34bc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:40.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc108d90 0x7f34bc19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41384/0 (socket says 172.21.15.131:41384) 2024-04-26T20:37:40.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 -- 172.21.15.131:0/4000569096 learned_addr learned my addr 172.21.15.131:0/4000569096 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:40.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34b2000640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc109f80 0x7f34bc19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:40.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34bb400640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34bc10a940 0x7f34bc1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 -- 172.21.15.131:0/4000569096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34bc10a940 msgr2=0x7f34bc1a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:40.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34bc10a940 0x7f34bc1a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 -- 172.21.15.131:0/4000569096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc109f80 msgr2=0x7f34bc19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc109f80 0x7f34bc19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 -- 172.21.15.131:0/4000569096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc1a6b60 con 0x7f34bc108d90 2024-04-26T20:37:41.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.542+0000 7f34baa00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc108d90 0x7f34bc19ef40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e9a0 tx=0x7f34ac00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:41.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.543+0000 7f34b3e00640 1 -- 172.21.15.131:0/4000569096 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00cd80 con 0x7f34bc108d90 2024-04-26T20:37:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.543+0000 7f34b3e00640 1 -- 172.21.15.131:0/4000569096 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ac0108e0 con 0x7f34bc108d90 2024-04-26T20:37:41.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.543+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc1a6e50 con 0x7f34bc108d90 2024-04-26T20:37:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.543+0000 7f34b3e00640 1 -- 172.21.15.131:0/4000569096 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac018960 con 0x7f34bc108d90 2024-04-26T20:37:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.543+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a72b0 con 0x7f34bc108d90 2024-04-26T20:37:41.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.544+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc069ac0 con 0x7f34bc108d90 2024-04-26T20:37:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.548+0000 7f34b3e00640 1 -- 172.21.15.131:0/4000569096 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f34ac010a90 con 0x7f34bc108d90 2024-04-26T20:37:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.549+0000 7f34b3e00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34a00a3c40 0x7f34a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.549+0000 7f34b3e00640 1 -- 172.21.15.131:0/4000569096 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f34ac0d1440 con 0x7f34bc108d90 2024-04-26T20:37:41.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.550+0000 7f34b2000640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34a00a3c40 0x7f34a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:41.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.551+0000 7f34b2000640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34a00a3c40 0x7f34a00a6100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f34a40095f0 tx=0x7f34a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:41.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.552+0000 7f34b3e00640 1 -- 172.21.15.131:0/4000569096 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f34ac026070 con 0x7f34bc108d90 2024-04-26T20:37:41.284 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:40 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/4000569096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:37:41.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.757+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34bc02dbd0 con 0x7f34bc108d90 2024-04-26T20:37:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.760+0000 7f34b3e00640 1 -- 172.21.15.131:0/4000569096 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7f34ac09d870 con 0x7f34bc108d90 2024-04-26T20:37:41.308 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:37:41.364 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:37:41.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34a00a3c40 msgr2=0x7f34a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:41.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34a00a3c40 0x7f34a00a6100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f34a40095f0 tx=0x7f34a4005f90 comp rx=0 tx=0).stop 2024-04-26T20:37:41.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc108d90 msgr2=0x7f34bc19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:41.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc108d90 0x7f34bc19ef40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e9a0 tx=0x7f34ac00ee60 comp rx=0 tx=0).stop 2024-04-26T20:37:41.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 shutdown_connections 2024-04-26T20:37:41.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f34a00a3c40 0x7f34a00a6100 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:41.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34bc10a940 0x7f34bc1a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:41.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc109f80 0x7f34bc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:41.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 --2- 172.21.15.131:0/4000569096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34bc108d90 0x7f34bc19ef40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:41.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.762+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 >> 172.21.15.131:0/4000569096 conn(0x7f34bc077d30 msgr2=0x7f34bc10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:41.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.763+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 shutdown_connections 2024-04-26T20:37:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:40.763+0000 7f34bbe00640 1 -- 172.21.15.131:0/4000569096 wait complete. 2024-04-26T20:37:41.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:01:57.626290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:07:30.370934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:07:33.112161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:07:36.797488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:37:41.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:37:41.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:40 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/4000569096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:37:41.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:40 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/4000569096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:37:41.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1309252311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a01028c0 msgr2=0x7ff2a0102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:41.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1309252311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a01028c0 0x7ff2a0102ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff298009900 tx=0x7ff29802a3f0 comp rx=0 tx=0).stop 2024-04-26T20:37:41.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1309252311 shutdown_connections 2024-04-26T20:37:42.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1309252311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2a0104470 0x7ff2a010ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1309252311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a0103ab0 0x7ff2a0103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1309252311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a01028c0 0x7ff2a0102ce0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1309252311 >> 172.21.15.131:0/1309252311 conn(0x7ff2a00fde30 msgr2=0x7ff2a0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1309252311 shutdown_connections 2024-04-26T20:37:42.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.923+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1309252311 wait complete. 2024-04-26T20:37:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.924+0000 7ff2a6c00640 1 Processor -- start 2024-04-26T20:37:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.924+0000 7ff2a6c00640 1 -- start start 2024-04-26T20:37:42.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a6c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a01028c0 0x7ff2a019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:42.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a6c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2a0103ab0 0x7ff2a019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:42.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a6c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a0104470 0x7ff2a01a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:42.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a6c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010e3c0 con 0x7ff2a0103ab0 2024-04-26T20:37:42.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a6c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010e240 con 0x7ff2a01028c0 2024-04-26T20:37:42.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a6c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010e540 con 0x7ff2a0104470 2024-04-26T20:37:42.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a01028c0 0x7ff2a019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:42.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2a0103ab0 0x7ff2a019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:42.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a01028c0 0x7ff2a019ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:54376/0 (socket says 172.21.15.131:54376) 2024-04-26T20:37:42.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 -- 172.21.15.131:0/1094615870 learned_addr learned my addr 172.21.15.131:0/1094615870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:42.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a6200640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a0104470 0x7ff2a01a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:42.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 -- 172.21.15.131:0/1094615870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a0104470 msgr2=0x7ff2a01a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:42.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a0104470 0x7ff2a01a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:42.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 -- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2a0103ab0 msgr2=0x7ff2a019b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:42.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2a0103ab0 0x7ff2a019b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:42.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.925+0000 7ff2a5800640 1 -- 172.21.15.131:0/1094615870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a01a2930 con 0x7ff2a01028c0 2024-04-26T20:37:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.926+0000 7ff2a5800640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a01028c0 0x7ff2a019ad20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff298031500 tx=0x7ff298003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:42.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.926+0000 7ff29e000640 1 -- 172.21.15.131:0/1094615870 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29803f070 con 0x7ff2a01028c0 2024-04-26T20:37:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.926+0000 7ff29e000640 1 -- 172.21.15.131:0/1094615870 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff298031c50 con 0x7ff2a01028c0 2024-04-26T20:37:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.926+0000 7ff29e000640 1 -- 172.21.15.131:0/1094615870 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298035030 con 0x7ff2a01028c0 2024-04-26T20:37:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.926+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a01a2bc0 con 0x7ff2a01028c0 2024-04-26T20:37:42.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.926+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a01a30b0 con 0x7ff2a01028c0 2024-04-26T20:37:42.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.927+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a01a3280 con 0x7ff2a01028c0 2024-04-26T20:37:42.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.929+0000 7ff29e000640 1 -- 172.21.15.131:0/1094615870 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff298042070 con 0x7ff2a01028c0 2024-04-26T20:37:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.931+0000 7ff29e000640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2840a3c40 0x7ff2840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.931+0000 7ff29e000640 1 -- 172.21.15.131:0/1094615870 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7ff2980f1890 con 0x7ff2a01028c0 2024-04-26T20:37:42.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.931+0000 7ff2a4e00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2840a3c40 0x7ff2840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:42.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.932+0000 7ff2a4e00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2840a3c40 0x7ff2840a6100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005fd0 tx=0x7ff28c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:42.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:41.933+0000 7ff29e000640 1 -- 172.21.15.131:0/1094615870 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2980bd670 con 0x7ff2a01028c0 2024-04-26T20:37:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.135+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff2a01a3450 con 0x7ff2a01028c0 2024-04-26T20:37:42.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.136+0000 7ff29e000640 1 -- 172.21.15.131:0/1094615870 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7ff2980c1970 con 0x7ff2a01028c0 2024-04-26T20:37:43.007 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-26T20:37:43.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:41 smithi148 ceph-mon[32281]: pgmap v1240: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:43.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:41 smithi157 ceph-mon[32703]: pgmap v1240: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:43.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2840a3c40 msgr2=0x7ff2840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:43.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2840a3c40 0x7ff2840a6100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005fd0 tx=0x7ff28c009500 comp rx=0 tx=0).stop 2024-04-26T20:37:43.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a01028c0 msgr2=0x7ff2a019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:43.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a01028c0 0x7ff2a019ad20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff298031500 tx=0x7ff298003c10 comp rx=0 tx=0).stop 2024-04-26T20:37:43.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 shutdown_connections 2024-04-26T20:37:43.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7ff2840a3c40 0x7ff2840a6100 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:43.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a0104470 0x7ff2a01a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:43.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2a0103ab0 0x7ff2a019b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 --2- 172.21.15.131:0/1094615870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a01028c0 0x7ff2a019ad20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 >> 172.21.15.131:0/1094615870 conn(0x7ff2a00fde30 msgr2=0x7ff2a0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:43.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 shutdown_connections 2024-04-26T20:37:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:42.139+0000 7ff2a6c00640 1 -- 172.21.15.131:0/1094615870 wait complete. 2024-04-26T20:37:43.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:41 smithi131 ceph-mon[27919]: pgmap v1240: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:43.285 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:37:43.356 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:42 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1094615870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:37:43.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:42 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1094615870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:37:43.375 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:42 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1094615870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T20:37:43.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 -- 172.21.15.131:0/1501260586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc434076970 msgr2=0x7fc434074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:43.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 --2- 172.21.15.131:0/1501260586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc434076970 0x7fc434074dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc424009930 tx=0x7fc42402a420 comp rx=0 tx=0).stop 2024-04-26T20:37:43.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 -- 172.21.15.131:0/1501260586 shutdown_connections 2024-04-26T20:37:43.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 --2- 172.21.15.131:0/1501260586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc43410ee10 0x7fc434111200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:43.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 --2- 172.21.15.131:0/1501260586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc434075390 0x7fc434075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:43.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 --2- 172.21.15.131:0/1501260586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc434076970 0x7fc434074dc0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:43.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 -- 172.21.15.131:0/1501260586 >> 172.21.15.131:0/1501260586 conn(0x7fc4340fddd0 msgr2=0x7fc4341001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:43.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 -- 172.21.15.131:0/1501260586 shutdown_connections 2024-04-26T20:37:43.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.579+0000 7fc433200640 1 -- 172.21.15.131:0/1501260586 wait complete. 2024-04-26T20:37:43.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 Processor -- start 2024-04-26T20:37:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 -- start start 2024-04-26T20:37:43.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc434075390 0x7fc43419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:43.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc434076970 0x7fc43419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:43.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43410ee10 0x7fc4341a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:43.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc434114960 con 0x7fc43410ee10 2024-04-26T20:37:43.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341147e0 con 0x7fc434075390 2024-04-26T20:37:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.580+0000 7fc433200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc434114ae0 con 0x7fc434076970 2024-04-26T20:37:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43410ee10 0x7fc4341a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc431400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc434076970 0x7fc43419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc431e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc434075390 0x7fc43419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:43.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc431400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc434076970 0x7fc43419f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59418/0 (socket says 172.21.15.131:59418) 2024-04-26T20:37:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43410ee10 0x7fc4341a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41416/0 (socket says 172.21.15.131:41416) 2024-04-26T20:37:44.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc431400640 1 -- 172.21.15.131:0/3234597962 learned_addr learned my addr 172.21.15.131:0/3234597962 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:44.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 -- 172.21.15.131:0/3234597962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc434076970 msgr2=0x7fc43419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:44.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc434076970 0x7fc43419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:44.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 -- 172.21.15.131:0/3234597962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc434075390 msgr2=0x7fc43419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:44.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc434075390 0x7fc43419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:44.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 -- 172.21.15.131:0/3234597962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4341a6cb0 con 0x7fc43410ee10 2024-04-26T20:37:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.581+0000 7fc432800640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43410ee10 0x7fc4341a6640 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009fd0 tx=0x7fc42c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.582+0000 7fc42aa00640 1 -- 172.21.15.131:0/3234597962 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00ce20 con 0x7fc43410ee10 2024-04-26T20:37:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.582+0000 7fc42aa00640 1 -- 172.21.15.131:0/3234597962 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc42c005020 con 0x7fc43410ee10 2024-04-26T20:37:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.582+0000 7fc42aa00640 1 -- 172.21.15.131:0/3234597962 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c010690 con 0x7fc43410ee10 2024-04-26T20:37:44.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.582+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4341a6fa0 con 0x7fc43410ee10 2024-04-26T20:37:44.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.582+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4341a7460 con 0x7fc43410ee10 2024-04-26T20:37:44.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.583+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc434104fe0 con 0x7fc43410ee10 2024-04-26T20:37:44.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.585+0000 7fc42aa00640 1 -- 172.21.15.131:0/3234597962 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc42c0054d0 con 0x7fc43410ee10 2024-04-26T20:37:44.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.587+0000 7fc42aa00640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4180a3c40 0x7fc4180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:44.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.588+0000 7fc42aa00640 1 -- 172.21.15.131:0/3234597962 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fc42c0d1070 con 0x7fc43410ee10 2024-04-26T20:37:44.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.588+0000 7fc431e00640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4180a3c40 0x7fc4180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:44.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.588+0000 7fc431e00640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4180a3c40 0x7fc4180a6100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc424006010 tx=0x7fc4240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:44.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.592+0000 7fc42aa00640 1 -- 172.21.15.131:0/3234597962 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc42c09d4a0 con 0x7fc43410ee10 2024-04-26T20:37:44.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.792+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc43402dd20 con 0x7fc43410ee10 2024-04-26T20:37:44.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.793+0000 7fc42aa00640 1 -- 172.21.15.131:0/3234597962 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fc42c09cda0 con 0x7fc43410ee10 2024-04-26T20:37:44.463 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:37:44.492 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:43 smithi148 ceph-mon[32281]: pgmap v1241: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:44.499 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:43 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3234597962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:37:44.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:43 smithi157 ceph-mon[32703]: pgmap v1241: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:44.513 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:43 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3234597962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:37:44.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4180a3c40 msgr2=0x7fc4180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:44.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4180a3c40 0x7fc4180a6100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc424006010 tx=0x7fc4240314a0 comp rx=0 tx=0).stop 2024-04-26T20:37:44.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43410ee10 msgr2=0x7fc4341a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:44.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43410ee10 0x7fc4341a6640 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009fd0 tx=0x7fc42c00ef10 comp rx=0 tx=0).stop 2024-04-26T20:37:44.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 shutdown_connections 2024-04-26T20:37:44.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43410ee10 0x7fc4341a6640 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:44.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc4180a3c40 0x7fc4180a6100 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:44.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc434076970 0x7fc43419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:44.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 --2- 172.21.15.131:0/3234597962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc434075390 0x7fc43419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:44.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.796+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 >> 172.21.15.131:0/3234597962 conn(0x7fc4340fddd0 msgr2=0x7fc4341021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:44.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.797+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 shutdown_connections 2024-04-26T20:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:43.797+0000 7fc433200640 1 -- 172.21.15.131:0/3234597962 wait complete. 2024-04-26T20:37:44.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:37:44.817 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:43 smithi131 ceph-mon[27919]: pgmap v1241: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:44.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:43 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3234597962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T20:37:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.046+0000 7f068ec00640 1 -- 172.21.15.131:0/206846262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 msgr2=0x7f0688074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:45.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.046+0000 7f068ec00640 1 --2- 172.21.15.131:0/206846262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 0x7f0688074d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f067c009860 tx=0x7f067c02a3d0 comp rx=0 tx=0).stop 2024-04-26T20:37:45.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.047+0000 7f068ec00640 1 -- 172.21.15.131:0/206846262 shutdown_connections 2024-04-26T20:37:45.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.047+0000 7f068ec00640 1 --2- 172.21.15.131:0/206846262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f068810a5d0 0x7f068810ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.047+0000 7f068ec00640 1 --2- 172.21.15.131:0/206846262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0688075330 0x7f0688075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:45.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.047+0000 7f068ec00640 1 --2- 172.21.15.131:0/206846262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 0x7f0688074d60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:45.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.047+0000 7f068ec00640 1 -- 172.21.15.131:0/206846262 >> 172.21.15.131:0/206846262 conn(0x7f06880fdcb0 msgr2=0x7f06881000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:45.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.047+0000 7f068ec00640 1 -- 172.21.15.131:0/206846262 shutdown_connections 2024-04-26T20:37:45.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.047+0000 7f068ec00640 1 -- 172.21.15.131:0/206846262 wait complete. 2024-04-26T20:37:45.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 Processor -- start 2024-04-26T20:37:45.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 -- start start 2024-04-26T20:37:45.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0688075330 0x7f068819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:45.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 0x7f068819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:45.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f068810a5d0 0x7f06881a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0688110490 con 0x7f068810a5d0 2024-04-26T20:37:45.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0688110310 con 0x7f0688076910 2024-04-26T20:37:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0688110610 con 0x7f0688075330 2024-04-26T20:37:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f068810a5d0 0x7f06881a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f068810a5d0 0x7f06881a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55246/0 (socket says 172.21.15.131:55246) 2024-04-26T20:37:45.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 0x7f068819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.048+0000 7f068e200640 1 -- 172.21.15.131:0/1674755118 learned_addr learned my addr 172.21.15.131:0/1674755118 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:45.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068d800640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0688075330 0x7f068819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:45.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068e200640 1 -- 172.21.15.131:0/1674755118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0688075330 msgr2=0x7f068819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:45.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068e200640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0688075330 0x7f068819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:45.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068e200640 1 -- 172.21.15.131:0/1674755118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 msgr2=0x7f068819b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068e200640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 0x7f068819b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068e200640 1 -- 172.21.15.131:0/1674755118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06881a2770 con 0x7f068810a5d0 2024-04-26T20:37:45.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068e200640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f068810a5d0 0x7f06881a2100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f068000eb10 tx=0x7f068000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:45.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f0686000640 1 -- 172.21.15.131:0/1674755118 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680004b20 con 0x7f068810a5d0 2024-04-26T20:37:45.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f0686000640 1 -- 172.21.15.131:0/1674755118 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0680004cc0 con 0x7f068810a5d0 2024-04-26T20:37:45.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06881a2a60 con 0x7f068810a5d0 2024-04-26T20:37:45.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.049+0000 7f0686000640 1 -- 172.21.15.131:0/1674755118 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680010670 con 0x7f068810a5d0 2024-04-26T20:37:45.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.050+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06881a2f60 con 0x7f068810a5d0 2024-04-26T20:37:45.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.051+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0688109390 con 0x7f068810a5d0 2024-04-26T20:37:45.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.053+0000 7f0686000640 1 -- 172.21.15.131:0/1674755118 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0680005020 con 0x7f068810a5d0 2024-04-26T20:37:45.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.054+0000 7f0686000640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f06700a3c40 0x7f06700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:45.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.054+0000 7f0686000640 1 -- 172.21.15.131:0/1674755118 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f06800d0f90 con 0x7f068810a5d0 2024-04-26T20:37:45.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.054+0000 7f068d800640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f06700a3c40 0x7f06700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.055+0000 7f068d800640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f06700a3c40 0x7f06700a6100 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f067c02a9c0 tx=0x7f067c030980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.059+0000 7f0686000640 1 -- 172.21.15.131:0/1674755118 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f068009d3c0 con 0x7f068810a5d0 2024-04-26T20:37:45.822 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:44 smithi157 ceph-mon[32703]: pgmap v1242: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:45.831 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:44 smithi148 ceph-mon[32281]: pgmap v1242: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:45.840 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:44 smithi131 ceph-mon[27919]: pgmap v1242: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T20:37:45.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.257+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0688063530 con 0x7f068810a5d0 2024-04-26T20:37:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.259+0000 7f0686000640 1 -- 172.21.15.131:0/1674755118 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f068009ccc0 con 0x7f068810a5d0 2024-04-26T20:37:45.866 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-26T20:37:45.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f06700a3c40 msgr2=0x7f06700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:45.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f06700a3c40 0x7f06700a6100 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f067c02a9c0 tx=0x7f067c030980 comp rx=0 tx=0).stop 2024-04-26T20:37:46.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f068810a5d0 msgr2=0x7f06881a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:46.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f068810a5d0 0x7f06881a2100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f068000eb10 tx=0x7f068000c2d0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 shutdown_connections 2024-04-26T20:37:46.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f068810a5d0 0x7f06881a2100 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0688076910 0x7f068819b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f06700a3c40 0x7f06700a6100 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 --2- 172.21.15.131:0/1674755118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0688075330 0x7f068819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.261+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 >> 172.21.15.131:0/1674755118 conn(0x7f06880fdcb0 msgr2=0x7f0688106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:46.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.262+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 shutdown_connections 2024-04-26T20:37:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:45.262+0000 7f068ec00640 1 -- 172.21.15.131:0/1674755118 wait complete. 2024-04-26T20:37:46.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:37:46.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:45 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1674755118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:37:46.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:45 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1674755118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:37:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.412+0000 7fd422800640 1 -- 172.21.15.131:0/2333441341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c104bb0 msgr2=0x7fd41c104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.412+0000 7fd422800640 1 --2- 172.21.15.131:0/2333441341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c104bb0 0x7fd41c104fd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd408009900 tx=0x7fd40802a410 comp rx=0 tx=0).stop 2024-04-26T20:37:46.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.412+0000 7fd422800640 1 -- 172.21.15.131:0/2333441341 shutdown_connections 2024-04-26T20:37:46.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.412+0000 7fd422800640 1 --2- 172.21.15.131:0/2333441341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c106760 0x7fd41c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.412+0000 7fd422800640 1 --2- 172.21.15.131:0/2333441341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41c105da0 0x7fd41c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.412+0000 7fd422800640 1 --2- 172.21.15.131:0/2333441341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c104bb0 0x7fd41c104fd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.412+0000 7fd422800640 1 -- 172.21.15.131:0/2333441341 >> 172.21.15.131:0/2333441341 conn(0x7fd41c100120 msgr2=0x7fd41c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.413+0000 7fd422800640 1 -- 172.21.15.131:0/2333441341 shutdown_connections 2024-04-26T20:37:46.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.413+0000 7fd422800640 1 -- 172.21.15.131:0/2333441341 wait complete. 2024-04-26T20:37:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 Processor -- start 2024-04-26T20:37:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 -- start start 2024-04-26T20:37:46.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41c104bb0 0x7fd41c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:46.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c105da0 0x7fd41c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c106760 0x7fd41c1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:46.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1106d0 con 0x7fd41c105da0 2024-04-26T20:37:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c110550 con 0x7fd41c106760 2024-04-26T20:37:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd422800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c110850 con 0x7fd41c104bb0 2024-04-26T20:37:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd41aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c105da0 0x7fd41c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd41b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41c104bb0 0x7fd41c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:46.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd41aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c105da0 0x7fd41c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55256/0 (socket says 172.21.15.131:55256) 2024-04-26T20:37:46.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.414+0000 7fd41aa00640 1 -- 172.21.15.131:0/597311 learned_addr learned my addr 172.21.15.131:0/597311 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:46.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd41be00640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c106760 0x7fd41c1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:46.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd41aa00640 1 -- 172.21.15.131:0/597311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41c104bb0 msgr2=0x7fd41c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:46.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd41aa00640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41c104bb0 0x7fd41c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:46.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd41aa00640 1 -- 172.21.15.131:0/597311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c106760 msgr2=0x7fd41c1a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:46.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd41aa00640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c106760 0x7fd41c1a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd41aa00640 1 -- 172.21.15.131:0/597311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c1a2a20 con 0x7fd41c105da0 2024-04-26T20:37:47.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd41aa00640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c105da0 0x7fd41c19b340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e9f0 tx=0x7fd40c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:47.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd413e00640 1 -- 172.21.15.131:0/597311 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00cd80 con 0x7fd41c105da0 2024-04-26T20:37:47.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd413e00640 1 -- 172.21.15.131:0/597311 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd40c0054d0 con 0x7fd41c105da0 2024-04-26T20:37:47.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.415+0000 7fd413e00640 1 -- 172.21.15.131:0/597311 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c010690 con 0x7fd41c105da0 2024-04-26T20:37:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.416+0000 7fd422800640 1 -- 172.21.15.131:0/597311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c1a2d10 con 0x7fd41c105da0 2024-04-26T20:37:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.416+0000 7fd422800640 1 -- 172.21.15.131:0/597311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c1a3200 con 0x7fd41c105da0 2024-04-26T20:37:47.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.417+0000 7fd422800640 1 -- 172.21.15.131:0/597311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c10bfa0 con 0x7fd41c105da0 2024-04-26T20:37:47.109 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:45 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1674755118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T20:37:47.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.419+0000 7fd413e00640 1 -- 172.21.15.131:0/597311 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd40c004e90 con 0x7fd41c105da0 2024-04-26T20:37:47.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.421+0000 7fd413e00640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4000a3cc0 0x7fd4000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.421+0000 7fd413e00640 1 -- 172.21.15.131:0/597311 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fd40c0d0d40 con 0x7fd41c105da0 2024-04-26T20:37:47.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.421+0000 7fd41b400640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4000a3cc0 0x7fd4000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:47.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.422+0000 7fd41b400640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4000a3cc0 0x7fd4000a6180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd408009230 tx=0x7fd4080314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.425+0000 7fd413e00640 1 -- 172.21.15.131:0/597311 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd40c09d170 con 0x7fd41c105da0 2024-04-26T20:37:47.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.629+0000 7fd422800640 1 -- 172.21.15.131:0/597311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd41c02dd20 con 0x7fd41c105da0 2024-04-26T20:37:47.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.630+0000 7fd413e00640 1 -- 172.21.15.131:0/597311 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7fd40c09ca70 con 0x7fd41c105da0 2024-04-26T20:37:47.429 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-26T20:37:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.632+0000 7fd422800640 1 -- 172.21.15.131:0/597311 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4000a3cc0 msgr2=0x7fd4000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4000a3cc0 0x7fd4000a6180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd408009230 tx=0x7fd4080314a0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 -- 172.21.15.131:0/597311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c105da0 msgr2=0x7fd41c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:47.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c105da0 0x7fd41c19b340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e9f0 tx=0x7fd40c00edf0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 -- 172.21.15.131:0/597311 shutdown_connections 2024-04-26T20:37:47.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c106760 0x7fd41c1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41c105da0 0x7fd41c19b340 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fd4000a3cc0 0x7fd4000a6180 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 --2- 172.21.15.131:0/597311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41c104bb0 0x7fd41c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 -- 172.21.15.131:0/597311 >> 172.21.15.131:0/597311 conn(0x7fd41c100120 msgr2=0x7fd41c109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:47.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 -- 172.21.15.131:0/597311 shutdown_connections 2024-04-26T20:37:47.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:46.633+0000 7fd422800640 1 -- 172.21.15.131:0/597311 wait complete. 2024-04-26T20:37:47.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:37:47.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:47 smithi131 ceph-mon[27919]: pgmap v1243: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:47.676 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:47 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/597311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:37:47.686 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:47 smithi148 ceph-mon[32281]: pgmap v1243: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:47.757 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:47 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/597311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:37:47.766 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:47 smithi157 ceph-mon[32703]: pgmap v1243: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T20:37:47.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:47 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/597311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T20:37:47.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.888+0000 7eff10c00640 1 -- 172.21.15.131:0/3564652626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c109f80 msgr2=0x7eff0c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:47.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.888+0000 7eff10c00640 1 --2- 172.21.15.131:0/3564652626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c109f80 0x7eff0c10a400 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efef4009900 tx=0x7efef402a410 comp rx=0 tx=0).stop 2024-04-26T20:37:47.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.888+0000 7eff10c00640 1 -- 172.21.15.131:0/3564652626 shutdown_connections 2024-04-26T20:37:47.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.888+0000 7eff10c00640 1 --2- 172.21.15.131:0/3564652626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c10a940 0x7eff0c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.888+0000 7eff10c00640 1 --2- 172.21.15.131:0/3564652626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c109f80 0x7eff0c10a400 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.888+0000 7eff10c00640 1 --2- 172.21.15.131:0/3564652626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c108d90 0x7eff0c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:47.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.888+0000 7eff10c00640 1 -- 172.21.15.131:0/3564652626 >> 172.21.15.131:0/3564652626 conn(0x7eff0c077d30 msgr2=0x7eff0c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:48.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.889+0000 7eff10c00640 1 -- 172.21.15.131:0/3564652626 shutdown_connections 2024-04-26T20:37:48.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.889+0000 7eff10c00640 1 -- 172.21.15.131:0/3564652626 wait complete. 2024-04-26T20:37:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.889+0000 7eff10c00640 1 Processor -- start 2024-04-26T20:37:48.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff10c00640 1 -- start start 2024-04-26T20:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff10c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c108d90 0x7eff0c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:48.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff10c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c109f80 0x7eff0c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:48.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff10c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c10a940 0x7eff0c1a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:48.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff10c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1148c0 con 0x7eff0c10a940 2024-04-26T20:37:48.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff10c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c114740 con 0x7eff0c109f80 2024-04-26T20:37:48.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff10c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c114a40 con 0x7eff0c108d90 2024-04-26T20:37:48.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff0b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c108d90 0x7eff0c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff0b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c108d90 0x7eff0c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54888/0 (socket says 172.21.15.131:54888) 2024-04-26T20:37:48.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff0be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c10a940 0x7eff0c1a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff0b400640 1 -- 172.21.15.131:0/2012747742 learned_addr learned my addr 172.21.15.131:0/2012747742 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:48.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.890+0000 7eff02a00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c109f80 0x7eff0c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:48.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff0b400640 1 -- 172.21.15.131:0/2012747742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c109f80 msgr2=0x7eff0c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff0b400640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c109f80 0x7eff0c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:48.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff0b400640 1 -- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c10a940 msgr2=0x7eff0c1a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:48.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff0b400640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c10a940 0x7eff0c1a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:48.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff0b400640 1 -- 172.21.15.131:0/2012747742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c1a6bb0 con 0x7eff0c108d90 2024-04-26T20:37:48.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff02a00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c109f80 0x7eff0c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:37:48.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff0b400640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c108d90 0x7eff0c19ef90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efefc009d20 tx=0x7efefc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:48.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff08c00640 1 -- 172.21.15.131:0/2012747742 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc010630 con 0x7eff0c108d90 2024-04-26T20:37:48.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff08c00640 1 -- 172.21.15.131:0/2012747742 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efefc010c80 con 0x7eff0c108d90 2024-04-26T20:37:48.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c1a6ea0 con 0x7eff0c108d90 2024-04-26T20:37:48.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff08c00640 1 -- 172.21.15.131:0/2012747742 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc018d50 con 0x7eff0c108d90 2024-04-26T20:37:48.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.891+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c1a7300 con 0x7eff0c108d90 2024-04-26T20:37:48.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.893+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c110100 con 0x7eff0c108d90 2024-04-26T20:37:48.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.894+0000 7eff08c00640 1 -- 172.21.15.131:0/2012747742 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efefc01f020 con 0x7eff0c108d90 2024-04-26T20:37:48.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.896+0000 7eff08c00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efef00a3c40 0x7efef00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:48.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.896+0000 7eff08c00640 1 -- 172.21.15.131:0/2012747742 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7efefc0d0eb0 con 0x7eff0c108d90 2024-04-26T20:37:48.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.896+0000 7eff02a00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efef00a3c40 0x7efef00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:48.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.897+0000 7eff02a00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efef00a3c40 0x7efef00a6100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7eff0c1a04a0 tx=0x7efef40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:48.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:47.900+0000 7eff08c00640 1 -- 172.21.15.131:0/2012747742 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efefc09d2e0 con 0x7eff0c108d90 2024-04-26T20:37:48.703 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:48 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2012747742' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:37:48.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:48 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2012747742' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:37:48.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.119+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff0c02dad0 con 0x7eff0c108d90 2024-04-26T20:37:48.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.120+0000 7eff08c00640 1 -- 172.21.15.131:0/2012747742 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+601 (secure 0 0 0) 0x7eff0c02dad0 con 0x7eff0c108d90 2024-04-26T20:37:48.838 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3514900702 2024-04-27T19:59:13.443197+0000 2024-04-26T20:37:48.847 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/3789992830 2024-04-27T19:59:13.443197+0000 2024-04-26T20:37:48.856 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/3789992830 2024-04-27T19:59:13.443197+0000 2024-04-26T20:37:48.912 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/2560715060 2024-04-27T19:58:38.942812+0000 2024-04-26T20:37:48.921 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/2579641682 2024-04-27T19:59:13.443197+0000 2024-04-26T20:37:48.930 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/2560715060 2024-04-27T19:58:38.942812+0000 2024-04-26T20:37:48.938 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3589087841 2024-04-27T19:58:38.942812+0000 2024-04-26T20:37:48.945 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3946098705 2024-04-27T19:59:13.443197+0000 2024-04-26T20:37:48.952 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1232073682 2024-04-27T19:58:38.942812+0000 2024-04-26T20:37:48.960 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/537078740 2024-04-27T19:58:38.942812+0000 2024-04-26T20:37:48.968 INFO:teuthology.orchestra.run.smithi131.stderr:listed 10 entries 2024-04-26T20:37:48.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efef00a3c40 msgr2=0x7efef00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:49.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efef00a3c40 0x7efef00a6100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7eff0c1a04a0 tx=0x7efef40314a0 comp rx=0 tx=0).stop 2024-04-26T20:37:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c108d90 msgr2=0x7eff0c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:49.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c108d90 0x7eff0c19ef90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efefc009d20 tx=0x7efefc00c6a0 comp rx=0 tx=0).stop 2024-04-26T20:37:49.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 shutdown_connections 2024-04-26T20:37:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff0c10a940 0x7eff0c1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c109f80 0x7eff0c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:49.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7efef00a3c40 0x7efef00a6100 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:49.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.123+0000 7eff10c00640 1 --2- 172.21.15.131:0/2012747742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff0c108d90 0x7eff0c19ef90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:49.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.124+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 >> 172.21.15.131:0/2012747742 conn(0x7eff0c077d30 msgr2=0x7eff0c10d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:49.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.124+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 shutdown_connections 2024-04-26T20:37:49.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:48.124+0000 7eff10c00640 1 -- 172.21.15.131:0/2012747742 wait complete. 2024-04-26T20:37:49.266 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:37:49.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:37:49.280 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T20:37:49.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:48 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2012747742' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T20:37:49.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:37:49.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:37:49.432 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:37:49.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:49 smithi157 ceph-mon[32703]: pgmap v1244: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:49.571 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:49 smithi148 ceph-mon[32281]: pgmap v1244: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:49.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:49 smithi131 ceph-mon[27919]: pgmap v1244: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T20:37:49.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:37:49.667 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-26T20:37:49.761 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:37:49.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:37:49.847 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:37:49.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:37:49.958 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-04-26T20:37:50.082 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T20:37:50.099 INFO:tasks.cephadm:Teardown begin 2024-04-26T20:37:50.107 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:37:50.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:37:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.313+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3616107764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0104990 msgr2=0x7f3fa0104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:50.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.313+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3616107764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0104990 0x7f3fa0104db0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009900 tx=0x7f3f9002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:37:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.313+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3616107764 shutdown_connections 2024-04-26T20:37:50.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.313+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3616107764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa0106540 0x7f3fa010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:50.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.313+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3616107764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa0105b80 0x7f3fa0106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:50.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.313+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3616107764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0104990 0x7f3fa0104db0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:50.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.313+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3616107764 >> 172.21.15.131:0/3616107764 conn(0x7f3fa00fff00 msgr2=0x7f3fa0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:50.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.314+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3616107764 shutdown_connections 2024-04-26T20:37:50.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.314+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3616107764 wait complete. 2024-04-26T20:37:50.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 Processor -- start 2024-04-26T20:37:50.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 -- start start 2024-04-26T20:37:50.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa0104990 0x7f3fa019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0105b80 0x7f3fa019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:50.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa0106540 0x7f3fa01a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:50.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa01104b0 con 0x7f3fa0105b80 2024-04-26T20:37:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa0110330 con 0x7f3fa0104990 2024-04-26T20:37:50.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa0110630 con 0x7f3fa0106540 2024-04-26T20:37:50.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0105b80 0x7f3fa019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:50.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0105b80 0x7f3fa019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55306/0 (socket says 172.21.15.131:55306) 2024-04-26T20:37:50.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9e000640 1 -- 172.21.15.131:0/3383466243 learned_addr learned my addr 172.21.15.131:0/3383466243 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9ea00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa0104990 0x7f3fa019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:50.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.315+0000 7f3f9f400640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa0106540 0x7f3fa01a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:50.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f9e000640 1 -- 172.21.15.131:0/3383466243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa0106540 msgr2=0x7f3fa01a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:50.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f9e000640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa0106540 0x7f3fa01a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:50.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f9e000640 1 -- 172.21.15.131:0/3383466243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa0104990 msgr2=0x7f3fa019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:50.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f9e000640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa0104990 0x7f3fa019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f9e000640 1 -- 172.21.15.131:0/3383466243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa01a2800 con 0x7f3fa0105b80 2024-04-26T20:37:51.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f9e000640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0105b80 0x7f3fa019b120 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00e9f0 tx=0x7f3f8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:51.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f97400640 1 -- 172.21.15.131:0/3383466243 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00ccc0 con 0x7f3fa0105b80 2024-04-26T20:37:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f97400640 1 -- 172.21.15.131:0/3383466243 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c00ce60 con 0x7f3fa0105b80 2024-04-26T20:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f97400640 1 -- 172.21.15.131:0/3383466243 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c010430 con 0x7f3fa0105b80 2024-04-26T20:37:51.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.316+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa01a2af0 con 0x7f3fa0105b80 2024-04-26T20:37:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.317+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa01a2ff0 con 0x7f3fa0105b80 2024-04-26T20:37:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.318+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa010bd80 con 0x7f3fa0105b80 2024-04-26T20:37:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.320+0000 7f3f97400640 1 -- 172.21.15.131:0/3383466243 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3f8c0046c0 con 0x7f3fa0105b80 2024-04-26T20:37:51.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.322+0000 7f3f97400640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f840a3c40 0x7f3f840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:51.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.322+0000 7f3f97400640 1 -- 172.21.15.131:0/3383466243 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7f3f8c0d0db0 con 0x7f3fa0105b80 2024-04-26T20:37:51.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.322+0000 7f3f9ea00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f840a3c40 0x7f3f840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:51.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.323+0000 7f3f9ea00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f840a3c40 0x7f3f840a6100 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009230 tx=0x7f3f900314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.326+0000 7f3f97400640 1 -- 172.21.15.131:0/3383466243 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f8c0d5070 con 0x7f3fa0105b80 2024-04-26T20:37:51.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:50 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3383466243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:37:51.274 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:50 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3383466243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.571+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fa002dcb0 con 0x7f3fa0105b80 2024-04-26T20:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.574+0000 7f3f97400640 1 -- 172.21.15.131:0/3383466243 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7f3f8c09d1e0 con 0x7f3fa0105b80 2024-04-26T20:37:51.353 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:37:51.363 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:12:47.696526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24571,24557]},"id":1}]} 2024-04-26T20:37:51.378 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-04-26T20:37:51.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f840a3c40 msgr2=0x7f3f840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f840a3c40 0x7f3f840a6100 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009230 tx=0x7f3f900314a0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0105b80 msgr2=0x7f3fa019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:51.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0105b80 0x7f3fa019b120 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00e9f0 tx=0x7f3f8c00edf0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 shutdown_connections 2024-04-26T20:37:51.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f840a3c40 0x7f3f840a6100 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa0106540 0x7f3fa01a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa0105b80 0x7f3fa019b120 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 --2- 172.21.15.131:0/3383466243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa0104990 0x7f3fa019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 >> 172.21.15.131:0/3383466243 conn(0x7f3fa00fff00 msgr2=0x7f3fa0108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:51.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 shutdown_connections 2024-04-26T20:37:51.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:50.578+0000 7f3f9fe00640 1 -- 172.21.15.131:0/3383466243 wait complete. 2024-04-26T20:37:51.607 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:37:51.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:50 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3383466243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:37:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.885+0000 7fea0b200640 1 -- 172.21.15.131:0/3647512057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c109f80 msgr2=0x7fea0c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:51.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.885+0000 7fea0b200640 1 --2- 172.21.15.131:0/3647512057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c109f80 0x7fea0c10a400 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009900 tx=0x7fe9f402a400 comp rx=0 tx=0).stop 2024-04-26T20:37:51.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.885+0000 7fea0b200640 1 -- 172.21.15.131:0/3647512057 shutdown_connections 2024-04-26T20:37:51.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.885+0000 7fea0b200640 1 --2- 172.21.15.131:0/3647512057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c10a940 0x7fea0c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.885+0000 7fea0b200640 1 --2- 172.21.15.131:0/3647512057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c109f80 0x7fea0c10a400 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.885+0000 7fea0b200640 1 --2- 172.21.15.131:0/3647512057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea0c108d90 0x7fea0c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:51.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.885+0000 7fea0b200640 1 -- 172.21.15.131:0/3647512057 >> 172.21.15.131:0/3647512057 conn(0x7fea0c077d30 msgr2=0x7fea0c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.886+0000 7fea0b200640 1 -- 172.21.15.131:0/3647512057 shutdown_connections 2024-04-26T20:37:51.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.886+0000 7fea0b200640 1 -- 172.21.15.131:0/3647512057 wait complete. 2024-04-26T20:37:52.001 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:51 smithi157 ceph-mon[32703]: pgmap v1245: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.886+0000 7fea0b200640 1 Processor -- start 2024-04-26T20:37:52.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0b200640 1 -- start start 2024-04-26T20:37:52.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea0c108d90 0x7fea0c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:52.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c109f80 0x7fea0c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:52.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c10a940 0x7fea0c1a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1148c0 con 0x7fea0c109f80 2024-04-26T20:37:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c114740 con 0x7fea0c10a940 2024-04-26T20:37:52.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c114a40 con 0x7fea0c108d90 2024-04-26T20:37:52.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea03e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c109f80 0x7fea0c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:52.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea09e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea0c108d90 0x7fea0c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:52.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea0a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c10a940 0x7fea0c1a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea03e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c109f80 0x7fea0c19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55328/0 (socket says 172.21.15.131:55328) 2024-04-26T20:37:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.887+0000 7fea03e00640 1 -- 172.21.15.131:0/1659870508 learned_addr learned my addr 172.21.15.131:0/1659870508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:52.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea0a800640 1 -- 172.21.15.131:0/1659870508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea0c108d90 msgr2=0x7fea0c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:52.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea0a800640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea0c108d90 0x7fea0c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:52.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea0a800640 1 -- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c109f80 msgr2=0x7fea0c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:52.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea0a800640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c109f80 0x7fea0c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:52.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea0a800640 1 -- 172.21.15.131:0/1659870508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c1a6be0 con 0x7fea0c10a940 2024-04-26T20:37:52.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea03e00640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c109f80 0x7fea0c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:37:52.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea0a800640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c10a940 0x7fea0c1a6570 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fea04009fd0 tx=0x7fea0400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:52.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea02a00640 1 -- 172.21.15.131:0/1659870508 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0400ce20 con 0x7fea0c10a940 2024-04-26T20:37:52.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea02a00640 1 -- 172.21.15.131:0/1659870508 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea04005020 con 0x7fea0c10a940 2024-04-26T20:37:52.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea02a00640 1 -- 172.21.15.131:0/1659870508 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04010690 con 0x7fea0c10a940 2024-04-26T20:37:52.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.888+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c1a6ed0 con 0x7fea0c10a940 2024-04-26T20:37:52.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.889+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c1a7390 con 0x7fea0c10a940 2024-04-26T20:37:52.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.890+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0c110180 con 0x7fea0c10a940 2024-04-26T20:37:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.892+0000 7fea02a00640 1 -- 172.21.15.131:0/1659870508 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fea040054d0 con 0x7fea0c10a940 2024-04-26T20:37:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.893+0000 7fea02a00640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9f00a3cc0 0x7fe9f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.894+0000 7fea02a00640 1 -- 172.21.15.131:0/1659870508 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fea040d1460 con 0x7fea0c10a940 2024-04-26T20:37:52.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.894+0000 7fea09e00640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9f00a3cc0 0x7fe9f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:52.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.894+0000 7fea09e00640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9f00a3cc0 0x7fe9f00a6180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea0c109de0 tx=0x7fe9fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:52.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:51.896+0000 7fea02a00640 1 -- 172.21.15.131:0/1659870508 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fea0409d890 con 0x7fea0c10a940 2024-04-26T20:37:52.648 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:51 smithi131 ceph-mon[27919]: pgmap v1245: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:52.703 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:51 smithi148 ceph-mon[32281]: pgmap v1245: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T20:37:52.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.140+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea0c02dd20 con 0x7fea0c10a940 2024-04-26T20:37:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.142+0000 7fea02a00640 1 -- 172.21.15.131:0/1659870508 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7fea0409d190 con 0x7fea0c10a940 2024-04-26T20:37:52.805 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:37:52.852 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3820661643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3820661643},{"type":"v1","addr":"172.21.15.131:6835","nonce":3820661643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3688291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3688291148},{"type":"v1","addr":"172.21.15.131:6837","nonce":3688291148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1138993646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1138993646},{"type":"v1","addr":"172.21.15.157:6833","nonce":1138993646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/248659002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":248659002},{"type":"v1","addr":"172.21.15.148:6833","nonce":248659002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2202047184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2202047184},{"type":"v1","addr":"172.21.15.148:6835","nonce":2202047184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1007065155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1007065155},{"type":"v1","addr":"172.21.15.157:6835","nonce":1007065155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T20:07:34.194239+0000","modified":"2024-04-26T20:12:47.696526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24571,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/3509306790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3509306790},{"type":"v1","addr":"172.21.15.157:6837","nonce":3509306790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6837/1158905219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1158905219},{"type":"v1","addr":"172.21.15.148:6837","nonce":1158905219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1960857281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1960857281},{"type":"v1","addr":"172.21.15.157:6839","nonce":1960857281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":28,"addr":"172.21.15.131:6839/2686418281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2686418281},{"type":"v1","addr":"172.21.15.131:6839","nonce":2686418281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.131:6841/1288851219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1288851219},{"type":"v1","addr":"172.21.15.131:6841","nonce":1288851219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6839/4047119723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4047119723},{"type":"v1","addr":"172.21.15.148:6839","nonce":4047119723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24571,24557]},"id":1}]} 2024-04-26T20:37:52.862 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-04-26T20:37:52.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.144+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9f00a3cc0 msgr2=0x7fe9f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:52.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.144+0000 7fea0b200640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9f00a3cc0 0x7fe9f00a6180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea0c109de0 tx=0x7fe9fc005f90 comp rx=0 tx=0).stop 2024-04-26T20:37:52.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.144+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c10a940 msgr2=0x7fea0c1a6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:52.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.144+0000 7fea0b200640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c10a940 0x7fea0c1a6570 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fea04009fd0 tx=0x7fea0400ef10 comp rx=0 tx=0).stop 2024-04-26T20:37:52.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 shutdown_connections 2024-04-26T20:37:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c10a940 0x7fea0c1a6570 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:52.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c109f80 0x7fea0c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:52.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe9f00a3cc0 0x7fe9f00a6180 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:52.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 --2- 172.21.15.131:0/1659870508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea0c108d90 0x7fea0c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:53.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 >> 172.21.15.131:0/1659870508 conn(0x7fea0c077d30 msgr2=0x7fea0c10d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 shutdown_connections 2024-04-26T20:37:53.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:52.145+0000 7fea0b200640 1 -- 172.21.15.131:0/1659870508 wait complete. 2024-04-26T20:37:53.056 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:37:53.125 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:52 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1659870508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:37:53.135 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:52 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1659870508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:37:53.158 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:52 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1659870508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T20:37:53.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 -- 172.21.15.131:0/880027191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240102770 msgr2=0x7fa240102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 --2- 172.21.15.131:0/880027191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240102770 0x7fa240102b90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa230009900 tx=0x7fa23002a410 comp rx=0 tx=0).stop 2024-04-26T20:37:53.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 -- 172.21.15.131:0/880027191 shutdown_connections 2024-04-26T20:37:53.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 --2- 172.21.15.131:0/880027191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240104320 0x7fa24010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:53.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 --2- 172.21.15.131:0/880027191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa240103960 0x7fa240103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:53.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 --2- 172.21.15.131:0/880027191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240102770 0x7fa240102b90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:53.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 -- 172.21.15.131:0/880027191 >> 172.21.15.131:0/880027191 conn(0x7fa2400fdd00 msgr2=0x7fa240100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:53.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 -- 172.21.15.131:0/880027191 shutdown_connections 2024-04-26T20:37:53.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.349+0000 7fa245800640 1 -- 172.21.15.131:0/880027191 wait complete. 2024-04-26T20:37:53.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.350+0000 7fa245800640 1 Processor -- start 2024-04-26T20:37:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.350+0000 7fa245800640 1 -- start start 2024-04-26T20:37:53.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa245800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240102770 0x7fa240198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa245800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240103960 0x7fa240198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:53.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa245800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa240104320 0x7fa24019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:53.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa245800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e2f0 con 0x7fa240103960 2024-04-26T20:37:53.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa245800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e170 con 0x7fa240104320 2024-04-26T20:37:53.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa245800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e470 con 0x7fa240102770 2024-04-26T20:37:53.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa23f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240103960 0x7fa240198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:53.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa23fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240102770 0x7fa240198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa23f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240103960 0x7fa240198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55342/0 (socket says 172.21.15.131:55342) 2024-04-26T20:37:53.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa23fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240102770 0x7fa240198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54938/0 (socket says 172.21.15.131:54938) 2024-04-26T20:37:53.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa23f400640 1 -- 172.21.15.131:0/2996807309 learned_addr learned my addr 172.21.15.131:0/2996807309 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:53.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.351+0000 7fa244e00640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa240104320 0x7fa24019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:53.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23fe00640 1 -- 172.21.15.131:0/2996807309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa240104320 msgr2=0x7fa24019ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:53.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23fe00640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa240104320 0x7fa24019ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:53.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23fe00640 1 -- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240103960 msgr2=0x7fa240198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:53.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23fe00640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240103960 0x7fa240198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:53.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23fe00640 1 -- 172.21.15.131:0/2996807309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2401a0580 con 0x7fa240102770 2024-04-26T20:37:53.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23fe00640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240102770 0x7fa240198960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa23002a930 tx=0x7fa230003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:53.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23cc00640 1 -- 172.21.15.131:0/2996807309 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23003f070 con 0x7fa240102770 2024-04-26T20:37:53.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23cc00640 1 -- 172.21.15.131:0/2996807309 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa230031c50 con 0x7fa240102770 2024-04-26T20:37:54.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2401a0810 con 0x7fa240102770 2024-04-26T20:37:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa23cc00640 1 -- 172.21.15.131:0/2996807309 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230035030 con 0x7fa240102770 2024-04-26T20:37:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.352+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2401a0d20 con 0x7fa240102770 2024-04-26T20:37:54.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.353+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa240109a90 con 0x7fa240102770 2024-04-26T20:37:54.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.356+0000 7fa23cc00640 1 -- 172.21.15.131:0/2996807309 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa230042070 con 0x7fa240102770 2024-04-26T20:37:54.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.357+0000 7fa23cc00640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2240a3cc0 0x7fa2240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:54.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.357+0000 7fa23cc00640 1 -- 172.21.15.131:0/2996807309 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fa2300f19b0 con 0x7fa240102770 2024-04-26T20:37:54.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.357+0000 7fa23f400640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2240a3cc0 0x7fa2240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.358+0000 7fa23f400640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2240a3cc0 0x7fa2240a6180 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa240199e70 tx=0x7fa22c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:54.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.359+0000 7fa23cc00640 1 -- 172.21.15.131:0/2996807309 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa2300bd790 con 0x7fa240102770 2024-04-26T20:37:54.190 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:53 smithi148 ceph-mon[32281]: pgmap v1246: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:37:54.199 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:53 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2996807309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:37:54.208 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:53 smithi157 ceph-mon[32703]: pgmap v1246: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:37:54.219 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:53 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2996807309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:37:54.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.563+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa24002dca0 con 0x7fa240102770 2024-04-26T20:37:54.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.565+0000 7fa23cc00640 1 -- 172.21.15.131:0/2996807309 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24966 (secure 0 0 0) 0x7fa2300c1a90 con 0x7fa240102770 2024-04-26T20:37:54.286 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-26T20:37:54.364 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"2027e7ca-0407-11ef-bc93-c7b262605968","created":"2024-04-26T19:58:05.764598+0000","modified":"2024-04-26T20:09:15.781789+0000","last_up_change":"2024-04-26T20:06:18.805363+0000","last_in_change":"2024-04-26T20:05:59.090410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:01:57.626290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:07:30.370934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:07:33.112161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:07:36.797488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ec125b6b-d0b4-42f7-80a6-66edb6de6715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6803","nonce":1131821741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6805","nonce":1131821741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6809","nonce":1131821741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1131821741},{"type":"v1","addr":"172.21.15.131:6807","nonce":1131821741}]},"public_addr":"172.21.15.131:6803/1131821741","cluster_addr":"172.21.15.131:6805/1131821741","heartbeat_back_addr":"172.21.15.131:6809/1131821741","heartbeat_front_addr":"172.21.15.131:6807/1131821741","state":["exists","up"]},{"osd":1,"uuid":"35db8b5f-c13f-4d01-872d-aae172804538","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6801","nonce":2432608282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6803","nonce":2432608282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6807","nonce":2432608282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2432608282},{"type":"v1","addr":"172.21.15.148:6805","nonce":2432608282}]},"public_addr":"172.21.15.148:6801/2432608282","cluster_addr":"172.21.15.148:6803/2432608282","heartbeat_back_addr":"172.21.15.148:6807/2432608282","heartbeat_front_addr":"172.21.15.148:6805/2432608282","state":["exists","up"]},{"osd":2,"uuid":"cbb0c1b6-b78d-4ee2-8ccb-92775d4f8e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6801","nonce":527328621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6803","nonce":527328621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6807","nonce":527328621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":527328621},{"type":"v1","addr":"172.21.15.157:6805","nonce":527328621}]},"public_addr":"172.21.15.157:6801/527328621","cluster_addr":"172.21.15.157:6803/527328621","heartbeat_back_addr":"172.21.15.157:6807/527328621","heartbeat_front_addr":"172.21.15.157:6805/527328621","state":["exists","up"]},{"osd":3,"uuid":"c248bb04-986a-4b38-af3b-2e2302a4313e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6811","nonce":1043436815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6813","nonce":1043436815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6817","nonce":1043436815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1043436815},{"type":"v1","addr":"172.21.15.131:6815","nonce":1043436815}]},"public_addr":"172.21.15.131:6811/1043436815","cluster_addr":"172.21.15.131:6813/1043436815","heartbeat_back_addr":"172.21.15.131:6817/1043436815","heartbeat_front_addr":"172.21.15.131:6815/1043436815","state":["exists","up"]},{"osd":4,"uuid":"de2985f5-6971-41c7-aee0-1d437e281514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6809","nonce":1242934013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6811","nonce":1242934013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6815","nonce":1242934013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1242934013},{"type":"v1","addr":"172.21.15.148:6813","nonce":1242934013}]},"public_addr":"172.21.15.148:6809/1242934013","cluster_addr":"172.21.15.148:6811/1242934013","heartbeat_back_addr":"172.21.15.148:6815/1242934013","heartbeat_front_addr":"172.21.15.148:6813/1242934013","state":["exists","up"]},{"osd":5,"uuid":"fa8194d9-9332-4a74-abe4-27a006a4f40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6809","nonce":2019329968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6811","nonce":2019329968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6815","nonce":2019329968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2019329968},{"type":"v1","addr":"172.21.15.157:6813","nonce":2019329968}]},"public_addr":"172.21.15.157:6809/2019329968","cluster_addr":"172.21.15.157:6811/2019329968","heartbeat_back_addr":"172.21.15.157:6815/2019329968","heartbeat_front_addr":"172.21.15.157:6813/2019329968","state":["exists","up"]},{"osd":6,"uuid":"4db9d2b9-02f6-4d5f-a40f-afbafa778151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6819","nonce":1518784346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6821","nonce":1518784346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6825","nonce":1518784346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1518784346},{"type":"v1","addr":"172.21.15.131:6823","nonce":1518784346}]},"public_addr":"172.21.15.131:6819/1518784346","cluster_addr":"172.21.15.131:6821/1518784346","heartbeat_back_addr":"172.21.15.131:6825/1518784346","heartbeat_front_addr":"172.21.15.131:6823/1518784346","state":["exists","up"]},{"osd":7,"uuid":"5300878b-efa3-49be-afa6-09efbaae8c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6817","nonce":367491900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6819","nonce":367491900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6823","nonce":367491900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":367491900},{"type":"v1","addr":"172.21.15.148:6821","nonce":367491900}]},"public_addr":"172.21.15.148:6817/367491900","cluster_addr":"172.21.15.148:6819/367491900","heartbeat_back_addr":"172.21.15.148:6823/367491900","heartbeat_front_addr":"172.21.15.148:6821/367491900","state":["exists","up"]},{"osd":8,"uuid":"3fb399a8-15a1-48dc-a538-63745daf7fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6817","nonce":1452104897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6819","nonce":1452104897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6823","nonce":1452104897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1452104897},{"type":"v1","addr":"172.21.15.157:6821","nonce":1452104897}]},"public_addr":"172.21.15.157:6817/1452104897","cluster_addr":"172.21.15.157:6819/1452104897","heartbeat_back_addr":"172.21.15.157:6823/1452104897","heartbeat_front_addr":"172.21.15.157:6821/1452104897","state":["exists","up"]},{"osd":9,"uuid":"4b119ef2-f46f-4511-9440-283185ed5f09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6827","nonce":529934339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6829","nonce":529934339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6833","nonce":529934339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":529934339},{"type":"v1","addr":"172.21.15.131:6831","nonce":529934339}]},"public_addr":"172.21.15.131:6827/529934339","cluster_addr":"172.21.15.131:6829/529934339","heartbeat_back_addr":"172.21.15.131:6833/529934339","heartbeat_front_addr":"172.21.15.131:6831/529934339","state":["exists","up"]},{"osd":10,"uuid":"0272c03a-fb92-456b-aee8-3380875cef29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6825","nonce":3795716153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6827","nonce":3795716153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6831","nonce":3795716153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3795716153},{"type":"v1","addr":"172.21.15.148:6829","nonce":3795716153}]},"public_addr":"172.21.15.148:6825/3795716153","cluster_addr":"172.21.15.148:6827/3795716153","heartbeat_back_addr":"172.21.15.148:6831/3795716153","heartbeat_front_addr":"172.21.15.148:6829/3795716153","state":["exists","up"]},{"osd":11,"uuid":"0fc8dd61-ca71-4459-ba2e-2a6e07f933d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6825","nonce":1583012638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6827","nonce":1583012638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6831","nonce":1583012638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1583012638},{"type":"v1","addr":"172.21.15.157:6829","nonce":1583012638}]},"public_addr":"172.21.15.157:6825/1583012638","cluster_addr":"172.21.15.157:6827/1583012638","heartbeat_back_addr":"172.21.15.157:6831/1583012638","heartbeat_front_addr":"172.21.15.157:6829/1583012638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:05.105922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:30.714863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:01:54.694865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:27.727663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:02:54.223769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:19.797028+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:03:54.601987+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:22.003566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:04:48.109736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:23.090085+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:05:50.945611+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:06:17.458126+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1f","mappings":[{"from":1,"to":2}]},{"pgid":"3.20","mappings":[{"from":2,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3514900702":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/3789992830":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6801/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/2579641682":"2024-04-27T19:59:13.443197+0000","172.21.15.131:6800/2560715060":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3589087841":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/3946098705":"2024-04-27T19:59:13.443197+0000","172.21.15.131:0/1232073682":"2024-04-27T19:58:38.942812+0000","172.21.15.131:0/537078740":"2024-04-27T19:58:38.942812+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:37:54.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2240a3cc0 msgr2=0x7fa2240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:54.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2240a3cc0 0x7fa2240a6180 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa240199e70 tx=0x7fa22c005f90 comp rx=0 tx=0).stop 2024-04-26T20:37:54.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240102770 msgr2=0x7fa240198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240102770 0x7fa240198960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa23002a930 tx=0x7fa230003c10 comp rx=0 tx=0).stop 2024-04-26T20:37:54.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 shutdown_connections 2024-04-26T20:37:54.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa240104320 0x7fa24019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa240103960 0x7fa240198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fa2240a3cc0 0x7fa2240a6180 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 --2- 172.21.15.131:0/2996807309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa240102770 0x7fa240198960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 >> 172.21.15.131:0/2996807309 conn(0x7fa2400fdd00 msgr2=0x7fa240106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:54.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 shutdown_connections 2024-04-26T20:37:54.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:53.568+0000 7fa245800640 1 -- 172.21.15.131:0/2996807309 wait complete. 2024-04-26T20:37:54.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:37:54.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:53 smithi131 ceph-mon[27919]: pgmap v1246: 129 pgs: 129 active+clean; 1.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T20:37:54.590 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:53 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2996807309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T20:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 -- 172.21.15.131:0/503258156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee201033f0 msgr2=0x7fee201057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:54.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 --2- 172.21.15.131:0/503258156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee201033f0 0x7fee201057e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fee10009900 tx=0x7fee1002a3f0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 -- 172.21.15.131:0/503258156 shutdown_connections 2024-04-26T20:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 --2- 172.21.15.131:0/503258156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee20108650 0x7fee2010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 --2- 172.21.15.131:0/503258156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee20105d20 0x7fee20108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 --2- 172.21.15.131:0/503258156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee201033f0 0x7fee201057e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 -- 172.21.15.131:0/503258156 >> 172.21.15.131:0/503258156 conn(0x7fee200fcde0 msgr2=0x7fee200ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:54.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 -- 172.21.15.131:0/503258156 shutdown_connections 2024-04-26T20:37:55.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.812+0000 7fee26400640 1 -- 172.21.15.131:0/503258156 wait complete. 2024-04-26T20:37:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.813+0000 7fee26400640 1 Processor -- start 2024-04-26T20:37:55.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee26400640 1 -- start start 2024-04-26T20:37:55.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee26400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee201033f0 0x7fee20102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:55.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee26400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee20105d20 0x7fee201013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:55.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee26400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee20108650 0x7fee201018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:55.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee26400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee2010e520 con 0x7fee20105d20 2024-04-26T20:37:55.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee26400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee2010e3a0 con 0x7fee20108650 2024-04-26T20:37:55.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee26400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee2010e6a0 con 0x7fee201033f0 2024-04-26T20:37:55.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee1f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee201033f0 0x7fee20102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:55.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee1f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee201033f0 0x7fee20102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45286/0 (socket says 172.21.15.131:45286) 2024-04-26T20:37:55.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee1f400640 1 -- 172.21.15.131:0/1887553907 learned_addr learned my addr 172.21.15.131:0/1887553907 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:55.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee1ea00640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee20105d20 0x7fee201013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.814+0000 7fee1fe00640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee20108650 0x7fee201018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:55.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee1f400640 1 -- 172.21.15.131:0/1887553907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee20108650 msgr2=0x7fee201018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee1f400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee20108650 0x7fee201018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:55.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee1f400640 1 -- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee20105d20 msgr2=0x7fee201013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee1f400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee20105d20 0x7fee201013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee1f400640 1 -- 172.21.15.131:0/1887553907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee20102170 con 0x7fee201033f0 2024-04-26T20:37:55.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee1f400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee201033f0 0x7fee20102d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fee10005ab0 tx=0x7fee10003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:55.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee17e00640 1 -- 172.21.15.131:0/1887553907 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1003f070 con 0x7fee201033f0 2024-04-26T20:37:55.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee17e00640 1 -- 172.21.15.131:0/1887553907 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee10031c50 con 0x7fee201033f0 2024-04-26T20:37:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee201a2cb0 con 0x7fee201033f0 2024-04-26T20:37:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.815+0000 7fee17e00640 1 -- 172.21.15.131:0/1887553907 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10035030 con 0x7fee201033f0 2024-04-26T20:37:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.816+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee201a31b0 con 0x7fee201033f0 2024-04-26T20:37:55.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.817+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee200fe8d0 con 0x7fee201033f0 2024-04-26T20:37:55.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.819+0000 7fee17e00640 1 -- 172.21.15.131:0/1887553907 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fee10042070 con 0x7fee201033f0 2024-04-26T20:37:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.820+0000 7fee17e00640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fee000a3c40 0x7fee000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:55.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.820+0000 7fee17e00640 1 -- 172.21.15.131:0/1887553907 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9245+0+0 (secure 0 0 0) 0x7fee100f1840 con 0x7fee201033f0 2024-04-26T20:37:55.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.820+0000 7fee1ea00640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fee000a3c40 0x7fee000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.821+0000 7fee1ea00640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fee000a3c40 0x7fee000a6100 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee20102ad0 tx=0x7fee08009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:55.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:54.822+0000 7fee17e00640 1 -- 172.21.15.131:0/1887553907 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fee100bd620 con 0x7fee201033f0 2024-04-26T20:37:55.627 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b[63000]: ignoring --setuser ceph since I am not root 2024-04-26T20:37:55.685 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b[63000]: ignoring --setgroup ceph since I am not root 2024-04-26T20:37:55.694 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mds[63004]: -- 172.21.15.148:0/697318310 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe62189a40 con 0x55fe6215c800 2024-04-26T20:37:55.703 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b[63000]: starting mds.b at 2024-04-26T20:37:55.712 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c[63527]: ignoring --setuser ceph since I am not root 2024-04-26T20:37:55.720 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c[63527]: ignoring --setgroup ceph since I am not root 2024-04-26T20:37:55.729 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mds[63531]: -- 172.21.15.157:0/269583557 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622faabba40 con 0x5622faa8e800 2024-04-26T20:37:55.737 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c[63527]: starting mds.c at 2024-04-26T20:37:55.746 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h[62679]: ignoring --setuser ceph since I am not root 2024-04-26T20:37:55.756 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h[62679]: ignoring --setgroup ceph since I am not root 2024-04-26T20:37:55.784 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mds[62683]: -- 172.21.15.148:0/3002435773 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56101b2bfa40 con 0x56101b293000 2024-04-26T20:37:55.793 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h[62679]: starting mds.h at 2024-04-26T20:37:55.854 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i[63846]: ignoring --setuser ceph since I am not root 2024-04-26T20:37:55.862 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i[63846]: ignoring --setgroup ceph since I am not root 2024-04-26T20:37:55.897 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mds[63850]: -- 172.21.15.157:0/1338053004 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f7b8dda40 con 0x561f7b8b1000 2024-04-26T20:37:55.906 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i[63846]: starting mds.i at 2024-04-26T20:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.070+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fee20063530 con 0x7fee201033f0 2024-04-26T20:37:55.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.081+0000 7fee17e00640 1 -- 172.21.15.131:0/1887553907 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fee100c1920 con 0x7fee201033f0 2024-04-26T20:37:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:37:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fee000a3c40 msgr2=0x7fee000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fee000a3c40 0x7fee000a6100 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee20102ad0 tx=0x7fee08009500 comp rx=0 tx=0).stop 2024-04-26T20:37:56.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee201033f0 msgr2=0x7fee20102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee201033f0 0x7fee20102d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fee10005ab0 tx=0x7fee10003b90 comp rx=0 tx=0).stop 2024-04-26T20:37:56.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 shutdown_connections 2024-04-26T20:37:56.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee20108650 0x7fee201018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:56.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee20105d20 0x7fee201013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:56.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fee000a3c40 0x7fee000a6100 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:56.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 --2- 172.21.15.131:0/1887553907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee201033f0 0x7fee20102d60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:56.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 >> 172.21.15.131:0/1887553907 conn(0x7fee200fcde0 msgr2=0x7fee20103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 shutdown_connections 2024-04-26T20:37:56.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:55.084+0000 7fee26400640 1 -- 172.21.15.131:0/1887553907 wait complete. 2024-04-26T20:37:56.247 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:37:56.255 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a[27915]: 2024-04-26T20:37:55.072+0000 7fdfc9e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:37:56.264 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a[69256]: ignoring --setuser ceph since I am not root 2024-04-26T20:37:56.271 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a[69256]: ignoring --setgroup ceph since I am not root 2024-04-26T20:37:56.329 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mds[69260]: -- 172.21.15.131:0/719383439 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627b762da40 con 0x5627b7601000 2024-04-26T20:37:56.337 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a[69256]: starting mds.a at 2024-04-26T20:37:56.346 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g[68903]: ignoring --setuser ceph since I am not root 2024-04-26T20:37:56.354 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g[68903]: ignoring --setgroup ceph since I am not root 2024-04-26T20:37:56.362 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mds[68912]: -- 172.21.15.131:0/4043506565 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55706f5a3a40 con 0x55706f576c00 2024-04-26T20:37:56.371 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g[68903]: starting mds.g at 2024-04-26T20:37:56.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: pgmap v1247: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T20:37:56.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1887553907' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T20:37:56.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T20:37:56.405 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T20:37:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:37:56.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T20:37:56.585 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T20:37:56.659 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:55 smithi131 ceph-mon[27919]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T20:37:56.668 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: pgmap v1247: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T20:37:56.678 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1887553907' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T20:37:56.686 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T20:37:56.769 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T20:37:56.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:37:56.788 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T20:37:56.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T20:37:56.805 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:55 smithi148 ceph-mon[32281]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T20:37:56.813 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: pgmap v1247: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T20:37:56.822 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1887553907' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T20:37:56.830 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T20:37:56.839 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T20:37:56.848 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:37:56.892 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T20:37:56.955 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T20:37:56.964 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:55 smithi157 ceph-mon[32703]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T20:37:56.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 -- 172.21.15.131:0/4044105209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc076910 msgr2=0x7fe1cc074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/4044105209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc076910 0x7fe1cc074d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009ca0 tx=0x7fe1c402a950 comp rx=0 tx=0).stop 2024-04-26T20:37:57.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 -- 172.21.15.131:0/4044105209 shutdown_connections 2024-04-26T20:37:57.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/4044105209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1cc10a6f0 0x7fe1cc10cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:57.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/4044105209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 0x7fe1cc075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/4044105209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc076910 0x7fe1cc074d60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:57.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 -- 172.21.15.131:0/4044105209 >> 172.21.15.131:0/4044105209 conn(0x7fe1cc0fddd0 msgr2=0x7fe1cc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:57.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.887+0000 7fe1d1c00640 1 -- 172.21.15.131:0/4044105209 shutdown_connections 2024-04-26T20:37:57.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.888+0000 7fe1d1c00640 1 -- 172.21.15.131:0/4044105209 wait complete. 2024-04-26T20:37:57.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.888+0000 7fe1d1c00640 1 Processor -- start 2024-04-26T20:37:57.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1c00640 1 -- start start 2024-04-26T20:37:57.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 0x7fe1cc19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1cc076910 0x7fe1cc19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc10a6f0 0x7fe1cc1a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:57.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc110580 con 0x7fe1cc075330 2024-04-26T20:37:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc110400 con 0x7fe1cc076910 2024-04-26T20:37:57.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc110700 con 0x7fe1cc10a6f0 2024-04-26T20:37:57.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 0x7fe1cc19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:57.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 0x7fe1cc19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37586/0 (socket says 172.21.15.131:37586) 2024-04-26T20:37:57.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1cbe00640 1 -- 172.21.15.131:0/2368625841 learned_addr learned my addr 172.21.15.131:0/2368625841 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:57.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1cb400640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1cc076910 0x7fe1cc19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.889+0000 7fe1d1200640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc10a6f0 0x7fe1cc1a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:57.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1cbe00640 1 -- 172.21.15.131:0/2368625841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc10a6f0 msgr2=0x7fe1cc1a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1cbe00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc10a6f0 0x7fe1cc1a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:57.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1cbe00640 1 -- 172.21.15.131:0/2368625841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1cc076910 msgr2=0x7fe1cc19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:57.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1cbe00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1cc076910 0x7fe1cc19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:57.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1cbe00640 1 -- 172.21.15.131:0/2368625841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc1a28e0 con 0x7fe1cc075330 2024-04-26T20:37:57.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1cbe00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 0x7fe1cc19aca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4000f80 tx=0x7fe1c402fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:57.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1c8c00640 1 -- 172.21.15.131:0/2368625841 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4034040 con 0x7fe1cc075330 2024-04-26T20:37:57.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc1a2b70 con 0x7fe1cc075330 2024-04-26T20:37:57.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1c8c00640 1 -- 172.21.15.131:0/2368625841 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c40308f0 con 0x7fe1cc075330 2024-04-26T20:37:57.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1c8c00640 1 -- 172.21.15.131:0/2368625841 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4039450 con 0x7fe1cc075330 2024-04-26T20:37:57.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.890+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc1a30b0 con 0x7fe1cc075330 2024-04-26T20:37:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.891+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc1094b0 con 0x7fe1cc075330 2024-04-26T20:37:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.893+0000 7fe1c8c00640 1 -- 172.21.15.131:0/2368625841 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe1c4037020 con 0x7fe1cc075330 2024-04-26T20:37:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.895+0000 7fe1c8c00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe1b00a3c40 0x7fe1b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:57.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.895+0000 7fe1c8c00640 1 -- 172.21.15.131:0/2368625841 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9503+0+0 (secure 0 0 0) 0x7fe1c40f18e0 con 0x7fe1cc075330 2024-04-26T20:37:57.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.895+0000 7fe1cb400640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe1b00a3c40 0x7fe1b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.896+0000 7fe1cb400640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe1b00a3c40 0x7fe1b00a6100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc19c1a0 tx=0x7fe1b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:56.899+0000 7fe1c8c00640 1 -- 172.21.15.131:0/2368625841 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe1c40bddb0 con 0x7fe1cc075330 2024-04-26T20:37:57.725 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.157:6837/1433319703,v1:172.21.15.157:6839/1433319703] up:boot 2024-04-26T20:37:57.733 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.157:6836/3104932279,v1:172.21.15.157:6838/3104932279] up:boot 2024-04-26T20:37:57.742 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.148:6836/3316661236,v1:172.21.15.148:6838/3316661236] up:boot 2024-04-26T20:37:57.750 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.148:6837/1201109406,v1:172.21.15.148:6839/1201109406] up:boot 2024-04-26T20:37:57.759 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6838/1621075092,v1:172.21.15.131:6840/1621075092] up:boot 2024-04-26T20:37:57.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: mds.? [v2:172.21.15.131:6839/4246336311,v1:172.21.15.131:6841/4246336311] up:boot 2024-04-26T20:37:57.777 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T20:37:57.785 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T20:37:57.843 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T20:37:57.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T20:37:57.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T20:37:57.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T20:37:57.990 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T20:37:57.999 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:57 smithi148 ceph-mon[32281]: pgmap v1249: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-26T20:37:58.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.157:6837/1433319703,v1:172.21.15.157:6839/1433319703] up:boot 2024-04-26T20:37:58.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.157:6836/3104932279,v1:172.21.15.157:6838/3104932279] up:boot 2024-04-26T20:37:58.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.148:6836/3316661236,v1:172.21.15.148:6838/3316661236] up:boot 2024-04-26T20:37:58.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.148:6837/1201109406,v1:172.21.15.148:6839/1201109406] up:boot 2024-04-26T20:37:58.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6838/1621075092,v1:172.21.15.131:6840/1621075092] up:boot 2024-04-26T20:37:58.096 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: mds.? [v2:172.21.15.131:6839/4246336311,v1:172.21.15.131:6841/4246336311] up:boot 2024-04-26T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T20:37:58.116 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T20:37:58.125 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T20:37:58.135 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T20:37:58.143 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T20:37:58.206 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T20:37:58.261 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T20:37:58.281 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:57 smithi157 ceph-mon[32703]: pgmap v1249: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-26T20:37:58.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:57.145+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe1cc02dd20 con 0x7fe1cc075330 2024-04-26T20:37:58.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.157:6837/1433319703,v1:172.21.15.157:6839/1433319703] up:boot 2024-04-26T20:37:58.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.157:6836/3104932279,v1:172.21.15.157:6838/3104932279] up:boot 2024-04-26T20:37:58.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.148:6836/3316661236,v1:172.21.15.148:6838/3316661236] up:boot 2024-04-26T20:37:58.470 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.148:6837/1201109406,v1:172.21.15.148:6839/1201109406] up:boot 2024-04-26T20:37:58.479 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6838/1621075092,v1:172.21.15.131:6840/1621075092] up:boot 2024-04-26T20:37:58.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: mds.? [v2:172.21.15.131:6839/4246336311,v1:172.21.15.131:6841/4246336311] up:boot 2024-04-26T20:37:58.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T20:37:58.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T20:37:58.512 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T20:37:58.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T20:37:58.530 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T20:37:58.539 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T20:37:58.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: from='mgr.14152 172.21.15.131:0/279815964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T20:37:58.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:57 smithi131 ceph-mon[27919]: pgmap v1249: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-26T20:37:58.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.105+0000 7fe1c8c00640 1 -- 172.21.15.131:0/2368625841 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fe1c4030ac0 con 0x7fe1cc075330 2024-04-26T20:37:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.109+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe1b00a3c40 msgr2=0x7fe1b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:58.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.109+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe1b00a3c40 0x7fe1b00a6100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc19c1a0 tx=0x7fe1b8005f90 comp rx=0 tx=0).stop 2024-04-26T20:37:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.109+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 msgr2=0x7fe1cc19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:58.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.109+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 0x7fe1cc19aca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4000f80 tx=0x7fe1c402fc20 comp rx=0 tx=0).stop 2024-04-26T20:37:58.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.109+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 shutdown_connections 2024-04-26T20:37:58.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.110+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fe1b00a3c40 0x7fe1b00a6100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:58.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.110+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc10a6f0 0x7fe1cc1a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:58.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.110+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1cc076910 0x7fe1cc19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:58.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.110+0000 7fe1d1c00640 1 --2- 172.21.15.131:0/2368625841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc075330 0x7fe1cc19aca0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:58.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.110+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 >> 172.21.15.131:0/2368625841 conn(0x7fe1cc0fddd0 msgr2=0x7fe1cc106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:58.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.110+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 shutdown_connections 2024-04-26T20:37:58.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:58.110+0000 7fe1d1c00640 1 -- 172.21.15.131:0/2368625841 wait complete. 2024-04-26T20:37:58.874 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:37:58.882 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:58 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2368625841' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:37:58.942 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:58 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2368625841' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:37:58.951 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:58 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2368625841' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T20:37:59.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 -- 172.21.15.131:0/648516834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414104510 msgr2=0x7fc41410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:59.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 --2- 172.21.15.131:0/648516834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414104510 0x7fc41410ada0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009be0 tx=0x7fc40c02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:37:59.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 -- 172.21.15.131:0/648516834 shutdown_connections 2024-04-26T20:37:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 --2- 172.21.15.131:0/648516834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414104510 0x7fc41410ada0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:59.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 --2- 172.21.15.131:0/648516834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc414103b50 0x7fc414103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 --2- 172.21.15.131:0/648516834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414102960 0x7fc414102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 -- 172.21.15.131:0/648516834 >> 172.21.15.131:0/648516834 conn(0x7fc4140fdef0 msgr2=0x7fc414100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:37:59.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 -- 172.21.15.131:0/648516834 shutdown_connections 2024-04-26T20:37:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.161+0000 7fc41c200640 1 -- 172.21.15.131:0/648516834 wait complete. 2024-04-26T20:37:59.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.162+0000 7fc41c200640 1 Processor -- start 2024-04-26T20:37:59.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.162+0000 7fc41c200640 1 -- start start 2024-04-26T20:37:59.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41c200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc414102960 0x7fc41419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:59.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414103b50 0x7fc41419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:59.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41c200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414104510 0x7fc4141a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:59.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41c200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410e490 con 0x7fc414102960 2024-04-26T20:37:59.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41c200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410e310 con 0x7fc414104510 2024-04-26T20:37:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410e610 con 0x7fc414103b50 2024-04-26T20:37:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc419800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc414102960 0x7fc41419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:59.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414104510 0x7fc4141a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414104510 0x7fc4141a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:60464/0 (socket says 172.21.15.131:60464) 2024-04-26T20:37:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 -- 172.21.15.131:0/3704405802 learned_addr learned my addr 172.21.15.131:0/3704405802 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:37:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc418e00640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414103b50 0x7fc41419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 -- 172.21.15.131:0/3704405802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414103b50 msgr2=0x7fc41419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414103b50 0x7fc41419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:59.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 -- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc414102960 msgr2=0x7fc41419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:37:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc414102960 0x7fc41419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:37:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.163+0000 7fc41a200640 1 -- 172.21.15.131:0/3704405802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4141a2960 con 0x7fc414104510 2024-04-26T20:37:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.164+0000 7fc41a200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414104510 0x7fc4141a22f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc40c004af0 tx=0x7fc40c030d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:59.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.164+0000 7fc412000640 1 -- 172.21.15.131:0/3704405802 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c041020 con 0x7fc414104510 2024-04-26T20:37:59.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.164+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4141a2bf0 con 0x7fc414104510 2024-04-26T20:37:59.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.164+0000 7fc412000640 1 -- 172.21.15.131:0/3704405802 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c0318f0 con 0x7fc414104510 2024-04-26T20:37:59.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.164+0000 7fc412000640 1 -- 172.21.15.131:0/3704405802 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c036040 con 0x7fc414104510 2024-04-26T20:37:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.164+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141a3130 con 0x7fc414104510 2024-04-26T20:37:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.165+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4141a3300 con 0x7fc414104510 2024-04-26T20:37:59.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.167+0000 7fc412000640 1 -- 172.21.15.131:0/3704405802 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc40c044020 con 0x7fc414104510 2024-04-26T20:37:59.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.168+0000 7fc412000640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3fc0a3c40 0x7fc3fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:37:59.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.169+0000 7fc412000640 1 -- 172.21.15.131:0/3704405802 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9503+0+0 (secure 0 0 0) 0x7fc40c0f1750 con 0x7fc414104510 2024-04-26T20:37:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.169+0000 7fc419800640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3fc0a3c40 0x7fc3fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:37:59.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.169+0000 7fc419800640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3fc0a3c40 0x7fc3fc0a6100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc4141039b0 tx=0x7fc408005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:37:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.173+0000 7fc412000640 1 -- 172.21.15.131:0/3704405802 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc40c04b020 con 0x7fc414104510 2024-04-26T20:38:00.068 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:59 smithi148 ceph-mon[32281]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T20:38:00.077 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:59 smithi148 ceph-mon[32281]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T20:38:00.086 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:59 smithi148 ceph-mon[32281]: Cluster is now healthy 2024-04-26T20:38:00.095 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:59 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/2368625841' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T20:38:00.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:59 smithi148 ceph-mon[32281]: fsmap 12 up:standby 2024-04-26T20:38:00.114 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:37:59 smithi148 ceph-mon[32281]: pgmap v1250: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 818 B/s wr, 6 op/s 2024-04-26T20:38:00.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:59 smithi157 ceph-mon[32703]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T20:38:00.131 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:59 smithi157 ceph-mon[32703]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T20:38:00.139 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:59 smithi157 ceph-mon[32703]: Cluster is now healthy 2024-04-26T20:38:00.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:59 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/2368625841' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T20:38:00.208 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:59 smithi157 ceph-mon[32703]: fsmap 12 up:standby 2024-04-26T20:38:00.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:37:59 smithi157 ceph-mon[32703]: pgmap v1250: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 818 B/s wr, 6 op/s 2024-04-26T20:38:00.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:37:59.378+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc4141a34d0 con 0x7fc414104510 2024-04-26T20:38:00.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:59 smithi131 ceph-mon[27919]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T20:38:00.319 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:59 smithi131 ceph-mon[27919]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T20:38:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:59 smithi131 ceph-mon[27919]: Cluster is now healthy 2024-04-26T20:38:00.336 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:59 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/2368625841' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T20:38:00.408 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:59 smithi131 ceph-mon[27919]: fsmap 12 up:standby 2024-04-26T20:38:00.418 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:37:59 smithi131 ceph-mon[27919]: pgmap v1250: 129 pgs: 129 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 818 B/s wr, 6 op/s 2024-04-26T20:38:00.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.115+0000 7fc412000640 1 -- 172.21.15.131:0/3704405802 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7fc40c0bdb90 con 0x7fc414104510 2024-04-26T20:38:00.437 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:38:00.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.121+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3fc0a3c40 msgr2=0x7fc3fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:00.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.121+0000 7fc41c200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3fc0a3c40 0x7fc3fc0a6100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc4141039b0 tx=0x7fc408005f90 comp rx=0 tx=0).stop 2024-04-26T20:38:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.121+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414104510 msgr2=0x7fc4141a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:00.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.121+0000 7fc41c200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414104510 0x7fc4141a22f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc40c004af0 tx=0x7fc40c030d50 comp rx=0 tx=0).stop 2024-04-26T20:38:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.122+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 shutdown_connections 2024-04-26T20:38:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.122+0000 7fc41c200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc414104510 0x7fc4141a22f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:00.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.122+0000 7fc41c200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7fc3fc0a3c40 0x7fc3fc0a6100 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.122+0000 7fc41c200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc414103b50 0x7fc41419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:00.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.122+0000 7fc41c200640 1 --2- 172.21.15.131:0/3704405802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc414102960 0x7fc41419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:00.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.122+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 >> 172.21.15.131:0/3704405802 conn(0x7fc4140fdef0 msgr2=0x7fc414074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:38:00.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.123+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 shutdown_connections 2024-04-26T20:38:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:00.123+0000 7fc41c200640 1 -- 172.21.15.131:0/3704405802 wait complete. 2024-04-26T20:38:00.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:38:00.758 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:00 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/3704405802' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:00.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:00 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:00 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/3704405802' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:00.784 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:00 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:00.793 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:00 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/3704405802' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:00.802 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:00 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 -- 172.21.15.131:0/274998959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 msgr2=0x7f3fb4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:01.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/274998959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 0x7f3fb4078990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009900 tx=0x7f3f9c02a3f0 comp rx=0 tx=0).stop 2024-04-26T20:38:01.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 -- 172.21.15.131:0/274998959 shutdown_connections 2024-04-26T20:38:01.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/274998959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb4078ed0 0x7f3fb4079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:01.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/274998959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 0x7f3fb4078990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:01.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/274998959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb40772f0 0x7f3fb4077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:01.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 -- 172.21.15.131:0/274998959 >> 172.21.15.131:0/274998959 conn(0x7f3fb40fffe0 msgr2=0x7f3fb4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:38:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 -- 172.21.15.131:0/274998959 shutdown_connections 2024-04-26T20:38:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.021+0000 7f3fb8c00640 1 -- 172.21.15.131:0/274998959 wait complete. 2024-04-26T20:38:01.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 Processor -- start 2024-04-26T20:38:01.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 -- start start 2024-04-26T20:38:01.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb40772f0 0x7f3fb419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:01.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 0x7f3fb419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:01.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4078ed0 0x7f3fb41a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:01.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41149b0 con 0x7f3fb40772f0 2024-04-26T20:38:01.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4114830 con 0x7f3fb4078530 2024-04-26T20:38:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.022+0000 7f3fb8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4114b30 con 0x7f3fb4078ed0 2024-04-26T20:38:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb40772f0 0x7f3fb419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb40772f0 0x7f3fb419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37622/0 (socket says 172.21.15.131:37622) 2024-04-26T20:38:01.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 -- 172.21.15.131:0/564548564 learned_addr learned my addr 172.21.15.131:0/564548564 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:38:01.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3faaa00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 0x7f3fb419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:01.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3e00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4078ed0 0x7f3fb41a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 -- 172.21.15.131:0/564548564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4078ed0 msgr2=0x7f3fb41a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:01.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4078ed0 0x7f3fb41a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 -- 172.21.15.131:0/564548564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 msgr2=0x7f3fb419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:01.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 0x7f3fb419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:01.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 -- 172.21.15.131:0/564548564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb41a6cd0 con 0x7f3fb40772f0 2024-04-26T20:38:01.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3faaa00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 0x7f3fb419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.023+0000 7f3fb3400640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb40772f0 0x7f3fb419f0b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e9f0 tx=0x7f3fa400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:38:01.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.024+0000 7f3fb0c00640 1 -- 172.21.15.131:0/564548564 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400cd80 con 0x7f3fb40772f0 2024-04-26T20:38:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.024+0000 7f3fb0c00640 1 -- 172.21.15.131:0/564548564 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fa40108e0 con 0x7f3fb40772f0 2024-04-26T20:38:01.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.024+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb41a6fc0 con 0x7f3fb40772f0 2024-04-26T20:38:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.024+0000 7f3fb0c00640 1 -- 172.21.15.131:0/564548564 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4018980 con 0x7f3fb40772f0 2024-04-26T20:38:01.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.024+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb41a7430 con 0x7f3fb40772f0 2024-04-26T20:38:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.025+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb41113e0 con 0x7f3fb40772f0 2024-04-26T20:38:01.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.027+0000 7f3fb0c00640 1 -- 172.21.15.131:0/564548564 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3fa4010430 con 0x7f3fb40772f0 2024-04-26T20:38:01.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.028+0000 7f3fb0c00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f980a3cc0 0x7f3f980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:01.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.029+0000 7f3fb0c00640 1 -- 172.21.15.131:0/564548564 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9007+0+0 (secure 0 0 0) 0x7f3fa40d0b50 con 0x7f3fb40772f0 2024-04-26T20:38:01.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.029+0000 7f3faaa00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f980a3cc0 0x7f3f980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:02.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.029+0000 7f3faaa00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f980a3cc0 0x7f3f980a6180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000c00 tx=0x7f3f9c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:38:02.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.033+0000 7f3fb0c00640 1 -- 172.21.15.131:0/564548564 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3fa409ca20 con 0x7f3fb40772f0 2024-04-26T20:38:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:01.237+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3fb402dad0 con 0x7f3fb40772f0 2024-04-26T20:38:02.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:01 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:02.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:01 smithi131 ceph-mon[27919]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T20:38:02.092 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:01 smithi131 ceph-mon[27919]: pgmap v1252: 97 pgs: 97 active+clean; 577 KiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:02.159 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:01 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:02.168 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:01 smithi148 ceph-mon[32281]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T20:38:02.177 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:01 smithi148 ceph-mon[32281]: pgmap v1252: 97 pgs: 97 active+clean; 577 KiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:02.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:01 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:02.194 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:01 smithi157 ceph-mon[32703]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T20:38:02.202 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:01 smithi157 ceph-mon[32703]: pgmap v1252: 97 pgs: 97 active+clean; 577 KiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:02.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.145+0000 7f3fb0c00640 1 -- 172.21.15.131:0/564548564 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f3fa40a0d20 con 0x7f3fb40772f0 2024-04-26T20:38:02.222 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-04-26T20:38:02.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f980a3cc0 msgr2=0x7f3f980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:02.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f980a3cc0 0x7f3f980a6180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000c00 tx=0x7f3f9c0314a0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb40772f0 msgr2=0x7f3fb419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb40772f0 0x7f3fb419f0b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e9f0 tx=0x7f3fa400eeb0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 shutdown_connections 2024-04-26T20:38:02.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f3f980a3cc0 0x7f3f980a6180 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000c00 tx=0x7f3f9c0314a0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4078ed0 0x7f3fb41a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4078530 0x7f3fb419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.149+0000 7f3fb8c00640 1 --2- 172.21.15.131:0/564548564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fb40772f0 0x7f3fb419f0b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.150+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 >> 172.21.15.131:0/564548564 conn(0x7f3fb40fffe0 msgr2=0x7f3fb4103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:38:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.152+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 shutdown_connections 2024-04-26T20:38:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.152+0000 7f3fb8c00640 1 -- 172.21.15.131:0/564548564 wait complete. 2024-04-26T20:38:02.549 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T20:38:02.559 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:02 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/564548564' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:02.567 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:02 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/564548564' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:02.576 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:02 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/564548564' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:02.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 -- 172.21.15.131:0/1415110160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c105d00 msgr2=0x7f246c1080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:02.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 --2- 172.21.15.131:0/1415110160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c105d00 0x7f246c1080f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2458009900 tx=0x7f245802a410 comp rx=0 tx=0).stop 2024-04-26T20:38:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 -- 172.21.15.131:0/1415110160 shutdown_connections 2024-04-26T20:38:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 --2- 172.21.15.131:0/1415110160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f246c108630 0x7f246c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 --2- 172.21.15.131:0/1415110160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c105d00 0x7f246c1080f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 --2- 172.21.15.131:0/1415110160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f246c1033d0 0x7f246c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:02.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 -- 172.21.15.131:0/1415110160 >> 172.21.15.131:0/1415110160 conn(0x7f246c0fcdc0 msgr2=0x7f246c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:38:03.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 -- 172.21.15.131:0/1415110160 shutdown_connections 2024-04-26T20:38:03.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.798+0000 7f2470a00640 1 -- 172.21.15.131:0/1415110160 wait complete. 2024-04-26T20:38:03.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.799+0000 7f2470a00640 1 Processor -- start 2024-04-26T20:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.799+0000 7f2470a00640 1 -- start start 2024-04-26T20:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2470a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c1033d0 0x7f246c102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:03.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2470a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f246c105d00 0x7f246c101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:03.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2470a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f246c108630 0x7f246c1018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:03.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2470a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f246c10e490 con 0x7f246c108630 2024-04-26T20:38:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2470a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f246c10e310 con 0x7f246c1033d0 2024-04-26T20:38:03.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2470a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f246c10e610 con 0x7f246c105d00 2024-04-26T20:38:03.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f246c105d00 0x7f246c101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:03.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c1033d0 0x7f246c102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:03.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f246a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f246c108630 0x7f246c1018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:03.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c1033d0 0x7f246c102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.131:60498/0 (socket says 172.21.15.131:60498) 2024-04-26T20:38:03.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 -- 172.21.15.131:0/1787161240 learned_addr learned my addr 172.21.15.131:0/1787161240 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-26T20:38:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f246c105d00 0x7f246c101380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45416/0 (socket says 172.21.15.131:45416) 2024-04-26T20:38:03.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 -- 172.21.15.131:0/1787161240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f246c105d00 msgr2=0x7f246c101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:03.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f246c105d00 0x7f246c101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:03.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 -- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f246c108630 msgr2=0x7f246c1018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f246c108630 0x7f246c1018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.800+0000 7f2469e00640 1 -- 172.21.15.131:0/1787161240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c102150 con 0x7f246c1033d0 2024-04-26T20:38:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.801+0000 7f2469e00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c1033d0 0x7f246c102d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f245c00eb90 tx=0x7f245c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:38:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.801+0000 7f2462a00640 1 -- 172.21.15.131:0/1787161240 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c004d20 con 0x7f246c1033d0 2024-04-26T20:38:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.801+0000 7f2462a00640 1 -- 172.21.15.131:0/1787161240 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c00ee90 con 0x7f246c1033d0 2024-04-26T20:38:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.801+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c1a2cf0 con 0x7f246c1033d0 2024-04-26T20:38:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.801+0000 7f2462a00640 1 -- 172.21.15.131:0/1787161240 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c010670 con 0x7f246c1033d0 2024-04-26T20:38:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.801+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c1a31c0 con 0x7f246c1033d0 2024-04-26T20:38:03.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.802+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f246c069750 con 0x7f246c1033d0 2024-04-26T20:38:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.804+0000 7f2462a00640 1 -- 172.21.15.131:0/1787161240 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f245c010870 con 0x7f246c1033d0 2024-04-26T20:38:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.806+0000 7f2462a00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24500a3c40 0x7f24500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:38:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.806+0000 7f2462a00640 1 -- 172.21.15.131:0/1787161240 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8324+0+0 (secure 0 0 0) 0x7f245c0d0dd0 con 0x7f246c1033d0 2024-04-26T20:38:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.806+0000 7f2469400640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24500a3c40 0x7f24500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:38:03.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.807+0000 7f2469400640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24500a3c40 0x7f24500a6100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2458009230 tx=0x7f24580314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:38:03.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:02.808+0000 7f2462a00640 1 -- 172.21.15.131:0/1787161240 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f245c09d590 con 0x7f246c1033d0 2024-04-26T20:38:03.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.016+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f246c0fe8b0 con 0x7f246c1033d0 2024-04-26T20:38:03.683 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:03 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/564548564' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:03.691 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:03 smithi148 ceph-mon[32281]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T20:38:03.739 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:03 smithi148 ceph-mon[32281]: pgmap v1254: 33 pgs: 33 active+clean; 577 KiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:03.746 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:03 smithi148 ceph-mon[32281]: from='client.? 172.21.15.131:0/1787161240' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:03.755 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:03 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:03.763 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:03 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/564548564' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:03.772 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:03 smithi157 ceph-mon[32703]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T20:38:03.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:03 smithi157 ceph-mon[32703]: pgmap v1254: 33 pgs: 33 active+clean; 577 KiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:03.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:03 smithi157 ceph-mon[32703]: from='client.? 172.21.15.131:0/1787161240' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:03.796 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:03 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:03.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.162+0000 7f2462a00640 1 -- 172.21.15.131:0/1787161240 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v101) ==== 178+0+0 (secure 0 0 0) 0x7f245c09ce90 con 0x7f246c1033d0 2024-04-26T20:38:03.834 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' removed 2024-04-26T20:38:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24500a3c40 msgr2=0x7f24500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24500a3c40 0x7f24500a6100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2458009230 tx=0x7f24580314a0 comp rx=0 tx=0).stop 2024-04-26T20:38:03.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c1033d0 msgr2=0x7f246c102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:38:03.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c1033d0 0x7f246c102d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f245c00eb90 tx=0x7f245c00c6a0 comp rx=0 tx=0).stop 2024-04-26T20:38:03.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 shutdown_connections 2024-04-26T20:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f246c108630 0x7f246c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.131:6800/2537375548,v1:172.21.15.131:6801/2537375548] conn(0x7f24500a3c40 0x7f24500a6100 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:04.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f246c105d00 0x7f246c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:04.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 --2- 172.21.15.131:0/1787161240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246c1033d0 0x7f246c102d40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:38:04.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 >> 172.21.15.131:0/1787161240 conn(0x7f246c0fcdc0 msgr2=0x7f246c105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:38:04.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 shutdown_connections 2024-04-26T20:38:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-26T20:38:03.169+0000 7f2470a00640 1 -- 172.21.15.131:0/1787161240 wait complete. 2024-04-26T20:38:04.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:38:04.117 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:03 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/564548564' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:04.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:03 smithi131 ceph-mon[27919]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T20:38:04.134 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:03 smithi131 ceph-mon[27919]: pgmap v1254: 33 pgs: 33 active+clean; 577 KiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:04.166 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:03 smithi131 ceph-mon[27919]: from='client.? 172.21.15.131:0/1787161240' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:04.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:03 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T20:38:04.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:38:04.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:38:04.449 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:04 smithi157 ceph-mon[32703]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:04.460 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:04 smithi157 ceph-mon[32703]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T20:38:04.470 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T20:38:04.479 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T20:38:04.533 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:04 smithi131 ceph-mon[27919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:04.542 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:04 smithi131 ceph-mon[27919]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T20:38:04.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:04 smithi148 ceph-mon[32281]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T20:38:04.576 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:04 smithi148 ceph-mon[32281]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T20:38:04.609 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T20:38:04.636 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T20:38:04.663 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T20:38:04.673 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T20:38:04.681 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a 2024-04-26T20:38:05.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:04 smithi131 systemd[1]: Stopping Ceph mon.a for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:05.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:05 smithi131 ceph-mon[27919]: pgmap v1256: 1 pgs: 1 active+clean; 577 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:05.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a[27915]: 2024-04-26T20:38:05.348+0000 7fdfd0c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:05.473 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a[27915]: 2024-04-26T20:38:05.348+0000 7fdfd0c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T20:38:05.519 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:05 smithi157 ceph-mon[32703]: pgmap v1256: 1 pgs: 1 active+clean; 577 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:05.566 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:05 smithi148 ceph-mon[32281]: pgmap v1256: 1 pgs: 1 active+clean; 577 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T20:38:05.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:05 smithi131 podman[96531]: 2024-04-26 20:38:05.45309795 +0000 UTC m=+0.257299843 container died fc567c89abfaf9b6235ff933556da3adaa11a5bc6eb86eece13c9c822b48b8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:38:06.237 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:05 smithi131 podman[96531]: 2024-04-26 20:38:05.943273005 +0000 UTC m=+0.747474891 container cleanup fc567c89abfaf9b6235ff933556da3adaa11a5bc6eb86eece13c9c822b48b8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:38:06.256 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:05 smithi131 bash[96531]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a 2024-04-26T20:38:06.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Apr 26 20:38:06 smithi131 podman[96545]: 2024-04-26 20:38:06.236406798 +0000 UTC m=+0.767529486 container remove fc567c89abfaf9b6235ff933556da3adaa11a5bc6eb86eece13c9c822b48b8c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T20:38:06.759 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T20:38:06.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:06.945 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T20:38:06.954 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T20:38:06.962 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.b 2024-04-26T20:38:07.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:07 smithi148 systemd[1]: Stopping Ceph mon.b for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:07.817 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-b[32277]: 2024-04-26T20:38:07.476+0000 7f0d0ed6e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:07.824 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-b[32277]: 2024-04-26T20:38:07.476+0000 7f0d0ed6e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T20:38:07.834 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:07 smithi148 podman[72547]: 2024-04-26 20:38:07.71232411 +0000 UTC m=+0.542337173 container died 7a344342510058799cbe017d092e202e163e42f9ebdbe3e800c4c87e8002b0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T20:38:08.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:08 smithi148 podman[72547]: 2024-04-26 20:38:08.144621254 +0000 UTC m=+0.974634313 container cleanup 7a344342510058799cbe017d092e202e163e42f9ebdbe3e800c4c87e8002b0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T20:38:08.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:08 smithi148 bash[72547]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-b 2024-04-26T20:38:08.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 26 20:38:08 smithi148 podman[72560]: 2024-04-26 20:38:08.37134076 +0000 UTC m=+0.657847560 container remove 7a344342510058799cbe017d092e202e163e42f9ebdbe3e800c4c87e8002b0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T20:38:08.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T20:38:08.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:08.752 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T20:38:08.760 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T20:38:08.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.c 2024-04-26T20:38:09.082 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:08 smithi157 systemd[1]: Stopping Ceph mon.c for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:09.391 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-c[32688]: 2024-04-26T20:38:09.081+0000 7fbf5f463700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:09.399 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-c[32688]: 2024-04-26T20:38:09.081+0000 7fbf5f463700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T20:38:09.406 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:09 smithi157 podman[73522]: 2024-04-26 20:38:09.102097445 +0000 UTC m=+0.130916612 container died 1d4e58156150c30dd66fad1debc52b1ac129f26dc024b42b06566527f101a443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T20:38:09.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:09 smithi157 podman[73522]: 2024-04-26 20:38:09.490746127 +0000 UTC m=+0.519565300 container cleanup 1d4e58156150c30dd66fad1debc52b1ac129f26dc024b42b06566527f101a443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T20:38:09.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 26 20:38:09 smithi157 bash[73522]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mon-c 2024-04-26T20:38:09.919 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T20:38:09.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:10.054 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T20:38:10.078 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T20:38:10.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.x 2024-04-26T20:38:10.506 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:38:10 smithi131 systemd[1]: Stopping Ceph mgr.x for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:11.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:38:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:38:10.726+0000 7f63fa200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:11.015 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:38:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x[28147]: 2024-04-26T20:38:10.726+0000 7f63fa200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T20:38:11.024 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:38:10 smithi131 podman[96651]: 2024-04-26 20:38:10.82102993 +0000 UTC m=+0.300510970 container died c98006765fd693dd1d58feb06380dd486d24644f57b108ee9b2170fd1b88999b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T20:38:11.506 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:38:11 smithi131 podman[96651]: 2024-04-26 20:38:11.340545904 +0000 UTC m=+0.820026972 container cleanup c98006765fd693dd1d58feb06380dd486d24644f57b108ee9b2170fd1b88999b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T20:38:11.514 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:38:11 smithi131 bash[96651]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x 2024-04-26T20:38:12.006 INFO:journalctl@ceph.mgr.x.smithi131.stdout:Apr 26 20:38:11 smithi131 podman[96665]: 2024-04-26 20:38:11.601051647 +0000 UTC m=+0.768254275 container remove c98006765fd693dd1d58feb06380dd486d24644f57b108ee9b2170fd1b88999b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T20:38:12.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T20:38:12.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:12.223 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T20:38:12.233 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T20:38:12.242 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.y 2024-04-26T20:38:12.687 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:38:12 smithi148 systemd[1]: Stopping Ceph mgr.y for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:12.967 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:38:12 smithi148 podman[72680]: 2024-04-26 20:38:12.687117387 +0000 UTC m=+0.189700991 container died 4409d5162bb4dbf87b4bd955035529e48f39ceb47869b6d4f065a83388e3413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:38:13.316 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:38:13 smithi148 podman[72680]: 2024-04-26 20:38:13.041814371 +0000 UTC m=+0.544397974 container cleanup 4409d5162bb4dbf87b4bd955035529e48f39ceb47869b6d4f065a83388e3413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:38:13.327 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 26 20:38:13 smithi148 bash[72680]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-y 2024-04-26T20:38:13.507 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T20:38:13.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:13.556 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T20:38:13.647 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T20:38:13.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.z 2024-04-26T20:38:13.980 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:38:13 smithi157 systemd[1]: Stopping Ceph mgr.z for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:14.364 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:38:14 smithi157 podman[73644]: 2024-04-26 20:38:14.066829909 +0000 UTC m=+0.207695481 container died 8e82ed36c14dba33ec661c1e1887293ddd2cb92c70423ce981662cb3a3d7dd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T20:38:14.730 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:38:14 smithi157 podman[73644]: 2024-04-26 20:38:14.463112343 +0000 UTC m=+0.603977915 container cleanup 8e82ed36c14dba33ec661c1e1887293ddd2cb92c70423ce981662cb3a3d7dd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:38:14.738 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 26 20:38:14 smithi157 bash[73644]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mgr-z 2024-04-26T20:38:14.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T20:38:14.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:14.940 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T20:38:15.007 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T20:38:15.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.0 2024-04-26T20:38:15.506 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:15 smithi131 systemd[1]: Stopping Ceph osd.0 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:16.007 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0[42289]: 2024-04-26T20:38:15.660+0000 7f5878200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:16.020 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0[42289]: 2024-04-26T20:38:15.660+0000 7f5878200700 -1 osd.0 101 *** Got signal Terminated *** 2024-04-26T20:38:16.029 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0[42289]: 2024-04-26T20:38:15.660+0000 7f5878200700 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:38:21.256 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:21 smithi131 podman[96772]: 2024-04-26 20:38:21.04456839 +0000 UTC m=+5.545393991 container died 002caef914eae2b6a773acc4d61e2dc957616f11d946f64375bd0135428a64df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T20:38:21.756 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:21 smithi131 podman[96772]: 2024-04-26 20:38:21.537801053 +0000 UTC m=+6.038626630 container cleanup 002caef914eae2b6a773acc4d61e2dc957616f11d946f64375bd0135428a64df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:38:21.764 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:21 smithi131 bash[96772]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0 2024-04-26T20:38:22.256 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:21 smithi131 podman[96801]: 2024-04-26 20:38:21.814242565 +0000 UTC m=+0.753747432 container remove 002caef914eae2b6a773acc4d61e2dc957616f11d946f64375bd0135428a64df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T20:38:23.004 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:22 smithi131 podman[96897]: 2024-04-26 20:38:22.54218482 +0000 UTC m=+0.032755401 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:38:23.011 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:22 smithi131 podman[96897]: 2024-04-26 20:38:22.675363601 +0000 UTC m=+0.165934219 container create c32b2d2eb655b1738f99e593d2ba4c15beee81a4eb0f165addd48e0773711236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T20:38:23.256 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:23 smithi131 podman[96897]: 2024-04-26 20:38:23.002818479 +0000 UTC m=+0.493389104 container init c32b2d2eb655b1738f99e593d2ba4c15beee81a4eb0f165addd48e0773711236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T20:38:23.306 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:23 smithi131 podman[96897]: 2024-04-26 20:38:23.014615622 +0000 UTC m=+0.505186263 container start c32b2d2eb655b1738f99e593d2ba4c15beee81a4eb0f165addd48e0773711236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T20:38:23.314 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:23 smithi131 podman[96897]: 2024-04-26 20:38:23.127075931 +0000 UTC m=+0.617646586 container attach c32b2d2eb655b1738f99e593d2ba4c15beee81a4eb0f165addd48e0773711236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T20:38:23.756 INFO:journalctl@ceph.osd.0.smithi131.stdout:Apr 26 20:38:23 smithi131 podman[96923]: 2024-04-26 20:38:23.51602934 +0000 UTC m=+0.039938788 container died c32b2d2eb655b1738f99e593d2ba4c15beee81a4eb0f165addd48e0773711236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T20:38:24.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T20:38:24.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:24.506 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T20:38:24.515 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T20:38:24.524 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.1 2024-04-26T20:38:24.816 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:24 smithi148 systemd[1]: Stopping Ceph osd.1 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:25.316 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1[36919]: 2024-04-26T20:38:24.855+0000 7f623f31a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:25.335 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1[36919]: 2024-04-26T20:38:24.855+0000 7f623f31a700 -1 osd.1 101 *** Got signal Terminated *** 2024-04-26T20:38:25.342 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1[36919]: 2024-04-26T20:38:24.855+0000 7f623f31a700 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:38:30.489 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:30 smithi148 podman[72804]: 2024-04-26 20:38:30.193331546 +0000 UTC m=+5.469802315 container died d85ff1de9c7041393d5e79f66fb30398f21b6d3e6bf2aebe2751de86bbd1f9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T20:38:30.816 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:30 smithi148 podman[72804]: 2024-04-26 20:38:30.571512631 +0000 UTC m=+5.847983401 container cleanup d85ff1de9c7041393d5e79f66fb30398f21b6d3e6bf2aebe2751de86bbd1f9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T20:38:30.825 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:30 smithi148 bash[72804]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1 2024-04-26T20:38:31.192 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:30 smithi148 podman[72822]: 2024-04-26 20:38:30.847934515 +0000 UTC m=+0.651384901 container remove d85ff1de9c7041393d5e79f66fb30398f21b6d3e6bf2aebe2751de86bbd1f9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T20:38:31.468 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:31 smithi148 podman[72909]: 2024-04-26 20:38:31.093917128 +0000 UTC m=+0.018063260 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:38:31.475 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:31 smithi148 podman[72909]: 2024-04-26 20:38:31.20182606 +0000 UTC m=+0.125972189 container create 2f7b2ccfd5aff1c2ec5cf0295fb80df14a4035cc625f11923e2aa72a4e770202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T20:38:31.777 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:31 smithi148 podman[72909]: 2024-04-26 20:38:31.468427253 +0000 UTC m=+0.392573381 container init 2f7b2ccfd5aff1c2ec5cf0295fb80df14a4035cc625f11923e2aa72a4e770202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T20:38:31.784 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:31 smithi148 podman[72909]: 2024-04-26 20:38:31.471659379 +0000 UTC m=+0.395805516 container start 2f7b2ccfd5aff1c2ec5cf0295fb80df14a4035cc625f11923e2aa72a4e770202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:38:31.794 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:31 smithi148 podman[72909]: 2024-04-26 20:38:31.56015396 +0000 UTC m=+0.484300117 container attach 2f7b2ccfd5aff1c2ec5cf0295fb80df14a4035cc625f11923e2aa72a4e770202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:38:32.067 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:31 smithi148 podman[72909]: 2024-04-26 20:38:31.777765515 +0000 UTC m=+0.701911648 container died 2f7b2ccfd5aff1c2ec5cf0295fb80df14a4035cc625f11923e2aa72a4e770202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:38:32.408 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 26 20:38:32 smithi148 podman[72935]: 2024-04-26 20:38:32.389417525 +0000 UTC m=+0.605906628 container remove 2f7b2ccfd5aff1c2ec5cf0295fb80df14a4035cc625f11923e2aa72a4e770202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T20:38:32.419 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T20:38:32.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:32.474 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T20:38:32.483 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T20:38:32.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.2 2024-04-26T20:38:32.826 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:32 smithi157 systemd[1]: Stopping Ceph osd.2 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:33.230 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2[37867]: 2024-04-26T20:38:32.824+0000 7f4b721e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:33.248 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2[37867]: 2024-04-26T20:38:32.824+0000 7f4b721e0700 -1 osd.2 101 *** Got signal Terminated *** 2024-04-26T20:38:33.257 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2[37867]: 2024-04-26T20:38:32.824+0000 7f4b721e0700 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:38:38.442 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:38 smithi157 podman[73771]: 2024-04-26 20:38:38.14215155 +0000 UTC m=+5.447432340 container died bba2c7b9b37538c2208f757ca716c5b11d566b970af2eeef61fe60cd3659159f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T20:38:38.730 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:38 smithi157 podman[73771]: 2024-04-26 20:38:38.533320763 +0000 UTC m=+5.838601562 container cleanup bba2c7b9b37538c2208f757ca716c5b11d566b970af2eeef61fe60cd3659159f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T20:38:38.740 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:38 smithi157 bash[73771]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2 2024-04-26T20:38:39.146 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:38 smithi157 podman[73789]: 2024-04-26 20:38:38.81841253 +0000 UTC m=+0.665201829 container remove bba2c7b9b37538c2208f757ca716c5b11d566b970af2eeef61fe60cd3659159f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T20:38:39.439 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:39 smithi157 podman[73878]: 2024-04-26 20:38:39.049267817 +0000 UTC m=+0.017451979 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:38:39.493 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:39 smithi157 podman[73878]: 2024-04-26 20:38:39.169519109 +0000 UTC m=+0.137703300 container create 58c0be6e1789294405511b28f0054f9c5f54ecfc2d145e57d612066368973ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T20:38:39.730 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:39 smithi157 podman[73878]: 2024-04-26 20:38:39.439128709 +0000 UTC m=+0.407312867 container init 58c0be6e1789294405511b28f0054f9c5f54ecfc2d145e57d612066368973ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:38:39.846 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:39 smithi157 podman[73878]: 2024-04-26 20:38:39.44316458 +0000 UTC m=+0.411348735 container start 58c0be6e1789294405511b28f0054f9c5f54ecfc2d145e57d612066368973ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T20:38:39.854 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:39 smithi157 podman[73878]: 2024-04-26 20:38:39.547370668 +0000 UTC m=+0.515554820 container attach 58c0be6e1789294405511b28f0054f9c5f54ecfc2d145e57d612066368973ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T20:38:40.109 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 26 20:38:39 smithi157 podman[73878]: 2024-04-26 20:38:39.741862817 +0000 UTC m=+0.710046969 container died 58c0be6e1789294405511b28f0054f9c5f54ecfc2d145e57d612066368973ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T20:38:40.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T20:38:40.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:40.508 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T20:38:40.517 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T20:38:40.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.3 2024-04-26T20:38:41.006 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:40 smithi131 systemd[1]: Stopping Ceph osd.3 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:41.506 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:41 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:38:41.098+0000 7f1646000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:41.522 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:41 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:38:41.098+0000 7f1646000700 -1 osd.3 101 *** Got signal Terminated *** 2024-04-26T20:38:41.531 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:41 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:38:41.098+0000 7f1646000700 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:38:41.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:41.436+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:42.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:41 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:41.683+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:42.013 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:41 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:41.624+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:42.605 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:42 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:38:42.301+0000 7f163c800700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.993741+0000 front 2024-04-26T20:38:18.993562+0000 (oldest deadline 2024-04-26T20:38:41.893349+0000) 2024-04-26T20:38:42.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:42.405+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:43.006 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:42 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:42.604+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:43.037 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:42 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:42.672+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:43.641 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:43 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:38:43.312+0000 7f163c800700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.993741+0000 front 2024-04-26T20:38:18.993562+0000 (oldest deadline 2024-04-26T20:38:41.893349+0000) 2024-04-26T20:38:43.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:43.419+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:44.006 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:43 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:43.641+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:44.074 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:43 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:43.647+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:44.628 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:38:44.280+0000 7f163c800700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.993741+0000 front 2024-04-26T20:38:18.993562+0000 (oldest deadline 2024-04-26T20:38:41.893349+0000) 2024-04-26T20:38:44.708 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:44.293+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:44.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:44.379+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:44.980 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:44.921+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:44.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:44.707+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:45.006 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:44.653+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:45.015 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:44 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:44.627+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:45.583 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:45 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3[49101]: 2024-04-26T20:38:45.329+0000 7f163c800700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.993741+0000 front 2024-04-26T20:38:18.993562+0000 (oldest deadline 2024-04-26T20:38:41.893349+0000) 2024-04-26T20:38:45.727 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:45.312+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:45.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:45.330+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:45.980 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:45.964+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:46.063 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:45.726+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:46.072 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:45 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:45.617+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:46.080 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:45 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:45.582+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:46.630 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:46 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:46.611+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:46.688 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:46.285+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:46.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:46.316+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:46.982 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:46.688+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:46.991 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:46.926+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:47.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:46 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:46.611+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:47.015 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:46 smithi131 podman[97003]: 2024-04-26 20:38:46.652637067 +0000 UTC m=+5.687343656 container died c24ee85052d2ca776ef1951d73ee418ee668bb109a97f43127bfc267e028c986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T20:38:47.411 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:47 smithi131 podman[97003]: 2024-04-26 20:38:47.209002723 +0000 UTC m=+6.243709349 container cleanup c24ee85052d2ca776ef1951d73ee418ee668bb109a97f43127bfc267e028c986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T20:38:47.420 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:47 smithi131 bash[97003]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3 2024-04-26T20:38:47.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:47 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:47.273+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:47.647 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:47.331+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:47.756 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:47 smithi131 podman[97026]: 2024-04-26 20:38:47.410237275 +0000 UTC m=+0.746838799 container remove c24ee85052d2ca776ef1951d73ee418ee668bb109a97f43127bfc267e028c986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T20:38:47.768 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:47 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:47.602+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:47.776 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:47 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:47.619+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:47.951 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:47.646+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:48.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:47.950+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:48.507 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:48 smithi131 podman[97116]: 2024-04-26 20:38:48.065419695 +0000 UTC m=+0.034939682 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:38:48.521 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:48 smithi131 podman[97116]: 2024-04-26 20:38:48.211694689 +0000 UTC m=+0.181214700 container create b3d6390b171e036747d1205776db8e799d7a6b05aa9625cbc199aae71134223a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:38:48.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:48 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:48.301+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:48.629 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:48.348+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:48.935 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:48.628+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:48.979 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:48 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:48.556+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:48.989 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:48 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:48.583+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:48.996 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:48 smithi131 podman[97116]: 2024-04-26 20:38:48.532813867 +0000 UTC m=+0.502333853 container init b3d6390b171e036747d1205776db8e799d7a6b05aa9625cbc199aae71134223a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T20:38:49.005 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:48 smithi131 podman[97116]: 2024-04-26 20:38:48.545302124 +0000 UTC m=+0.514822105 container start b3d6390b171e036747d1205776db8e799d7a6b05aa9625cbc199aae71134223a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T20:38:49.014 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:48 smithi131 podman[97116]: 2024-04-26 20:38:48.639327134 +0000 UTC m=+0.608847152 container attach b3d6390b171e036747d1205776db8e799d7a6b05aa9625cbc199aae71134223a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:38:49.023 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:48 smithi131 podman[97116]: 2024-04-26 20:38:48.978773417 +0000 UTC m=+0.948293484 container died b3d6390b171e036747d1205776db8e799d7a6b05aa9625cbc199aae71134223a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:38:49.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:48.934+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:49.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:49 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:49.278+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:49.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:49.382+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:49.737 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:49.621+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:49.819 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:49 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:49.591+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:49.826 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:49 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:49.562+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:49.833 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T20:38:50.006 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:49 smithi131 podman[97142]: 2024-04-26 20:38:49.734813488 +0000 UTC m=+0.740445012 container remove b3d6390b171e036747d1205776db8e799d7a6b05aa9625cbc199aae71134223a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:38:50.013 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:49 smithi131 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-26T20:38:50.019 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:49 smithi131 systemd[1]: Stopped Ceph osd.3 for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:38:50.027 INFO:journalctl@ceph.osd.3.smithi131.stdout:Apr 26 20:38:49 smithi131 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.3.service: Consumed 59.339s CPU time. 2024-04-26T20:38:50.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:50.096 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T20:38:50.103 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T20:38:50.110 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.4 2024-04-26T20:38:50.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:49.966+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:50.442 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:50 smithi148 systemd[1]: Stopping Ceph osd.4 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:50.449 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:50 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:50.281+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:50.655 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:50.390+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:50.816 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:50 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:38:50.442+0000 7fa59a288700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:50.823 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:50 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:38:50.442+0000 7fa59a288700 -1 osd.4 101 *** Got signal Terminated *** 2024-04-26T20:38:50.918 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:50 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:38:50.442+0000 7fa59a288700 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:38:50.938 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:50.653+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:51.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:50 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:50.561+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:51.053 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:50 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:50.600+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:51.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:50.937+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:51.236 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:50.937+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:51.566 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:51 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:38:51.447+0000 7fa59687c700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:25.113315+0000 front 2024-04-26T20:38:25.113198+0000 (oldest deadline 2024-04-26T20:38:51.013361+0000) 2024-04-26T20:38:51.577 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:51 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:51.282+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:51.690 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:51.424+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:51.698 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:51.424+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:51.980 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:51.689+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:51.990 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:51.928+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:51.999 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:51.928+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:52.008 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:51 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:51.581+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:52.058 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:51 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:51.533+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:52.566 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:52 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:38:52.408+0000 7fa59687c700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:25.113315+0000 front 2024-04-26T20:38:25.113198+0000 (oldest deadline 2024-04-26T20:38:51.013361+0000) 2024-04-26T20:38:52.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:52 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:52.303+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:52.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:52.398+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:52.739 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:52.398+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:53.006 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:52 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:52.540+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:53.050 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:52 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:52.543+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:53.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:52 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:52.745+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:38:53.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:52.737+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:53.240 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:52.954+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:53.325 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:52.954+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:53.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:53.440+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:53.738 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:53.440+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:53.778 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:53 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:38:53.432+0000 7fa59687c700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:25.113315+0000 front 2024-04-26T20:38:25.113198+0000 (oldest deadline 2024-04-26T20:38:51.013361+0000) 2024-04-26T20:38:53.787 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:53 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:53.332+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:54.006 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:53 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:53.574+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:54.019 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:53 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:53.526+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:54.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:53 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:53.777+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:38:54.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:53.777+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:54.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:53.995+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:54.292 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:53.995+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:54.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:54.437+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:54.736 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:54.437+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:54.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:54 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:54.479+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:54.803 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:54 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:54.562+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:54.810 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:54 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:54.562+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:38:54.817 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:54 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4[43873]: 2024-04-26T20:38:54.409+0000 7fa59687c700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:25.113315+0000 front 2024-04-26T20:38:25.113198+0000 (oldest deadline 2024-04-26T20:38:51.013361+0000) 2024-04-26T20:38:54.899 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:54 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:54.364+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:54.908 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:54 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:54.364+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:38:55.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:54 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:54.798+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:38:55.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:54.765+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:55.238 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:54.993+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:55.244 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:54.993+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:55.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:55.421+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:55.750 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:55.421+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:55.814 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:55.412+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:55.821 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:55.412+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:38:56.006 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:55.514+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:56.012 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:55.514+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:38:56.020 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:55.505+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:56.026 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:55 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:55.505+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:38:56.112 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:55 smithi148 podman[73027]: 2024-04-26 20:38:55.966951348 +0000 UTC m=+5.657020132 container died 9a82d73b1efcb9de8bf0c6734c7a28f343f24dc23b4427873ef27b71da1d0ced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T20:38:56.119 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:55 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:55.752+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:38:56.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:55.799+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:56.239 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:55.974+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:56.248 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:55 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:55.974+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:56.526 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:56 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:56.375+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:56.537 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:56 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:56.375+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:38:56.546 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:56 smithi148 podman[73027]: 2024-04-26 20:38:56.358579057 +0000 UTC m=+6.048647836 container cleanup 9a82d73b1efcb9de8bf0c6734c7a28f343f24dc23b4427873ef27b71da1d0ced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:38:56.554 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:56 smithi148 bash[73027]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4 2024-04-26T20:38:56.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:56 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:56.437+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:56.736 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:56 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:56.437+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:56.778 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:56 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:56.489+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:56.807 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:56 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:56.489+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:38:56.865 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:56 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:56.497+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:56.875 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:56 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:56.497+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:38:56.884 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:56 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:56.762+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:38:56.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:56 smithi148 podman[73044]: 2024-04-26 20:38:56.526677142 +0000 UTC m=+0.555964720 container remove 9a82d73b1efcb9de8bf0c6734c7a28f343f24dc23b4427873ef27b71da1d0ced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T20:38:57.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:56 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:56.779+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:57.239 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:56 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:56.932+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:57.246 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:56 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:56.932+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:57.315 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:56 smithi148 podman[73134]: 2024-04-26 20:38:56.87472966 +0000 UTC m=+0.017992093 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:38:57.322 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:56 smithi148 podman[73134]: 2024-04-26 20:38:56.994222472 +0000 UTC m=+0.137484884 container create 4fc8ce3b5854a41df0bd641e8dbb628b15e8048cbeca41dc9a1289d31825ea24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T20:38:57.329 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:57 smithi148 podman[73134]: 2024-04-26 20:38:57.21355019 +0000 UTC m=+0.356812603 container init 4fc8ce3b5854a41df0bd641e8dbb628b15e8048cbeca41dc9a1289d31825ea24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:38:57.336 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:57 smithi148 podman[73134]: 2024-04-26 20:38:57.217224097 +0000 UTC m=+0.360486511 container start 4fc8ce3b5854a41df0bd641e8dbb628b15e8048cbeca41dc9a1289d31825ea24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T20:38:57.343 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:57 smithi148 podman[73134]: 2024-04-26 20:38:57.313554602 +0000 UTC m=+0.456817022 container attach 4fc8ce3b5854a41df0bd641e8dbb628b15e8048cbeca41dc9a1289d31825ea24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T20:38:57.567 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 26 20:38:57 smithi148 podman[73134]: 2024-04-26 20:38:57.504611026 +0000 UTC m=+0.647873445 container died 4fc8ce3b5854a41df0bd641e8dbb628b15e8048cbeca41dc9a1289d31825ea24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T20:38:57.574 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:57 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:57.341+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:57.635 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:57 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:57.341+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:38:57.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:57 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:57.415+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:57.739 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:57 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:57.415+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:57.802 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:57 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:57.488+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:57.817 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:57 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:57.488+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:38:57.826 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:57 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:57.532+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:57.890 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:57 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:57.532+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:38:58.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:57 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:57.765+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:38:58.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T20:38:58.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:38:58.189 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T20:38:58.197 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T20:38:58.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.5 2024-04-26T20:38:58.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:57 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:57.735+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:58.239 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:57 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:57.890+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:58.247 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:57 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:57.890+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:58.538 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:58 smithi157 systemd[1]: Stopping Ceph osd.5 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:38:58.554 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:58.372+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:58.596 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:58.372+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:58.606 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:58.536+0000 7fda65c78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:38:58.674 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:58.536+0000 7fda65c78700 -1 osd.5 101 *** Got signal Terminated *** 2024-04-26T20:38:58.682 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:58.536+0000 7fda65c78700 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:38:58.691 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:58 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:58.318+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:58.751 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:58 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:58.318+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:38:58.760 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:58 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:58.545+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:58.769 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:58 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:58.545+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:38:58.778 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:58 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:58.460+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:58.830 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:58 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:58.460+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:38:58.838 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:58 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:58.460+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:38:58.925 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:58.775+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:38:59.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:58 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:58.795+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:38:59.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:58.924+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:38:59.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:58 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:58.924+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:38:59.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:59 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:59.340+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:38:59.739 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:38:59 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:38:59.340+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:38:59.805 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:59 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:59.352+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:38:59.814 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:38:59 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:38:59.352+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:38:59.916 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:59 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:59.588+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:38:59.924 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:38:59 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:38:59.588+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:38:59.932 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:59 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:59.441+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:38:59.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:59 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:59.441+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:38:59.969 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:38:59 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:38:59.441+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:00.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:38:59 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:38:59.767+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:00.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:59 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:59.762+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:00.238 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:38:59 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:38:59.762+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:00.247 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:59 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:59.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:00.257 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:59 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:59.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:00.265 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:38:59 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:38:59.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:00.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:00 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:00.307+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:00.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:00 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:00.307+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:00.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:00.368+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:39:00.737 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:00.368+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:39:00.779 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:00.368+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:35.886537+0000 front 2024-04-26T20:38:35.886565+0000 (oldest deadline 2024-04-26T20:38:59.986762+0000) 2024-04-26T20:39:00.787 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:00.627+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:00.870 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:00.627+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:00.877 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:00.479+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:00.885 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:00.479+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:00.959 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:00 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:00.479+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:01.067 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:00 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:00.759+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:01.074 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:00 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:00.759+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:01.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:00.767+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:01.238 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:00.767+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:01.246 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:00.912+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:01.255 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:00.912+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:01.264 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:00 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:00.912+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:01.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:01.395+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:39:01.737 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:01.395+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:39:01.746 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:01.395+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:35.886537+0000 front 2024-04-26T20:38:35.886565+0000 (oldest deadline 2024-04-26T20:38:59.986762+0000) 2024-04-26T20:39:01.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:01.451+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:01.832 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:01.451+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:01.840 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:01.451+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:01.922 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:01.660+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:01.931 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:01.660+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:01.939 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:01.660+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:01.970 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:01 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:01.349+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:01.978 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:01 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:01.349+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:02.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:01 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:01.805+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:02.074 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:01 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:01.805+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:02.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:01.763+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:02.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:01.763+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:02.244 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:01.892+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:02.251 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:01.892+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:02.259 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:01 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:01.892+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:02.689 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:02 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:02.430+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:02.696 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:02 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:02.430+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:02.759 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:02 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:02.430+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:02.812 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:02.411+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:39:02.822 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:02.411+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:39:02.925 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:02.411+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:35.886537+0000 front 2024-04-26T20:38:35.886565+0000 (oldest deadline 2024-04-26T20:38:59.986762+0000) 2024-04-26T20:39:02.933 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:02 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:02.390+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:02.940 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:02 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:02.390+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:02.999 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:02.722+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:03.006 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:02.722+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:03.014 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:02.914+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:03.021 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:02.914+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:03.077 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:02 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:02.914+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:03.086 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:02 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:02.688+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:03.095 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:02 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:02.688+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:03.104 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:02 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:02.688+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:03.316 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:02 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:02.829+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:03.322 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:02 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:02.829+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:03.730 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:03.445+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.384494+0000 front 2024-04-26T20:38:18.384172+0000 (oldest deadline 2024-04-26T20:38:44.284015+0000) 2024-04-26T20:39:03.740 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:03.445+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.885449+0000 front 2024-04-26T20:38:28.885494+0000 (oldest deadline 2024-04-26T20:38:51.185557+0000) 2024-04-26T20:39:03.794 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5[44825]: 2024-04-26T20:39:03.445+0000 7fda6226c700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:35.886537+0000 front 2024-04-26T20:38:35.886565+0000 (oldest deadline 2024-04-26T20:38:59.986762+0000) 2024-04-26T20:39:03.803 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:03 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:03.437+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:03.896 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:03 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:03.437+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:03.904 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:03 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:03.437+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:03.931 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:03 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:03.462+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:04.016 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:03 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:03.462+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:04.026 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:03 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:03.462+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:04.034 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:03 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:03.670+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:04.043 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:03 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:03.670+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:04.091 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:03 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:03.670+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:04.100 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:03.754+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:04.109 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:03.754+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:04.118 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:03 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:03.792+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:04.127 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:03 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:03.792+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:04.135 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:03.899+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:04.144 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:03.899+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:04.178 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:03 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:03.899+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:04.338 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:04 smithi157 podman[73998]: 2024-04-26 20:39:04.035911089 +0000 UTC m=+5.625278730 container died 5691987efdbd9c6b53cf8f91e7ca3643b648551b40395f5efb1fb919251a9a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T20:39:04.664 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:04 smithi157 podman[73998]: 2024-04-26 20:39:04.428691088 +0000 UTC m=+6.018058725 container cleanup 5691987efdbd9c6b53cf8f91e7ca3643b648551b40395f5efb1fb919251a9a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T20:39:04.691 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:04 smithi157 bash[73998]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5 2024-04-26T20:39:04.708 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:04.413+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:04.716 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:04.413+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:04.790 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:04.413+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:04.799 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:04 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:04.408+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:04.808 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:04 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:04.408+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:04.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:04 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:04.408+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:04.980 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:04 smithi157 podman[74015]: 2024-04-26 20:39:04.663352676 +0000 UTC m=+0.624208611 container remove 5691987efdbd9c6b53cf8f91e7ca3643b648551b40395f5efb1fb919251a9a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T20:39:04.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:04 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:04.764+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:04.995 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:04 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:04.764+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:05.003 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:04 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:04.907+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:05.011 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:04 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:04.907+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:05.019 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:04 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:04.907+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:05.027 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:04.707+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:05.035 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:04.707+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:05.103 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:04.707+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:05.112 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:04 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:04.758+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:05.208 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:04 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:04.758+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:05.259 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:05 smithi157 podman[74103]: 2024-04-26 20:39:04.90294691 +0000 UTC m=+0.018302028 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:39:05.267 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:05 smithi157 podman[74103]: 2024-04-26 20:39:05.022774967 +0000 UTC m=+0.138130071 container create 929a92581ef14c8359f0e379d8c1f2627d758f1b7bbbfe197d6df98f945c8356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:39:05.554 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:05 smithi157 podman[74103]: 2024-04-26 20:39:05.259227084 +0000 UTC m=+0.374582197 container init 929a92581ef14c8359f0e379d8c1f2627d758f1b7bbbfe197d6df98f945c8356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T20:39:05.563 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:05 smithi157 podman[74103]: 2024-04-26 20:39:05.26331421 +0000 UTC m=+0.378669315 container start 929a92581ef14c8359f0e379d8c1f2627d758f1b7bbbfe197d6df98f945c8356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T20:39:05.582 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:05 smithi157 podman[74103]: 2024-04-26 20:39:05.342173204 +0000 UTC m=+0.457528320 container attach 929a92581ef14c8359f0e379d8c1f2627d758f1b7bbbfe197d6df98f945c8356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T20:39:05.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:05.432+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:05.770 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:05.432+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:05.779 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:05.432+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:05.789 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:05.673+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:05.797 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:05.673+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:05.806 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:05 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:05.673+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:05.815 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:05 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:05.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:05.823 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:05 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:05.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:05.831 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:05 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:05.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:05.886 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 26 20:39:05 smithi157 podman[74103]: 2024-04-26 20:39:05.554006606 +0000 UTC m=+0.669361713 container died 929a92581ef14c8359f0e379d8c1f2627d758f1b7bbbfe197d6df98f945c8356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T20:39:05.895 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:05 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:05.776+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:05.965 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:05 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:05.776+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:06.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:05 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:05.774+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:06.073 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:05 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:05.774+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:06.133 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:05 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:05.934+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:06.142 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:05 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:05.934+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:06.151 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:05 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:05.934+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:06.161 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T20:39:06.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:39:06.259 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T20:39:06.266 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T20:39:06.273 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.6 2024-04-26T20:39:06.722 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:06 smithi131 systemd[1]: Stopping Ceph osd.6 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:39:06.729 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:06.464+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:06.736 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:06.464+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:06.797 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:06.464+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:06.844 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:06 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:06.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:06.851 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:06 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:06.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:06.947 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:06 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:06.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:07.007 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:06.721+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:07.014 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:06.721+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:07.044 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:06.721+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:07.052 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:06.878+0000 7f3b1f000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:39:07.061 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:06.878+0000 7f3b1f000700 -1 osd.6 101 *** Got signal Terminated *** 2024-04-26T20:39:07.069 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:06 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:06.878+0000 7f3b1f000700 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:39:07.141 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:06 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:06.797+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:07.148 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:06 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:06.797+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:07.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:06 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:06.768+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:07.238 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:06 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:06.768+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:07.246 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:06 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:06.951+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:07.281 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:06 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:06.951+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:07.290 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:06 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:06.951+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:07.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:07.499+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:07.765 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:07.499+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:07.773 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:07.499+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:07.846 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:07.695+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:07.855 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:07.695+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:07.864 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:07 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:07.695+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:07.937 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:07.384+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:07.944 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:07.384+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:07.953 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:07.384+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:08.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:07.813+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:08.080 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:07.813+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:08.087 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:07 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:07.813+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:08.132 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:07 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:07.781+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:08.140 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:07 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:07.781+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:08.148 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:07 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:07.781+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:08.158 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:07 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:07.940+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:08.167 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:07 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:07.940+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:08.175 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:07 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:07.940+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:08.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:08.502+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:08.763 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:08.502+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:08.845 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:08.502+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:08.852 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:08.681+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:08.859 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:08.681+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:08.931 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:08 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:08.681+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:08.938 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:08 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:08.368+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:08.945 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:08 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:08.368+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:08.952 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:08 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:08.368+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:09.067 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:08 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:08.815+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:09.074 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:08 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:08.815+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:09.080 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:08 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:08.815+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:09.145 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:08 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:08.799+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:09.154 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:08 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:08.799+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:09.163 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:08 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:08.799+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:09.172 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:08 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:08.924+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:09.228 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:08 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:08.924+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:09.284 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:08 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:08.924+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:09.757 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:09.482+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:09.770 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:09.482+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:09.778 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:09.482+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:09.787 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:09.482+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:09.856 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:09.728+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:09.864 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:09.728+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:09.873 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:09.728+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:09.882 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:09 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:09.358+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:09.955 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:09 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:09.358+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:09.963 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:09 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:09.358+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:09.972 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:09 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:09.358+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:10.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:09.812+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:10.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:09.812+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:10.251 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:09.812+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:10.259 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:09.967+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:10.308 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:09.967+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:10.367 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:09 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:09.967+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:10.375 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:09 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:09.852+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:10.477 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:09 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:09.852+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:10.485 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:09 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:09.852+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:10.811 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:10 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:10.327+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:10.818 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:10 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:10.327+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:10.826 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:10 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:10.327+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:10.834 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:10 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:10.327+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:11.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:10.512+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:11.027 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:10.512+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:11.036 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:10.512+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:11.044 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:10.512+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:11.052 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:10.700+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:11.061 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:10.700+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:11.069 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:10.700+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:11.077 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:10 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:10.700+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:45.971244+0000 front 2024-04-26T20:38:45.971209+0000 (oldest deadline 2024-04-26T20:39:10.670896+0000) 2024-04-26T20:39:11.086 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:10 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:10.810+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:11.160 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:10 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:10.810+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:11.169 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:10 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:10.810+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:11.244 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:10 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:10.788+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:11.253 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:10 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:10.788+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:11.261 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:10 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:10.788+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:11.312 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:10 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:10.958+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:11.323 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:10 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:10.958+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:11.333 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:10 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:10.958+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:11.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:11 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:11.314+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:11.576 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:11 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:11.314+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:11.585 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:11 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:11.314+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:11.596 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:11 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:11.314+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:11.747 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:11.476+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:11.753 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:11.476+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:11.762 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:11.476+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:11.770 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:11.476+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:12.006 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:11.745+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.759757+0000 front 2024-04-26T20:38:15.760366+0000 (oldest deadline 2024-04-26T20:38:41.059629+0000) 2024-04-26T20:39:12.014 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:11.745+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:29.262875+0000 front 2024-04-26T20:38:29.263001+0000 (oldest deadline 2024-04-26T20:38:54.563134+0000) 2024-04-26T20:39:12.021 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:11.745+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:37.366736+0000 front 2024-04-26T20:38:37.366573+0000 (oldest deadline 2024-04-26T20:39:00.767008+0000) 2024-04-26T20:39:12.096 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:11 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6[55644]: 2024-04-26T20:39:11.745+0000 7f3b1a800700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:45.971244+0000 front 2024-04-26T20:38:45.971209+0000 (oldest deadline 2024-04-26T20:39:10.670896+0000) 2024-04-26T20:39:12.109 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:11 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:11.796+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:12.116 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:11 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:11.796+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:12.145 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:11 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:11.796+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:12.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:11 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:11.838+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:12.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:11 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:11.838+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:12.243 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:11 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:11.838+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:12.323 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:11 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:11.981+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:12.330 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:11 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:11.981+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:12.338 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:11 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:11.981+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:12.601 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:12 smithi131 podman[97220]: 2024-04-26 20:39:12.287138378 +0000 UTC m=+5.595356038 container died 2c5d73af42d997030301fc5c5be2889d4c5a50b457349bb3a088f9d42493b6be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:39:12.608 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:12 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:12.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:12.618 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:12 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:12.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:12.627 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:12 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:12.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:12.635 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:12 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:12.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:12.758 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:12.340+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:12.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:12.340+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:12.821 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:12.340+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:12.830 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:12.340+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:12.973 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:12 smithi131 podman[97220]: 2024-04-26 20:39:12.745766865 +0000 UTC m=+6.053984572 container cleanup 2c5d73af42d997030301fc5c5be2889d4c5a50b457349bb3a088f9d42493b6be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T20:39:12.984 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:12 smithi131 bash[97220]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6 2024-04-26T20:39:13.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:12.757+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:13.074 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:12.757+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:13.083 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:12.757+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:13.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:12 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:12.808+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:13.238 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:12 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:12.808+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:13.244 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:12 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:12.808+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:13.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:13.004+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:13.261 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:13.004+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:13.270 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:13.004+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:13.279 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:12 smithi131 podman[97239]: 2024-04-26 20:39:12.971900385 +0000 UTC m=+0.670780768 container remove 2c5d73af42d997030301fc5c5be2889d4c5a50b457349bb3a088f9d42493b6be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T20:39:13.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:13 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:13.296+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:13.578 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:13 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:13.296+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:13.609 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:13 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:13.296+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:13.697 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:13 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:13.296+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:13.782 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:13 smithi131 podman[97324]: 2024-04-26 20:39:13.644520187 +0000 UTC m=+0.038623412 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:39:13.789 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:13 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:13.515+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:13.849 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:13 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:13.515+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:13.856 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:13 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:13.515+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:13.862 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:13 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:13.515+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:14.032 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:13 smithi131 podman[97324]: 2024-04-26 20:39:13.781072051 +0000 UTC m=+0.175175275 container create ea4046ec6e1998b4f41efe4b0333aa452b17476e9e956172c3b14076863fac39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:39:14.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:13 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:13.732+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:14.074 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:13 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:13.732+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:14.080 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:13 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:13.732+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:14.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:13.782+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:14.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:13.782+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:14.244 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:13.782+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:14.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:13.996+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:14.262 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:13.996+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:14.313 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:13 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:13.996+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:14.376 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:14 smithi131 podman[97324]: 2024-04-26 20:39:14.118743752 +0000 UTC m=+0.512847007 container init ea4046ec6e1998b4f41efe4b0333aa452b17476e9e956172c3b14076863fac39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T20:39:14.395 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:14 smithi131 podman[97324]: 2024-04-26 20:39:14.130440883 +0000 UTC m=+0.524544161 container start ea4046ec6e1998b4f41efe4b0333aa452b17476e9e956172c3b14076863fac39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T20:39:14.403 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:14 smithi131 podman[97324]: 2024-04-26 20:39:14.217153198 +0000 UTC m=+0.611256509 container attach ea4046ec6e1998b4f41efe4b0333aa452b17476e9e956172c3b14076863fac39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T20:39:14.695 INFO:journalctl@ceph.osd.6.smithi131.stdout:Apr 26 20:39:14 smithi131 podman[97324]: 2024-04-26 20:39:14.615561891 +0000 UTC m=+1.009665172 container died ea4046ec6e1998b4f41efe4b0333aa452b17476e9e956172c3b14076863fac39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T20:39:14.702 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:14 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:14.468+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:14.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:14 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:14.468+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:14.718 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:14 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:14.468+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:14.727 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:14 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:14.468+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:14.794 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:14 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:14.334+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:14.802 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:14 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:14.334+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:14.812 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:14 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:14.334+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:14.820 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:14 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:14.334+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:15.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:14 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:14.746+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:15.073 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:14 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:14.746+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:15.081 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:14 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:14.746+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:15.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:14 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:14.769+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:15.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:14 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:14.769+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:15.264 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:14 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:14.769+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:15.337 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:14 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:14.960+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:15.346 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:14 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:14.960+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:15.355 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:14 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:14.960+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:15.484 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T20:39:15.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:39:15.592 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T20:39:15.600 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T20:39:15.685 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.7 2024-04-26T20:39:15.720 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:15 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:15.322+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:15.726 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:15 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:15.322+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:15.734 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:15 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:15.322+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:15.742 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:15 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:15.322+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:15.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:15.508+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:15.770 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:15.508+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:15.827 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:15.508+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:15.837 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:15 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:15.508+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:16.020 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:15 smithi148 systemd[1]: Stopping Ceph osd.7 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:39:16.032 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:15 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:15.754+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:16.039 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:15 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:15.754+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:16.048 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:15 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:15.754+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:16.090 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:16.019+0000 7fefeabd7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:39:16.098 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:16.019+0000 7fefeabd7700 -1 osd.7 101 *** Got signal Terminated *** 2024-04-26T20:39:16.106 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:16.019+0000 7fefeabd7700 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:39:16.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:15 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:15.807+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:16.243 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:15 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:15.807+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:16.251 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:15 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:15.807+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:16.259 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:15 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:15.965+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:16.268 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:15 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:15.965+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:16.277 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:15 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:15.965+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:16.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:16.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:16.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:16.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:16.632 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:16.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:16.681 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:16.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:16.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:16 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:16.503+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:16.773 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:16 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:16.503+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:16.779 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:16 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:16.503+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:16.784 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:16 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:16.503+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:17.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:16.783+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:17.073 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:16.783+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:17.082 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:16.783+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:17.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:16 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:16.842+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:17.239 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:16 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:16.842+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:17.250 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:16 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:16.842+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:17.260 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:16 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:16.992+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:17.294 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:16 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:16.992+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:17.302 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:16 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:16.992+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:17.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:17 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:17.311+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:17.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:17 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:17.311+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:17.637 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:17 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:17.311+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:17.681 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:17 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:17.311+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:17.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:17 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:17.462+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:17.777 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:17 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:17.462+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:17.785 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:17 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:17.462+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:17.791 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:17 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:17.462+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:18.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:17.873+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:18.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:17.873+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:18.244 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:17.873+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:18.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:17.951+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:18.288 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:17.951+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:18.295 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:17 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:17.951+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:18.302 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:17 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:17.817+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:18.309 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:17 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:17.817+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:18.316 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:17 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:17.817+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:18.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:18.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:18.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:18.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:18.636 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:18.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:18.676 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:18.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:18.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:18.491+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:18.766 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:18.491+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:18.775 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:18.491+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:18.784 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:18 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:18.491+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:19.066 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:18.774+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:19.073 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:18.774+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:19.083 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:18.774+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:19.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:18 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:18.862+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:19.250 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:18 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:18.862+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:19.259 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:18 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:18.862+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:19.268 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:18 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:18.981+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:19.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:18 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:18.981+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:19.284 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:18 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:18.981+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:19.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:19 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:19.290+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:19.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:19 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:19.290+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:19.580 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:19 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:19.290+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:19.634 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:19 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:19.290+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:20.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:19 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:19.514+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:20.029 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:19 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:19.514+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:20.035 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:19 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:19.514+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:20.043 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:19 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:19.514+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:20.094 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:19 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:19.514+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:20.103 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:19 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:19.798+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:20.111 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:19 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:19.798+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:20.122 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:19 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:19.798+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:20.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:19.945+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:20.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:19.945+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:20.246 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:19.945+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:20.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:19.945+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:20.263 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:19.909+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:20.272 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:19.909+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:20.280 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:19.909+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:20.289 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:19 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:19.909+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:20.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:20.265+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:20.575 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:20.265+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:20.645 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:20.265+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:20.653 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:20.265+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:20.662 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:20.265+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:21.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:20.535+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:21.013 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:20.535+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:21.022 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:20.535+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:21.031 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:20.535+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:21.124 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:20 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:20.535+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:21.134 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:20.777+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.300385+0000 front 2024-04-26T20:38:27.300441+0000 (oldest deadline 2024-04-26T20:38:52.600577+0000) 2024-04-26T20:39:21.142 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:20.777+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.802105+0000 front 2024-04-26T20:38:34.802315+0000 (oldest deadline 2024-04-26T20:39:00.702097+0000) 2024-04-26T20:39:21.150 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:20.777+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:43.503826+0000 front 2024-04-26T20:38:43.503999+0000 (oldest deadline 2024-04-26T20:39:07.003542+0000) 2024-04-26T20:39:21.159 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:20 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7[50588]: 2024-04-26T20:39:20.777+0000 7fefe71cb700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:55.105651+0000 front 2024-04-26T20:38:55.105671+0000 (oldest deadline 2024-04-26T20:39:20.306301+0000) 2024-04-26T20:39:21.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:20.899+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:21.240 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:20.899+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:21.306 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:20.899+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:21.346 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:20.899+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:21.356 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:20.961+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:21.454 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:20.961+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:21.462 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:20.961+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:21.471 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:20 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:20.961+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:21.567 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:21 smithi148 podman[73252]: 2024-04-26 20:39:21.492295471 +0000 UTC m=+5.599578656 container died bd92ace124caba3efc18e32f7f890e542410edb8370ecc1400ef530bd035ae36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:39:21.576 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:21 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:21.302+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:21.585 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:21 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:21.302+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:21.593 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:21 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:21.302+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:21.635 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:21 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:21.302+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:21.665 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:21 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:21.302+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:22.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:21.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:22.015 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:21.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:22.024 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:21.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:22.032 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:21.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:22.043 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:21 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:21.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:22.067 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:21 smithi148 podman[73252]: 2024-04-26 20:39:21.910844141 +0000 UTC m=+6.018127336 container cleanup bd92ace124caba3efc18e32f7f890e542410edb8370ecc1400ef530bd035ae36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:39:22.122 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:21 smithi148 bash[73252]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7 2024-04-26T20:39:22.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:21.900+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:22.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:21.900+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:22.246 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:21.900+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:22.253 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:21.900+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:22.314 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:21.939+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:22.323 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:21.939+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:22.332 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:21.939+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:22.340 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:21 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:21.939+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:22.473 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:22 smithi148 podman[73270]: 2024-04-26 20:39:22.138204733 +0000 UTC m=+0.634375307 container remove bd92ace124caba3efc18e32f7f890e542410edb8370ecc1400ef530bd035ae36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T20:39:22.483 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:22 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:22.332+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:22.491 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:22 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:22.332+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:22.501 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:22 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:22.332+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:22.509 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:22 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:22.332+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:22.517 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:22 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:22.332+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:22.766 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:22 smithi148 podman[73360]: 2024-04-26 20:39:22.375800469 +0000 UTC m=+0.017944993 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:39:22.773 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:22 smithi148 podman[73360]: 2024-04-26 20:39:22.480770115 +0000 UTC m=+0.122914683 container create 0308c390c82ed90a7e20211c3f340fe68e79ebd9c7ff022fb520e05c478b1265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T20:39:22.815 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:22 smithi148 podman[73360]: 2024-04-26 20:39:22.700341235 +0000 UTC m=+0.342485764 container init 0308c390c82ed90a7e20211c3f340fe68e79ebd9c7ff022fb520e05c478b1265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T20:39:22.822 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:22 smithi148 podman[73360]: 2024-04-26 20:39:22.703867547 +0000 UTC m=+0.346012077 container start 0308c390c82ed90a7e20211c3f340fe68e79ebd9c7ff022fb520e05c478b1265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T20:39:22.886 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:22 smithi148 podman[73360]: 2024-04-26 20:39:22.766481585 +0000 UTC m=+0.408626110 container attach 0308c390c82ed90a7e20211c3f340fe68e79ebd9c7ff022fb520e05c478b1265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T20:39:23.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:22.574+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:23.014 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:22.574+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:23.068 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:22.574+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:23.075 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:22.574+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:23.084 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:22 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:22.574+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:23.093 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 26 20:39:23 smithi148 podman[73360]: 2024-04-26 20:39:23.010705848 +0000 UTC m=+0.652850378 container died 0308c390c82ed90a7e20211c3f340fe68e79ebd9c7ff022fb520e05c478b1265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T20:39:23.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:22.874+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:23.260 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:22.874+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:23.269 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:22.874+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:23.278 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:22.874+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:23.285 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:22.972+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:23.293 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:22.972+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:23.322 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:22.972+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:23.330 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:22 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:22.972+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:23.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:23 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:23.299+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:23.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:23 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:23.299+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:23.631 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:23 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:23.299+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:23.640 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:23 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:23.299+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:23.648 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:23 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:23.299+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:23.713 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T20:39:23.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:39:23.798 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T20:39:23.805 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T20:39:23.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.8 2024-04-26T20:39:24.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:23 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:23.526+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:24.013 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:23 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:23.526+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:24.019 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:23 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:23.526+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:24.025 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:23 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:23.526+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:24.031 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:23 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:23.526+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:24.182 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:23.932+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:24.187 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:23.932+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:24.193 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:23.932+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:24.298 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:23.932+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:24.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:23.845+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:24.385 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:23.845+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:24.393 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:23.845+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:24.442 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:23 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:23.845+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:24.451 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:23 smithi157 systemd[1]: Stopping Ceph osd.8 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:39:24.461 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:24.180+0000 7fef7685d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:39:24.469 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:24.180+0000 7fef7685d700 -1 osd.8 101 *** Got signal Terminated *** 2024-04-26T20:39:24.506 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:24.180+0000 7fef7685d700 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:39:24.756 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:24.481+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:24.763 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:24.481+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:24.771 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:24.481+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:24.780 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:24.481+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:24.788 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:24 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:24.481+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:24.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:24.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:24.847 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:24.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:24.895 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:24.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:24.904 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:24.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:24.959 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:24 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:24.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:25.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:24.862+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:25.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:24.862+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:25.294 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:24.862+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:25.303 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:24.862+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:25.311 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:24.916+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:25.318 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:24.916+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:25.390 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:24.916+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:25.398 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:24.916+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:25.406 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:24 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:24.916+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:25.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:25 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:25.328+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:25.823 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:25 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:25.328+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:25.831 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:25 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:25.328+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:25.898 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:25 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:25.328+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:25.905 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:25 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:25.328+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:26.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:25 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:25.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:26.012 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:25 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:25.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:26.075 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:25 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:25.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:26.087 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:25 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:25.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:26.093 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:25 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:25.516+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:26.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:25.908+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:26.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:25.908+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:26.243 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:25.908+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:26.251 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:25.908+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:26.312 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:25.908+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:01.307196+0000 front 2024-04-26T20:39:01.307230+0000 (oldest deadline 2024-04-26T20:39:25.407318+0000) 2024-04-26T20:39:26.319 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:25.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:26.326 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:25.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:26.333 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:25.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:26.340 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:25.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:26.346 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:25 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:25.903+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:26.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:26 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:26.306+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:26.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:26 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:26.306+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:26.631 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:26 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:26.306+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:26.637 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:26 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:26.306+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:26.710 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:26 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:26.306+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:26.817 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:26 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:26.554+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:26.824 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:26 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:26.554+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:26.883 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:26 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:26.554+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:26.892 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:26 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:26.554+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:26.900 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:26 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:26.554+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:27.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:26.894+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:27.292 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:26.894+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:27.299 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:26.894+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:27.307 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:26.894+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:27.335 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:26.894+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:01.307196+0000 front 2024-04-26T20:39:01.307230+0000 (oldest deadline 2024-04-26T20:39:25.407318+0000) 2024-04-26T20:39:27.343 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:26.904+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:27.352 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:26.904+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:27.360 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:26.904+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:27.381 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:26.904+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:27.448 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:26 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:26.904+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:27.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:27 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:27.316+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:27.838 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:27 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:27.316+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:27.847 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:27 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:27.316+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:27.855 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:27 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:27.316+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:27.863 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:27 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:27.316+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:28.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:27 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:27.527+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:28.012 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:27 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:27.527+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:28.021 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:27 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:27.527+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:28.088 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:27 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:27.527+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:28.095 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:27 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:27.527+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:28.103 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:27 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:27.527+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:28.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:27.906+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:28.240 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:27.906+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:28.248 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:27.906+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:28.309 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:27.906+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:28.317 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:27.906+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:01.307196+0000 front 2024-04-26T20:39:01.307230+0000 (oldest deadline 2024-04-26T20:39:25.407318+0000) 2024-04-26T20:39:28.360 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:27.858+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:28.368 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:27.858+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:28.439 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:27.858+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:28.445 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:27.858+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:28.452 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:27.858+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:28.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:28 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:28.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:28.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:28 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:28.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:28.581 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:28 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:28.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:28.589 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:28 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:28.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:28.634 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:28 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:28.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:28.642 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:28 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:28.284+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:29.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:28 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:28.544+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:29.013 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:28 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:28.544+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:29.020 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:28 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:28.544+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:29.027 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:28 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:28.544+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:29.057 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:28 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:28.544+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:29.066 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:28 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:28.544+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:29.230 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:28.930+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:20.198603+0000 front 2024-04-26T20:38:20.198832+0000 (oldest deadline 2024-04-26T20:38:44.298504+0000) 2024-04-26T20:39:29.239 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:28.930+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.701560+0000 front 2024-04-26T20:38:34.701538+0000 (oldest deadline 2024-04-26T20:38:59.401723+0000) 2024-04-26T20:39:29.294 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:28.930+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:42.303311+0000 front 2024-04-26T20:38:42.303138+0000 (oldest deadline 2024-04-26T20:39:07.003044+0000) 2024-04-26T20:39:29.343 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:28.930+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.406092+0000 front 2024-04-26T20:38:54.406033+0000 (oldest deadline 2024-04-26T20:39:19.006496+0000) 2024-04-26T20:39:29.352 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8[51374]: 2024-04-26T20:39:28.930+0000 7fef6ee49700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:01.307196+0000 front 2024-04-26T20:39:01.307230+0000 (oldest deadline 2024-04-26T20:39:25.407318+0000) 2024-04-26T20:39:29.360 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:28.851+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:29.434 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:28.851+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:29.443 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:28.851+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:29.450 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:28.851+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:29.537 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:28 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:28.851+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:29.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:29 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:29.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:29.576 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:29 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:29.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:29.584 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:29 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:29.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:29.592 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:29 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:29.271+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:29.601 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:29 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:29.272+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:29.655 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:29 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:29.272+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:29.840 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:29 smithi157 podman[74220]: 2024-04-26 20:39:29.545191266 +0000 UTC m=+5.489138174 container died 485fbecd04718167a83232a8453b1693f7db6bed68b8680c63346c711ae93162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T20:39:30.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:29.563+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:30.013 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:29.563+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:30.084 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:29.563+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:30.094 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:29.563+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:30.112 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:29.563+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:30.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:29 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:29.563+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:30.150 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:29 smithi157 podman[74220]: 2024-04-26 20:39:29.939600431 +0000 UTC m=+5.883547346 container cleanup 485fbecd04718167a83232a8453b1693f7db6bed68b8680c63346c711ae93162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T20:39:30.158 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:29 smithi157 bash[74220]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8 2024-04-26T20:39:30.167 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:29 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:29.862+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:30.215 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:29 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:29.862+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:30.223 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:29 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:29.862+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:30.231 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:29 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:29.862+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:30.238 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:29 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:29.862+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:30.480 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:30 smithi157 podman[74237]: 2024-04-26 20:39:30.140733563 +0000 UTC m=+0.595789994 container remove 485fbecd04718167a83232a8453b1693f7db6bed68b8680c63346c711ae93162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T20:39:30.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:30 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:30.305+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:30.573 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:30 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:30.305+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:30.585 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:30 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:30.305+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:30.593 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:30 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:30.305+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:30.648 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:30 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:30.305+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:30.725 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:30 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:30.305+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:30.794 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:30 smithi157 podman[74327]: 2024-04-26 20:39:30.541524305 +0000 UTC m=+0.113189753 container create 83444ccd11c16781cde5d6d684ff0da84bb6eba6adc706356fe60dfcb8f8077c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:39:30.800 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:30 smithi157 podman[74327]: 2024-04-26 20:39:30.446257696 +0000 UTC m=+0.017923103 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:39:30.806 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:30 smithi157 podman[74327]: 2024-04-26 20:39:30.793945761 +0000 UTC m=+0.365611174 container init 83444ccd11c16781cde5d6d684ff0da84bb6eba6adc706356fe60dfcb8f8077c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T20:39:31.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:30.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:31.012 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:30.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:31.021 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:30.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:31.079 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:30.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:31.086 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:30.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:31.094 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:30 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:30.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:31.102 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:30 smithi157 podman[74327]: 2024-04-26 20:39:30.797684637 +0000 UTC m=+0.369350044 container start 83444ccd11c16781cde5d6d684ff0da84bb6eba6adc706356fe60dfcb8f8077c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T20:39:31.110 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:30 smithi157 podman[74327]: 2024-04-26 20:39:30.902116774 +0000 UTC m=+0.473782186 container attach 83444ccd11c16781cde5d6d684ff0da84bb6eba6adc706356fe60dfcb8f8077c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T20:39:31.118 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:30 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:30.891+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:31.183 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:30 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:30.891+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:31.192 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:30 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:30.891+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:31.246 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:30 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:30.891+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:31.255 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:30 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:30.891+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:31.366 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 26 20:39:31 smithi157 podman[74327]: 2024-04-26 20:39:31.090427165 +0000 UTC m=+0.662092570 container died 83444ccd11c16781cde5d6d684ff0da84bb6eba6adc706356fe60dfcb8f8077c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T20:39:31.791 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T20:39:31.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:31.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:31.848 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:31.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:31.858 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:31.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:31.866 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:31.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:31.874 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:31.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:31.943 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:31 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:31.338+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:31.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:39:31.962 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T20:39:31.969 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T20:39:31.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.9 2024-04-26T20:39:32.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:31.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:32.040 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:31.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:32.048 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:31.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:32.095 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:31.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:32.102 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:31.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:32.166 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:31 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:31.602+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:32.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:31 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:31.865+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:32.236 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:31 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:31.865+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:32.264 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:31 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:31.865+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:32.270 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:31 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:31.865+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:32.277 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:31 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:31.865+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:32.498 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 systemd[1]: Stopping Ceph osd.9 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:39:32.514 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.496+0000 7fd475a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:39:32.520 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.496+0000 7fd475a00700 -1 osd.9 101 *** Got signal Terminated *** 2024-04-26T20:39:32.527 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.496+0000 7fd475a00700 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:39:32.757 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:32.763 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:32.770 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:32.777 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:32.830 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:32.871 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:32 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:32.581+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:32.878 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:32 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:32.324+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:32.938 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:32 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:32.324+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:32.945 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:32 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:32.324+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:32.951 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:32 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:32.324+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:32.960 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:32 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:32.324+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:33.035 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:32 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:32.324+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:33.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:32.852+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:33.238 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:32.852+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:33.300 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:32.852+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:33.309 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:32.852+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:33.317 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:32.852+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:33.325 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:32.852+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:33.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:33 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:33.353+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:33.825 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:33 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:33.353+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:33.885 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:33 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:33.353+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:33.891 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:33 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:33.353+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:33.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:33 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:33.353+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:33.962 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:33 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:33.353+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:34.028 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:33.607+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:34.036 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:33.607+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:34.044 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:33.607+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:34.104 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:33.607+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:34.112 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:33.607+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:34.120 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:33 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:33.607+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:34.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:33.821+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:34.236 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:33.821+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:34.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:33.821+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:34.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:33.821+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:34.262 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:33.821+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:34.273 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:33 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:33.821+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:34.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:34 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:34.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:34.824 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:34 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:34.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:34.830 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:34 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:34.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:34.878 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:34 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:34.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:34.886 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:34 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:34.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:34.977 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:34 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:34.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:35.006 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:34 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:34.604+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:35.029 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:34 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:34.604+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:35.100 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:34 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:34.604+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:35.106 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:34 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:34.604+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:35.112 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:34 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:34.604+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:35.119 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:34 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:34.604+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:35.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:34 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:34.815+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:35.251 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:34 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:34.815+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:35.259 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:34 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:34.815+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:35.265 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:34 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:34.815+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:35.272 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:34 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:34.815+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:34 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:34.815+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:35.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:35.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:35.823 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:35.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:35.887 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:35.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:35.895 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:35.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:35.995 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:35.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:36.001 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:35 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:35.403+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:36.008 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:35.633+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:36.080 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:35.633+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:36.086 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:35.633+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:36.094 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:35.633+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:36.102 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:35.633+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:36.109 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:35 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:35.633+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:36.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:35.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:36.236 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:35.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:36.244 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:35.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:36.252 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:35.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:36.261 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:35.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:36.269 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:35 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:35.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:36.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:36.377+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:36.834 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:36.377+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:36.843 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:36.377+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:36.850 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:36.377+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:36.960 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:36.377+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:36.970 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:36 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:36.377+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:37.013 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:36.618+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:16.761779+0000 front 2024-04-26T20:38:16.761872+0000 (oldest deadline 2024-04-26T20:38:41.362278+0000) 2024-04-26T20:39:37.021 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:36.618+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:27.165564+0000 front 2024-04-26T20:38:27.165804+0000 (oldest deadline 2024-04-26T20:38:54.766927+0000) 2024-04-26T20:39:37.029 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:36.618+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:34.767819+0000 front 2024-04-26T20:38:34.767891+0000 (oldest deadline 2024-04-26T20:38:58.268015+0000) 2024-04-26T20:39:37.037 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:36.618+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.671065+0000 front 2024-04-26T20:38:44.671423+0000 (oldest deadline 2024-04-26T20:39:08.771229+0000) 2024-04-26T20:39:37.093 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:36.618+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:53.374214+0000 front 2024-04-26T20:38:53.374283+0000 (oldest deadline 2024-04-26T20:39:19.274477+0000) 2024-04-26T20:39:37.101 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:36 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9[62373]: 2024-04-26T20:39:36.618+0000 7fd46c200700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:38:59.275617+0000 front 2024-04-26T20:38:59.275563+0000 (oldest deadline 2024-04-26T20:39:27.476819+0000) 2024-04-26T20:39:37.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:36.836+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:37.236 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:36.836+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:37.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:36.836+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:37.255 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:36.836+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:37.318 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:36.836+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:37.326 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:36 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:36.836+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:37.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:37.383+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:37.826 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:37.383+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:37.890 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:37.383+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:37.898 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:37.383+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:37.906 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:37.383+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:37.972 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:37.383+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:37.979 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:37 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:37.383+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:38.132 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:37.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:38.140 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:37.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:38.204 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:37.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:38.213 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:37.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:38.222 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:37.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:38.231 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:37.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:38.362 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:38 smithi131 podman[97440]: 2024-04-26 20:39:38.042076566 +0000 UTC m=+5.686979440 container died 1696fbeb42ac366c8dbca05317255bfb8e24c3d1af0a5481a6448e74dceddfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:39:38.733 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:38 smithi131 podman[97440]: 2024-04-26 20:39:38.472688024 +0000 UTC m=+6.117590892 container cleanup 1696fbeb42ac366c8dbca05317255bfb8e24c3d1af0a5481a6448e74dceddfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:39:38.748 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:38 smithi131 bash[97440]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9 2024-04-26T20:39:38.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:38.387+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:38.827 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:38.387+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:38.835 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:38.387+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:38.844 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:38.387+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:38.937 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:38.387+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:38.945 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:38.387+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:39.010 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:38 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:38.387+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:39.019 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:38 smithi131 podman[97458]: 2024-04-26 20:39:38.731972449 +0000 UTC m=+0.671065728 container remove 1696fbeb42ac366c8dbca05317255bfb8e24c3d1af0a5481a6448e74dceddfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T20:39:39.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:38.824+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:39.239 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:38.824+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:39.304 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:38.824+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:39.312 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:38.824+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:39.319 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:38.824+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:39.327 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:38 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:38.824+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:39.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:39.425+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:39.823 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:39.425+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:39.871 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:39.425+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:39.880 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:39.425+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:39.940 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:39.425+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:39.946 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:39.425+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:40.056 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:39 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:39.425+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:40.066 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:39 smithi131 podman[97547]: 2024-04-26 20:39:39.425006582 +0000 UTC m=+0.032434788 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:39:40.072 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:39 smithi131 podman[97547]: 2024-04-26 20:39:39.541000362 +0000 UTC m=+0.148428605 container create a891414e70872528ba696794ce03d3e0f4f4a5d65230982b3f4be8a95483345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T20:39:40.133 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:39 smithi131 podman[97547]: 2024-04-26 20:39:39.854304576 +0000 UTC m=+0.461732879 container init a891414e70872528ba696794ce03d3e0f4f4a5d65230982b3f4be8a95483345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:39:40.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:39.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:40.236 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:39.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:40.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:39.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:40.281 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:39.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:40.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:39.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:40.299 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:39 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:39.820+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:40.308 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:39 smithi131 podman[97547]: 2024-04-26 20:39:39.865595172 +0000 UTC m=+0.473023404 container start a891414e70872528ba696794ce03d3e0f4f4a5d65230982b3f4be8a95483345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T20:39:40.316 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:39 smithi131 podman[97547]: 2024-04-26 20:39:39.968243935 +0000 UTC m=+0.575672228 container attach a891414e70872528ba696794ce03d3e0f4f4a5d65230982b3f4be8a95483345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T20:39:40.617 INFO:journalctl@ceph.osd.9.smithi131.stdout:Apr 26 20:39:40 smithi131 podman[97547]: 2024-04-26 20:39:40.313401251 +0000 UTC m=+0.920829500 container died a891414e70872528ba696794ce03d3e0f4f4a5d65230982b3f4be8a95483345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T20:39:40.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:40.413+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:40.823 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:40.413+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:40.877 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:40.413+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:40.886 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:40.413+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:40.953 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:40.413+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:40.960 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:40.413+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:40.969 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:40 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:40.413+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:41.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T20:39:41.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:39:41.183 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T20:39:41.190 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T20:39:41.259 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.10 2024-04-26T20:39:41.265 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:40.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:41.272 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:40.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:41.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:40.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:41.286 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:40.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:41.292 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:40.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:41.299 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:40.791+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:41.700 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 systemd[1]: Stopping Ceph osd.10 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:39:41.707 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.415+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:41.713 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.415+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:41.807 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.415+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:41.839 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.415+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:41.846 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.415+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:41.916 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.415+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:41.924 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.415+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:41.931 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.699+0000 7fbe4fac3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:39:41.938 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.699+0000 7fbe4fac3700 -1 osd.10 101 *** Got signal Terminated *** 2024-04-26T20:39:41.969 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:41 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:41.699+0000 7fbe4fac3700 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:39:42.231 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:41 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:41.840+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:42.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:41 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:41.840+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:42.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:41 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:41.840+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:42.251 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:41 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:41.840+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:42.258 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:41 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:41.840+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:42.267 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:41 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:41.840+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:42.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:42.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:42.824 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:42.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:42.832 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:42.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:42.906 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:42.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:42.914 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:42.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:42.922 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:42.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:42.985 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:42 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:42.386+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:43.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:42 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:42.817+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:43.239 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:42 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:42.817+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:43.248 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:42 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:42.817+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:43.287 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:42 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:42.817+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:43.296 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:42 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:42.817+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:43.304 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:42 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:42.817+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:43.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:43.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:43.823 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:43.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:43.831 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:43.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:43.886 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:43.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:43.895 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:43.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:43.957 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:43.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:43.966 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:43 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:43.426+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:44.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:43 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:43.798+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:44.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:43 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:43.798+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:44.246 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:43 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:43.798+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:44.334 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:43 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:43.798+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:44.342 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:43 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:43.798+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:44.351 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:43 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:43.798+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:44.816 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:44.468+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:44.844 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:44.468+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:44.853 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:44.468+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:44.862 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:44.468+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:44.887 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:44.468+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:44.896 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:44.468+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:44.904 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:44 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:44.468+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:45.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:44.825+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:45.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:44.825+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:45.317 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:44.825+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:45.326 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:44.825+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:45.422 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:44.825+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:45.430 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:44 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:44.825+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:45.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:45.512+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:45.823 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:45.512+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:45.831 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:45.512+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:45.839 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:45.512+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:45.848 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:45.512+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:45.939 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:45.512+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:45.947 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:45 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:45.512+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:46.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:45.845+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:46.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:45.845+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:46.244 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:45.845+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:46.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:45.845+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:46.256 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:45.845+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:46.309 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:45 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:45.845+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:46.817 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:46.534+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:15.806341+0000 front 2024-04-26T20:38:15.806276+0000 (oldest deadline 2024-04-26T20:38:41.106260+0000) 2024-04-26T20:39:46.825 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:46.535+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.107839+0000 front 2024-04-26T20:38:28.107859+0000 (oldest deadline 2024-04-26T20:38:53.407961+0000) 2024-04-26T20:39:46.924 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:46.535+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.710201+0000 front 2024-04-26T20:38:36.710030+0000 (oldest deadline 2024-04-26T20:39:02.609919+0000) 2024-04-26T20:39:46.937 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:46.535+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-04-26T20:38:44.711917+0000 front 2024-04-26T20:38:44.712095+0000 (oldest deadline 2024-04-26T20:39:08.811588+0000) 2024-04-26T20:39:46.982 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:46.535+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:54.612733+0000 front 2024-04-26T20:38:54.612795+0000 (oldest deadline 2024-04-26T20:39:19.912853+0000) 2024-04-26T20:39:47.042 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:46.535+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:03.314859+0000 front 2024-04-26T20:39:03.314766+0000 (oldest deadline 2024-04-26T20:39:27.414614+0000) 2024-04-26T20:39:47.051 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:46 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10[57647]: 2024-04-26T20:39:46.535+0000 7fbe4c0b7700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:11.515990+0000 front 2024-04-26T20:39:11.515920+0000 (oldest deadline 2024-04-26T20:39:36.815634+0000) 2024-04-26T20:39:47.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:46.797+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:47.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:46.797+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:47.297 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:46.797+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:47.305 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:46.797+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:47.312 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:46.797+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:47.318 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:46 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:46.797+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:47.325 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:47 smithi148 podman[73464]: 2024-04-26 20:39:47.009756562 +0000 UTC m=+5.453951897 container died 80652a7b488deeeea2652fd199f661789ab0636dd8bb02fee099400b8a17dbf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:39:47.567 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:47 smithi148 podman[73464]: 2024-04-26 20:39:47.351859489 +0000 UTC m=+5.796054823 container cleanup 80652a7b488deeeea2652fd199f661789ab0636dd8bb02fee099400b8a17dbf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T20:39:47.574 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:47 smithi148 bash[73464]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10 2024-04-26T20:39:47.580 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:47 smithi148 podman[73494]: 2024-04-26 20:39:47.527807946 +0000 UTC m=+0.520428773 container remove 80652a7b488deeeea2652fd199f661789ab0636dd8bb02fee099400b8a17dbf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:39:48.195 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:47.783+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:48.202 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:47.783+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:48.298 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:47.783+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:48.307 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:47.784+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:48.315 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:47.784+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:48.366 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:47.784+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:48.373 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:47 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:47.784+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:48.382 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:47 smithi148 podman[73586]: 2024-04-26 20:39:47.959442925 +0000 UTC m=+0.106438860 container create 7aac1b6c24cc83b6967fb0ec1491f06774a9b7d46cd6ab112c711b562925bc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T20:39:48.391 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:47 smithi148 podman[73586]: 2024-04-26 20:39:47.870776492 +0000 UTC m=+0.017772426 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:39:48.489 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:48 smithi148 podman[73586]: 2024-04-26 20:39:48.164433306 +0000 UTC m=+0.311429242 container init 7aac1b6c24cc83b6967fb0ec1491f06774a9b7d46cd6ab112c711b562925bc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T20:39:48.498 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:48 smithi148 podman[73586]: 2024-04-26 20:39:48.16742155 +0000 UTC m=+0.314417491 container start 7aac1b6c24cc83b6967fb0ec1491f06774a9b7d46cd6ab112c711b562925bc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T20:39:48.506 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:48 smithi148 podman[73586]: 2024-04-26 20:39:48.247490855 +0000 UTC m=+0.394486807 container attach 7aac1b6c24cc83b6967fb0ec1491f06774a9b7d46cd6ab112c711b562925bc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T20:39:48.566 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 26 20:39:48 smithi148 podman[73586]: 2024-04-26 20:39:48.465196114 +0000 UTC m=+0.612192043 container died 7aac1b6c24cc83b6967fb0ec1491f06774a9b7d46cd6ab112c711b562925bc57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T20:39:49.118 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T20:39:49.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:39:49.220 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T20:39:49.229 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T20:39:49.306 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.11 2024-04-26T20:39:49.314 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:48.770+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:49.387 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:48.770+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:49.398 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:48.770+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:49.407 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:48.770+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:49.454 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:48.770+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:49.462 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:48.770+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:49.472 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:48 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:48.770+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:49.767 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 systemd[1]: Stopping Ceph osd.11 for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:39:49.779 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.734+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:49.787 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.734+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:49.796 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.734+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:49.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.734+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:49.814 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.734+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:49.822 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.734+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:49.868 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.734+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:49.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.765+0000 7fd4ea8da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:39:49.942 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.765+0000 7fd4ea8da700 -1 osd.11 101 *** Got signal Terminated *** 2024-04-26T20:39:49.952 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:49 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:49.765+0000 7fd4ea8da700 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T20:39:51.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:50.745+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:51.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:50.745+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:51.273 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:50.745+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:51.280 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:50.745+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:51.304 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:50.745+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:51.311 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:50.745+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:51.318 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:50 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:50.745+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:52.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:51.778+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:52.238 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:51.778+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:52.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:51.778+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:52.252 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:51.778+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:52.259 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:51.778+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:52.266 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:51.778+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:52.333 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:51 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:51.778+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:53.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:53.238 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:53.247 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:53.255 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:53.264 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:53.272 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:53.366 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:53.373 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:52 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:52.755+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-04-26T20:39:26.836593+0000 front 2024-04-26T20:39:26.836549+0000 (oldest deadline 2024-04-26T20:39:52.736803+0000) 2024-04-26T20:39:54.230 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:54.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:54.313 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:54.319 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:54.326 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:54.333 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:54.340 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:54.347 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:53 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:53.757+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-04-26T20:39:26.836593+0000 front 2024-04-26T20:39:26.836549+0000 (oldest deadline 2024-04-26T20:39:52.736803+0000) 2024-04-26T20:39:55.193 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-04-26T20:38:18.126122+0000 front 2024-04-26T20:38:18.125988+0000 (oldest deadline 2024-04-26T20:38:44.025820+0000) 2024-04-26T20:39:55.200 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-26T20:38:28.027592+0000 front 2024-04-26T20:38:28.027390+0000 (oldest deadline 2024-04-26T20:38:50.927497+0000) 2024-04-26T20:39:55.307 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-26T20:38:36.228557+0000 front 2024-04-26T20:38:36.228586+0000 (oldest deadline 2024-04-26T20:38:59.728704+0000) 2024-04-26T20:39:55.325 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-26T20:38:50.830759+0000 front 2024-04-26T20:38:50.830838+0000 (oldest deadline 2024-04-26T20:39:19.031187+0000) 2024-04-26T20:39:55.334 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-26T20:39:00.132162+0000 front 2024-04-26T20:39:00.132199+0000 (oldest deadline 2024-04-26T20:39:24.232281+0000) 2024-04-26T20:39:55.343 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-04-26T20:39:10.534361+0000 front 2024-04-26T20:39:10.534480+0000 (oldest deadline 2024-04-26T20:39:32.834251+0000) 2024-04-26T20:39:55.352 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-26T20:39:20.936426+0000 front 2024-04-26T20:39:20.936385+0000 (oldest deadline 2024-04-26T20:39:46.836171+0000) 2024-04-26T20:39:55.440 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:54 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11[58411]: 2024-04-26T20:39:54.762+0000 7fd4e6ece700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-04-26T20:39:26.836593+0000 front 2024-04-26T20:39:26.836549+0000 (oldest deadline 2024-04-26T20:39:52.736803+0000) 2024-04-26T20:39:55.480 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:55 smithi157 podman[74445]: 2024-04-26 20:39:55.192728379 +0000 UTC m=+5.521066285 container died bdc2b35ce8221c451d568b8ccb00ea9aa50e235db381b611621c3a011f9edd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T20:39:55.817 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:55 smithi157 podman[74445]: 2024-04-26 20:39:55.565471477 +0000 UTC m=+5.893809383 container cleanup bdc2b35ce8221c451d568b8ccb00ea9aa50e235db381b611621c3a011f9edd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T20:39:55.823 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:55 smithi157 bash[74445]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11 2024-04-26T20:39:56.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:55 smithi157 podman[74460]: 2024-04-26 20:39:55.816572486 +0000 UTC m=+0.625458382 container remove bdc2b35ce8221c451d568b8ccb00ea9aa50e235db381b611621c3a011f9edd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:39:56.411 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:56 smithi157 podman[74552]: 2024-04-26 20:39:56.125885399 +0000 UTC m=+0.099192340 container create cb9be6d1e60b963adad0f2020e6ce13a1199e70c0146522b3efc6678dea0b535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T20:39:56.418 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:56 smithi157 podman[74552]: 2024-04-26 20:39:56.04449107 +0000 UTC m=+0.017798007 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T20:39:56.426 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:56 smithi157 podman[74552]: 2024-04-26 20:39:56.336442799 +0000 UTC m=+0.309749736 container init cb9be6d1e60b963adad0f2020e6ce13a1199e70c0146522b3efc6678dea0b535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T20:39:56.435 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:56 smithi157 podman[74552]: 2024-04-26 20:39:56.339566704 +0000 UTC m=+0.312873647 container start cb9be6d1e60b963adad0f2020e6ce13a1199e70c0146522b3efc6678dea0b535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:39:56.730 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:56 smithi157 podman[74552]: 2024-04-26 20:39:56.411200975 +0000 UTC m=+0.384507926 container attach cb9be6d1e60b963adad0f2020e6ce13a1199e70c0146522b3efc6678dea0b535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T20:39:56.749 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:56 smithi157 podman[74552]: 2024-04-26 20:39:56.632712506 +0000 UTC m=+0.606019444 container died cb9be6d1e60b963adad0f2020e6ce13a1199e70c0146522b3efc6678dea0b535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T20:39:57.166 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 26 20:39:57 smithi157 podman[74578]: 2024-04-26 20:39:57.147488324 +0000 UTC m=+0.509792405 container remove cb9be6d1e60b963adad0f2020e6ce13a1199e70c0146522b3efc6678dea0b535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:39:57.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T20:39:57.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:39:57.255 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T20:39:57.263 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T20:39:57.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.a 2024-04-26T20:39:57.756 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:39:57 smithi131 systemd[1]: Stopping Ceph mds.a for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:39:58.173 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:39:57 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a[69256]: 2024-04-26T20:39:57.961+0000 7f9023800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:39:58.180 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:39:57 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a[69256]: 2024-04-26T20:39:57.961+0000 7f9023800700 -1 mds.a *** got signal Terminated *** 2024-04-26T20:39:59.506 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:39:59 smithi131 podman[97653]: 2024-04-26 20:39:59.213386864 +0000 UTC m=+1.456253158 container died dd30dc3a6547d5475444bb4e5d70e6c6f76e4f0f96b1b486950144bd10a2a5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T20:39:59.955 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:39:59 smithi131 podman[97653]: 2024-04-26 20:39:59.691354717 +0000 UTC m=+1.934221018 container cleanup dd30dc3a6547d5475444bb4e5d70e6c6f76e4f0f96b1b486950144bd10a2a5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T20:40:00.022 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:39:59 smithi131 bash[97653]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a 2024-04-26T20:40:00.256 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:39:59 smithi131 podman[97678]: 2024-04-26 20:39:59.95448936 +0000 UTC m=+0.724707714 container remove dd30dc3a6547d5475444bb4e5d70e6c6f76e4f0f96b1b486950144bd10a2a5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T20:40:00.470 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T20:40:00.588 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:40:00 smithi131 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.a.service: Deactivated successfully. 2024-04-26T20:40:00.598 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:40:00 smithi131 systemd[1]: Stopped Ceph mds.a for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:40:00.607 INFO:journalctl@ceph.mds.a.smithi131.stdout:Apr 26 20:40:00 smithi131 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.a.service: Consumed 7.492s CPU time. 2024-04-26T20:40:00.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:00.630 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T20:40:00.637 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T20:40:00.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.d 2024-04-26T20:40:01.006 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:40:00 smithi131 systemd[1]: Stopping Ceph mds.d for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:01.756 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:40:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-d[68313]: 2024-04-26T20:40:01.295+0000 7f5229400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:01.772 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:40:01 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-d[68313]: 2024-04-26T20:40:01.295+0000 7f5229400700 -1 mds.d *** got signal Terminated *** 2024-04-26T20:40:02.743 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:40:02 smithi131 podman[97786]: 2024-04-26 20:40:02.460105208 +0000 UTC m=+1.362319256 container died 6dfa9a2d47bdcb36c506defe4909db782e2d5792315a63a3d989d5e63cd4a331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:40:03.006 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:40:02 smithi131 podman[97786]: 2024-04-26 20:40:02.867887823 +0000 UTC m=+1.770101864 container cleanup 6dfa9a2d47bdcb36c506defe4909db782e2d5792315a63a3d989d5e63cd4a331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T20:40:03.046 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:40:02 smithi131 bash[97786]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-d 2024-04-26T20:40:03.506 INFO:journalctl@ceph.mds.d.smithi131.stdout:Apr 26 20:40:03 smithi131 podman[97799]: 2024-04-26 20:40:03.13843544 +0000 UTC m=+0.658813062 container remove 6dfa9a2d47bdcb36c506defe4909db782e2d5792315a63a3d989d5e63cd4a331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T20:40:03.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T20:40:03.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:03.809 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T20:40:03.816 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T20:40:03.823 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.g 2024-04-26T20:40:04.256 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:40:03 smithi131 systemd[1]: Stopping Ceph mds.g for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:04.756 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:40:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g[68903]: 2024-04-26T20:40:04.393+0000 7fb8f0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:04.772 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:40:04 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g[68903]: 2024-04-26T20:40:04.393+0000 7fb8f0a00700 -1 mds.g *** got signal Terminated *** 2024-04-26T20:40:07.471 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:40:07 smithi131 podman[97904]: 2024-04-26 20:40:07.214102232 +0000 UTC m=+2.953998389 container died cb8a2959c13291de06aaa57053997e9911d8277b7d1c28529559f019bdab73c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T20:40:07.756 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:40:07 smithi131 podman[97904]: 2024-04-26 20:40:07.615970364 +0000 UTC m=+3.355866559 container cleanup cb8a2959c13291de06aaa57053997e9911d8277b7d1c28529559f019bdab73c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T20:40:07.774 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:40:07 smithi131 bash[97904]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g 2024-04-26T20:40:08.256 INFO:journalctl@ceph.mds.g.smithi131.stdout:Apr 26 20:40:07 smithi131 podman[97918]: 2024-04-26 20:40:07.817205353 +0000 UTC m=+0.591393865 container remove cb8a2959c13291de06aaa57053997e9911d8277b7d1c28529559f019bdab73c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T20:40:08.320 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T20:40:08.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:08.441 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T20:40:08.450 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T20:40:08.458 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.j 2024-04-26T20:40:09.006 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:40:08 smithi131 systemd[1]: Stopping Ceph mds.j for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:09.506 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:40:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j[67956]: 2024-04-26T20:40:09.009+0000 7f77e9c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:09.546 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:40:09 smithi131 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j[67956]: 2024-04-26T20:40:09.009+0000 7f77e9c00700 -1 mds.j *** got signal Terminated *** 2024-04-26T20:40:10.669 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:40:10 smithi131 podman[98023]: 2024-04-26 20:40:10.355919249 +0000 UTC m=+1.468534702 container died c68ace96567d22b1633ed884d9e6ba9326b755cd1232e876763072f0787a062c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T20:40:11.006 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:40:10 smithi131 podman[98023]: 2024-04-26 20:40:10.778475921 +0000 UTC m=+1.891091395 container cleanup c68ace96567d22b1633ed884d9e6ba9326b755cd1232e876763072f0787a062c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T20:40:11.016 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:40:10 smithi131 bash[98023]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j 2024-04-26T20:40:11.506 INFO:journalctl@ceph.mds.j.smithi131.stdout:Apr 26 20:40:11 smithi131 podman[98037]: 2024-04-26 20:40:11.054464352 +0000 UTC m=+0.687120485 container remove c68ace96567d22b1633ed884d9e6ba9326b755cd1232e876763072f0787a062c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-j, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:40:11.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T20:40:11.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:11.778 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T20:40:11.786 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T20:40:11.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.b 2024-04-26T20:40:12.066 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:40:11 smithi148 systemd[1]: Stopping Ceph mds.b for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:12.325 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:40:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b[63000]: 2024-04-26T20:40:12.100+0000 7fac70865700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:12.340 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:40:12 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b[63000]: 2024-04-26T20:40:12.100+0000 7fac70865700 -1 mds.b *** got signal Terminated *** 2024-04-26T20:40:15.443 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:40:15 smithi148 podman[73691]: 2024-04-26 20:40:15.154366276 +0000 UTC m=+3.160915162 container died 6ad835133a7c3f088aeab26a00ee6ef8763f011aba95592d2f5f1a26dd1b0852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T20:40:15.801 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:40:15 smithi148 podman[73691]: 2024-04-26 20:40:15.517082674 +0000 UTC m=+3.523631558 container cleanup 6ad835133a7c3f088aeab26a00ee6ef8763f011aba95592d2f5f1a26dd1b0852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T20:40:15.808 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 26 20:40:15 smithi148 bash[73691]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-b 2024-04-26T20:40:15.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T20:40:15.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:15.991 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T20:40:16.000 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T20:40:16.008 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.e 2024-04-26T20:40:16.316 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:40:16 smithi148 systemd[1]: Stopping Ceph mds.e for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:16.323 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:40:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e[62374]: 2024-04-26T20:40:16.304+0000 7fde81b3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:16.330 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:40:16 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e[62374]: 2024-04-26T20:40:16.304+0000 7fde81b3b700 -1 mds.e *** got signal Terminated *** 2024-04-26T20:40:17.614 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:40:17 smithi148 podman[73822]: 2024-04-26 20:40:17.317511026 +0000 UTC m=+1.117854175 container died e6a0e595490d00a4538297bd825624492dd5a081669c5131ce313ab35bff56bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T20:40:17.939 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:40:17 smithi148 podman[73822]: 2024-04-26 20:40:17.70427644 +0000 UTC m=+1.504619598 container cleanup e6a0e595490d00a4538297bd825624492dd5a081669c5131ce313ab35bff56bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-26T20:40:17.946 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 26 20:40:17 smithi148 bash[73822]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-e 2024-04-26T20:40:18.077 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T20:40:18.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:18.122 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T20:40:18.130 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T20:40:18.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.h 2024-04-26T20:40:18.567 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:18 smithi148 systemd[1]: Stopping Ceph mds.h for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:18.574 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h[62679]: 2024-04-26T20:40:18.491+0000 7f397f2b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:18.583 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:18 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h[62679]: 2024-04-26T20:40:18.491+0000 7f397f2b2700 -1 mds.h *** got signal Terminated *** 2024-04-26T20:40:22.759 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:22 smithi148 podman[73943]: 2024-04-26 20:40:22.496830394 +0000 UTC m=+4.099617648 container died 170567daf9efde61fa62a804fae3e8e848ec9c929bf6a1297821ecfa9cbd1517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T20:40:23.066 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:22 smithi148 podman[73943]: 2024-04-26 20:40:22.907493275 +0000 UTC m=+4.510280537 container cleanup 170567daf9efde61fa62a804fae3e8e848ec9c929bf6a1297821ecfa9cbd1517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T20:40:23.076 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:22 smithi148 bash[73943]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h 2024-04-26T20:40:23.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T20:40:23.385 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:23 smithi148 podman[73957]: 2024-04-26 20:40:23.133443724 +0000 UTC m=+0.630595216 container remove 170567daf9efde61fa62a804fae3e8e848ec9c929bf6a1297821ecfa9cbd1517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T20:40:23.395 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:23 smithi148 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.h.service: Deactivated successfully. 2024-04-26T20:40:23.404 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:23 smithi148 systemd[1]: Stopped Ceph mds.h for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:40:23.413 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 26 20:40:23 smithi148 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.553s CPU time. 2024-04-26T20:40:23.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:23.434 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T20:40:23.443 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T20:40:23.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.k 2024-04-26T20:40:23.736 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:23 smithi148 systemd[1]: Stopping Ceph mds.k for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:24.067 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:23 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k[62055]: 2024-04-26T20:40:23.736+0000 7f76eaa8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:24.116 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:23 smithi148 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k[62055]: 2024-04-26T20:40:23.736+0000 7f76eaa8a700 -1 mds.k *** got signal Terminated *** 2024-04-26T20:40:26.736 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:26 smithi148 podman[74065]: 2024-04-26 20:40:26.400790207 +0000 UTC m=+2.758821247 container died fbec80a25b19be1f692d062d9e65d3eadf53f54b2df6a663211358aac35220c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T20:40:27.020 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:26 smithi148 podman[74065]: 2024-04-26 20:40:26.827755943 +0000 UTC m=+3.185786973 container cleanup fbec80a25b19be1f692d062d9e65d3eadf53f54b2df6a663211358aac35220c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T20:40:27.037 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:26 smithi148 bash[74065]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k 2024-04-26T20:40:27.238 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T20:40:27.289 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:27 smithi148 podman[74077]: 2024-04-26 20:40:27.020176468 +0000 UTC m=+0.617067966 container remove fbec80a25b19be1f692d062d9e65d3eadf53f54b2df6a663211358aac35220c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T20:40:27.297 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:27 smithi148 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-26T20:40:27.306 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:27 smithi148 systemd[1]: Stopped Ceph mds.k for 2027e7ca-0407-11ef-bc93-c7b262605968. 2024-04-26T20:40:27.315 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 26 20:40:27 smithi148 systemd[1]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.336s CPU time. 2024-04-26T20:40:27.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:27.333 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T20:40:27.344 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T20:40:27.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.c 2024-04-26T20:40:27.665 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:40:27 smithi157 systemd[1]: Stopping Ceph mds.c for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:27.980 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:40:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c[63527]: 2024-04-26T20:40:27.662+0000 7f53373f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:27.989 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:40:27 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c[63527]: 2024-04-26T20:40:27.662+0000 7f53373f1700 -1 mds.c *** got signal Terminated *** 2024-04-26T20:40:31.423 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:40:31 smithi157 podman[74670]: 2024-04-26 20:40:31.137659656 +0000 UTC m=+3.590198424 container died dafe6d5c07503300fa3f5f8002289a0f4a3fcdaa0b21b46e9f52658e4e93c999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T20:40:31.730 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:40:31 smithi157 podman[74670]: 2024-04-26 20:40:31.513394972 +0000 UTC m=+3.965933736 container cleanup dafe6d5c07503300fa3f5f8002289a0f4a3fcdaa0b21b46e9f52658e4e93c999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T20:40:31.741 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 26 20:40:31 smithi157 bash[74670]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-c 2024-04-26T20:40:31.948 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T20:40:31.992 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:31.999 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T20:40:32.006 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T20:40:32.014 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.f 2024-04-26T20:40:32.309 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:40:32 smithi157 systemd[1]: Stopping Ceph mds.f for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:32.730 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:40:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f[63220]: 2024-04-26T20:40:32.308+0000 7fab02d81700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:32.737 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:40:32 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f[63220]: 2024-04-26T20:40:32.308+0000 7fab02d81700 -1 mds.f *** got signal Terminated *** 2024-04-26T20:40:36.634 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:40:36 smithi157 podman[74791]: 2024-04-26 20:40:36.322818912 +0000 UTC m=+4.118117019 container died 270914e0e0f8ec8f134404e34af2217396782ec7822addc243c2d83c27952558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T20:40:36.959 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:40:36 smithi157 podman[74791]: 2024-04-26 20:40:36.716430879 +0000 UTC m=+4.511728985 container cleanup 270914e0e0f8ec8f134404e34af2217396782ec7822addc243c2d83c27952558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:40:36.966 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 26 20:40:36 smithi157 bash[74791]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-f 2024-04-26T20:40:37.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T20:40:37.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:37.154 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T20:40:37.163 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T20:40:37.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.i 2024-04-26T20:40:37.480 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:40:37 smithi157 systemd[1]: Stopping Ceph mds.i for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:37.980 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:40:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i[63846]: 2024-04-26T20:40:37.503+0000 7f41d9dfd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:37.987 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:40:37 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i[63846]: 2024-04-26T20:40:37.503+0000 7f41d9dfd700 -1 mds.i *** got signal Terminated *** 2024-04-26T20:40:39.413 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:40:39 smithi157 podman[74912]: 2024-04-26 20:40:39.14357658 +0000 UTC m=+1.771497281 container died 7ea454b9e86acfc35cbb60a7fe49479d9b3d28d23042a2ec79795ef64ac47592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T20:40:39.730 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:40:39 smithi157 podman[74912]: 2024-04-26 20:40:39.512069298 +0000 UTC m=+2.139990006 container cleanup 7ea454b9e86acfc35cbb60a7fe49479d9b3d28d23042a2ec79795ef64ac47592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T20:40:39.752 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 26 20:40:39 smithi157 bash[74912]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-i 2024-04-26T20:40:39.938 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T20:40:39.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:39.984 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T20:40:39.992 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T20:40:40.001 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.l 2024-04-26T20:40:40.291 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:40:40 smithi157 systemd[1]: Stopping Ceph mds.l for 2027e7ca-0407-11ef-bc93-c7b262605968... 2024-04-26T20:40:40.730 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:40:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l[62914]: 2024-04-26T20:40:40.290+0000 7fa391abc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:40:40.740 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:40:40 smithi157 ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l[62914]: 2024-04-26T20:40:40.290+0000 7fa391abc700 -1 mds.l *** got signal Terminated *** 2024-04-26T20:40:43.716 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:40:43 smithi157 podman[75031]: 2024-04-26 20:40:43.423245246 +0000 UTC m=+3.239488429 container died d6aba73548fb02eb6fac9a895bb6d0716d066f39005433a2dc27fd3d70aa8b56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-26T20:40:43.980 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:40:43 smithi157 podman[75031]: 2024-04-26 20:40:43.798465214 +0000 UTC m=+3.614708437 container cleanup d6aba73548fb02eb6fac9a895bb6d0716d066f39005433a2dc27fd3d70aa8b56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T20:40:43.990 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 26 20:40:43 smithi157 bash[75031]: ceph-2027e7ca-0407-11ef-bc93-c7b262605968-mds-l 2024-04-26T20:40:44.214 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2027e7ca-0407-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T20:40:44.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T20:40:44.261 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T20:40:44.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T20:40:44.598 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:40:49.542 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T20:40:49.756 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:40:52.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T20:40:52.462 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:40:54.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:40:54.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:40:54.895 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:40:54.928 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T20:40:54.938 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi131/crash 2024-04-26T20:40:54.946 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash -- . 2024-04-26T20:40:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T20:40:55.059 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-04-26T20:40:55.068 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi148/crash 2024-04-26T20:40:55.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash -- . 2024-04-26T20:40:55.122 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T20:40:55.131 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-04-26T20:40:55.140 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi157/crash 2024-04-26T20:40:55.148 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash -- . 2024-04-26T20:40:55.195 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T20:40:55.204 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-04-26T20:40:55.213 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T20:40:55.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T20:40:55.400 INFO:tasks.cephadm:Compressing logs... 2024-04-26T20:40:55.407 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:40:55.419 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:40:55.431 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:40:55.456 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T20:40:55.464 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2024-04-26T20:40:55.485 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log 2024-04-26T20:40:55.494 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T20:40:55.503 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T20:40:55.510 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T20:40:55.517 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T20:40:55.523 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T20:40:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T20:40:55.540 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T20:40:55.548 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.4.log: /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T20:40:55.557 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T20:40:55.565 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T20:40:55.626 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T20:40:55.635 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T20:40:55.643 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T20:40:55.652 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T20:40:55.660 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T20:40:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.e.log: 93.6% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T20:40:55.677 INFO:teuthology.orchestra.run.smithi148.stderr: 94.3% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T20:40:55.685 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T20:40:55.693 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T20:40:55.705 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-04-26T20:40:55.714 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-26T20:40:55.722 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log 2024-04-26T20:40:55.730 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T20:40:55.738 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T20:40:55.746 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T20:40:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T20:40:55.763 INFO:teuthology.orchestra.run.smithi157.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T20:40:55.817 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T20:40:55.826 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T20:40:55.834 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T20:40:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T20:40:55.851 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.z.log: 88.8% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T20:40:55.860 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log 90.6% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T20:40:55.870 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-26T20:40:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T20:40:55.886 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T20:40:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T20:40:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.11.log: 94.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T20:40:55.912 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T20:40:55.922 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.8.log: 93.7% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T20:40:55.930 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T20:40:55.939 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.1% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T20:40:55.947 INFO:teuthology.orchestra.run.smithi157.stderr: 94.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T20:40:55.955 INFO:teuthology.orchestra.run.smithi157.stderr: 94.8% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T20:40:56.022 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T20:40:56.031 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T20:40:56.039 INFO:teuthology.orchestra.run.smithi148.stderr: 95.0% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T20:40:56.048 INFO:teuthology.orchestra.run.smithi157.stderr: 95.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T20:40:56.057 INFO:teuthology.orchestra.run.smithi157.stderr: 91.9% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T20:40:56.070 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory -5 --verbose 2024-04-26T20:40:56.078 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/cephadm.log 2024-04-26T20:40:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log 2024-04-26T20:40:56.094 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T20:40:56.103 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T20:40:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T20:40:56.120 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T20:40:56.128 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T20:40:56.136 INFO:teuthology.orchestra.run.smithi131.stderr: 90.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T20:40:56.144 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T20:40:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T20:40:56.160 INFO:teuthology.orchestra.run.smithi131.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T20:40:56.228 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T20:40:56.237 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T20:40:56.246 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T20:40:56.255 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.6.log: /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T20:40:56.264 INFO:teuthology.orchestra.run.smithi131.stderr: 94.5% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T20:40:56.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.a.log: 83.8% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T20:40:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T20:40:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T20:40:56.300 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T20:40:56.309 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-26T20:40:56.317 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.log: 91.0% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T20:40:56.327 INFO:teuthology.orchestra.run.smithi131.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-26T20:40:56.336 INFO:teuthology.orchestra.run.smithi131.stderr: 94.5% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T20:40:56.345 INFO:teuthology.orchestra.run.smithi131.stderr: 93.6% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T20:40:56.354 INFO:teuthology.orchestra.run.smithi131.stderr: 93.5% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T20:40:56.363 INFO:teuthology.orchestra.run.smithi131.stderr: 89.9% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T20:40:56.373 INFO:teuthology.orchestra.run.smithi148.stderr: 91.9% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T20:40:57.062 INFO:teuthology.orchestra.run.smithi131.stderr: 90.2% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T20:40:57.617 INFO:teuthology.orchestra.run.smithi148.stderr: 92.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T20:40:57.917 INFO:teuthology.orchestra.run.smithi148.stderr: 92.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T20:40:58.137 INFO:teuthology.orchestra.run.smithi131.stderr: 92.3% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T20:40:58.144 INFO:teuthology.orchestra.run.smithi157.stderr: 92.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T20:40:58.165 INFO:teuthology.orchestra.run.smithi131.stderr: 92.0% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T20:40:58.452 INFO:teuthology.orchestra.run.smithi131.stderr: 92.6% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T20:40:58.647 INFO:teuthology.orchestra.run.smithi148.stderr: 91.8% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T20:40:59.024 INFO:teuthology.orchestra.run.smithi157.stderr: 92.0% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T20:40:59.090 INFO:teuthology.orchestra.run.smithi157.stderr: 94.8% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T20:40:59.116 INFO:teuthology.orchestra.run.smithi148.stderr: 92.7% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T20:40:59.182 INFO:teuthology.orchestra.run.smithi131.stderr: 92.4% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T20:40:59.190 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-04-26T20:40:59.198 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m3.729s 2024-04-26T20:40:59.206 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m13.298s 2024-04-26T20:40:59.213 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m1.073s 2024-04-26T20:40:59.240 INFO:teuthology.orchestra.run.smithi157.stderr: 93.1% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T20:40:59.319 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-26T20:40:59.327 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m3.789s 2024-04-26T20:40:59.334 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m14.372s 2024-04-26T20:40:59.342 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m0.532s 2024-04-26T20:40:59.350 INFO:teuthology.orchestra.run.smithi148.stderr: 93.2% -- replaced with /var/log/ceph/2027e7ca-0407-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T20:40:59.358 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-26T20:40:59.367 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m3.897s 2024-04-26T20:40:59.374 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m15.824s 2024-04-26T20:40:59.380 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.520s 2024-04-26T20:40:59.388 INFO:tasks.cephadm:Archiving logs... 2024-04-26T20:40:59.395 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi131/log 2024-04-26T20:40:59.402 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:41:01.434 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi148/log 2024-04-26T20:41:01.441 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:41:03.693 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi157/log 2024-04-26T20:41:03.718 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:41:05.912 INFO:tasks.cephadm:Removing cluster... 2024-04-26T20:41:05.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 --force 2024-04-26T20:41:06.256 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:41:06.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 --force 2024-04-26T20:41:07.112 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:41:07.380 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2027e7ca-0407-11ef-bc93-c7b262605968 --force 2024-04-26T20:41:07.579 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 2027e7ca-0407-11ef-bc93-c7b262605968 2024-04-26T20:41:07.846 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T20:41:07.852 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T20:41:07.915 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T20:41:07.939 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T20:41:07.962 INFO:tasks.cephadm:Teardown complete 2024-04-26T20:41:07.971 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T20:41:07.989 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T20:41:07.997 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:41:08.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:41:08.045 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:41:08.168 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T20:41:08.202 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T20:41:08.211 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:41:08.227 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:41:08.275 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:41:08.285 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:41:08.294 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:41:08.304 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:41:08.312 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-04-26T20:41:08.321 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 7 377 71 +1528us[+1474us] +/- 66ms 2024-04-26T20:41:08.329 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -860us[ -860us] +/- 53ms 2024-04-26T20:41:08.341 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -741us[ -741us] +/- 65ms 2024-04-26T20:41:08.349 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:41:08.358 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:41:08.365 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-26T20:41:08.373 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 7 377 5 +1530us[+1479us] +/- 67ms 2024-04-26T20:41:08.380 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 2 -788us[ -788us] +/- 53ms 2024-04-26T20:41:08.386 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 8 -686us[ -736us] +/- 65ms 2024-04-26T20:41:08.474 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:41:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:41:08.547 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:41:08.553 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-26T20:41:08.560 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1579us[+1522us] +/- 67ms 2024-04-26T20:41:08.569 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 3 -725us[ -725us] +/- 53ms 2024-04-26T20:41:08.575 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 71 -655us[ -714us] +/- 64ms 2024-04-26T20:41:08.582 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:41:08.589 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T20:41:08.606 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T20:41:08.614 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T20:41:08.631 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:41:08.892 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:41:08.969 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:41:09.064 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:41:09.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:41:09.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:41:09.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:41:09.302 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:41:09.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:41:09.394 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T20:41:09.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T20:41:09.429 INFO:teuthology.task.internal:Duration was 3101.070514 seconds 2024-04-26T20:41:09.437 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T20:41:09.505 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T20:41:09.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:41:09.527 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:41:09.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:41:09.578 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:41:09.616 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:41:09.664 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:41:10.098 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T20:41:10.110 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-04-26T20:41:10.120 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:41:10.279 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-04-26T20:41:10.286 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:41:10.331 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-04-26T20:41:10.338 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:41:10.374 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T20:41:10.416 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:41:10.432 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:41:10.443 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:41:10.518 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T20:41:10.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:41:10.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:41:10.548 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:41:11.071 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T20:41:11.088 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T20:41:11.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:41:11.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:41:11.277 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:41:11.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T20:41:11.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:41:11.417 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:41:11.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:41:11.449 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-04-26T20:41:11.460 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-04-26T20:41:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-04-26T20:41:11.549 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:41:11.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:41:11.652 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:41:11.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:41:11.686 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:41:11.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:41:11.717 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T20:41:11.736 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T20:41:11.745 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi131 2024-04-26T20:41:11.782 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:41:11.937 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi148 2024-04-26T20:41:11.965 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:41:12.011 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675139/remote/smithi157 2024-04-26T20:41:12.021 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:41:12.066 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T20:41:12.074 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:41:12.091 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:41:12.101 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:41:12.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T20:41:12.194 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T20:41:12.203 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T20:41:12.222 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T20:41:12.231 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:41:12.247 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:41:12.259 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:41:12.272 INFO:teuthology.orchestra.run.smithi148.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:41 /home/ubuntu/cephtest 2024-04-26T20:41:12.282 INFO:teuthology.orchestra.run.smithi157.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:41 /home/ubuntu/cephtest 2024-04-26T20:41:12.347 INFO:teuthology.orchestra.run.smithi131.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:41 /home/ubuntu/cephtest 2024-04-26T20:41:12.407 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T20:41:12.427 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T20:41:12.496 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} duration: 3101.0705137252808 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T20:41:12.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:41:12.629 INFO:teuthology.run:pass